JP2007329702A - Sound-receiving device and voice-recognition device, and movable object mounted with them - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound receiving device equipped with a display unit for notifying the direction in which the existing source of received sound to the surrounding people. <P>SOLUTION: The sound receiving device comprises a microphone 20, which can be adjusted for a direction having high directionality (sound-receiving direction); a sound source direction detector 101 for detecting the direction, in which the source of sound received by the microphone 20 exists; and the display unit 10 which indicates the detected direction, in which the source of sound exists and the sound receiving direction of the microphone 20, in such a manner as to be visible from around a robot 1. The surrounding people can understand which source of sound is being received by the sound-receiving device. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a sound receiving device that receives sound emitted from a sound source.

A sound receiving device that can detect the propagation direction of sound received by a microphone is known. As one method for that purpose, a method of arranging a plurality of microphones on a straight line at equal intervals is known. For example, as shown in FIG. 11, a plurality of microphones 20a to 20f are arranged on a straight line L with an interval d.
If the sound source M exists at a position sufficiently away from the microphone group (for convenience of illustration, the sound source M is illustrated at a position close to the microphone group, but in reality, it is sufficiently far away. In FIG. 11, it may be said that the interval d is illustrated as being larger than the actual distance), and the sound emitted by the sound source M propagates as a substantially plane wave in the vicinity of the microphone group. For example, the sound reaching the microphone 20c has propagated along the path 610, and the sound reaching the microphone 20d has propagated along the path 612. Sound propagating as a plane wave has the same phase at the wavefront 616 and the wavefront 618. Accordingly, when the sound source M is in the direction of the angle θ when viewed from the microphones 20c and 20d, the sound that has propagated along the path 610 and reached the microphone 20c at a certain point in time reaches the point 620 along the path 612. It reaches the microphone 20d after further propagating by the length of d · cos θ. Therefore, between the adjacent microphones 20c and 20d, the sound reception time difference Δt is a value calculated by d · cos θ / λ. Here, λ is the speed of sound propagation. Therefore, the angle θ indicating the direction in which the sound source M exists can be calculated by detecting the sound reception time difference Δt. The angle θ is an angle with respect to the straight line L, and the straight line L is fixed to the sound receiving device. If the microphone pair 20c, 20d of FIG. 11 is used, the propagation direction of the received sound can be detected with reference to the sound receiving device. The above description is not limited to the microphone pair of the microphones 20c and 20d, and is valid for any microphone pair.

  Some sound receiving devices can switch the direction having good directivity (strong sound receiving sensitivity). As one of the methods, a method is known in which sound signals received by the microphones are overlapped after being delayed by a time determined from a direction in which good directivity is desired.

  As shown in FIG. 12, the sound signal received by the microphone 20b is delayed by d · cos θ / λ time from the sound signal received by the microphone 20a. The sound signal received by the microphone 20c is delayed by 2 · d · cos θ / λ time from the sound signal received by the microphone 20a. The sound signal received by the microphone 20d is delayed by 3 · d · cos θ / λ time from the sound signal received by the microphone 20a. The sound signal received by the microphone 20e is delayed by 4 · d · cos θ / λ time from the sound signal received by the microphone 20a. The sound signal received by the microphone 20f is delayed by 5 · d · cos θ / λ time from the sound signal received by the microphone 20a.

Therefore, the sound signal received by the microphone 20a is delayed by [5 · d · cos θ / λ] time, and the sound signal received by the microphone 20b is delayed by [4 · d · cos θ / λ] time, The sound signal received by the microphone 20c is delayed by [3 · d · cos θ / λ] time, the sound signal received by the microphone 20d is delayed by [2 · d · cos θ / λ] time, and the microphone 20e is delayed. If the sound signal received at the time is delayed by [d · cos θ / λ] time, and the sound signal received by the microphone 20f is not delayed, the phases of the sound signals are matched.
That is, the portion (the portion of the maximum amplitude value [A] of the sound signal) where the sound receiving volume of the sound signals a1, b1, c1, d1, e1, f1 of the sound emitted by the sound source M shown in FIG. When the delay processing is executed, they match.
When the sound signals delayed by the above processing are superimposed, a sound signal having a large intensity can be obtained for the sound propagating from the direction of the angle θ because the phases of the sound signals to be superimposed are the same.

For example, as shown in FIG. 11, it is assumed that the sound source N exists also in the front direction (angle θ = 90 degrees) of the microphones 20a to 20f. In this case, as shown in FIG. 12, the sound emitted by the sound source N reaches the microphones 20a to 20f simultaneously.
In this case, the sound signal received by the microphone 20a is delayed by [5 · d · cos θ / λ] time, and the sound signal received by the microphone 20b is delayed by [4 · d · cos θ / λ] time. The sound signal received by the microphone 20c is delayed by [3 · d · cos θ / λ] time, and the sound signal received by the microphone 20d is delayed by [2 · d · cos θ / λ] time. If the sound signal received by 20e is delayed by [d · cos θ / λ] time, and the sound signal received by the microphone 20f is not delayed, the phases of those sound signals are shifted.
That is, the portion where the received sound volume of the sound signals a2, b2, c2, d2, e2, f2 of the sound emitted by the sound source N shown in FIG. When the delay process is executed, it is shifted.
When the sound signals delayed by the above processing are superimposed, the sound signals with high strength cannot be obtained because the phases of the sound signals to be superimposed do not match.
That is, when the above delay processing is performed and then superimposed, a sound signal having a high intensity is obtained from the sound propagating from the angle θ direction, and a sound signal having a high intensity is obtained from the sound propagating from other directions. I can't. A sound receiving device having strong directivity in the direction of the angle θ is realized.

  By changing the value of the angle θ used for determining the delay time, the direction having strong directivity can be switched. If the delay time is determined by setting the angle θ to 90 degrees, a sound receiving device having strong directivity at an angle of 90 degrees can be obtained. In this case, the sound generated by the sound source N existing in the direction of 90 degrees is received with high sensitivity, while the sound generated by the sound source M existing at other angles is hardly received. Similarly, if the value of the angle θ used for determining the delay time is changed to an angle where the sound source M exists, the sound generated by the sound source M is received with high sensitivity, while the sound source N existing at other angles is received. Sounds that occur are hardly received.

  When the sound sources M and N are people and are simultaneously producing sound, it is difficult to receive them simultaneously and recognize them simultaneously. In this case, it is meaningful to switch the direction having strong directivity. If the directivity is matched with the direction in which the person N exists, the voice generated by the person M is hardly received, and the voice generated by the person N can be recognized. If the directivity is matched with the direction in which the person M exists, the voice generated by the person N is hardly received, and the voice generated by the person M can be recognized.

Using a technique that detects the propagation direction of sound received by a microphone and a technique that switches the direction with good directivity, it is possible to find the direction in which the sound source exists and direct the directivity to that direction. .
For example, if the technology for detecting the sound propagation direction knows that a sound source exists in the direction of 90 degrees and adjusts the directivity to 90 degrees, the sound generated by the person N is received intensively and recognized. It becomes possible. Knowing that there is a sound source in the θ direction and adjusting the directivity to θ, it is possible to receive the voice generated by the person M intensively and recognize the voice.

However, with this technique, the direction of directivity of the sound receiving device is not known from the outside. For example, if a technology for detecting the direction of sound propagation and a technology for switching the direction with good directivity are incorporated into a navigation device mounted on a vehicle, the driver's voice is received intensively when the driver speaks. When the passenger seat utters, the utterance at the passenger seat can be received intensively and recognized. However, when the driver seat and the passenger seat are uttered at the same time, it is not clear which voice is received intensively and recognized.
There is a need for a technique that allows a person existing in the vicinity to know the sound receiving direction (the direction having strong directivity) directed by the sound receiving device.

When the sound receiving device is mounted on a rotatable object such as a robot, the sound receiving direction to which the sound receiving device is directed can be made known to a person existing around by the orientation of the face of the robot.
Japanese Patent Application Laid-Open No. 2004-228561 describes a technique that allows a person around to know the direction of the sound receiving device having high directivity by matching the direction of the sound receiving device having high directivity with the direction of the face of the robot. Yes.

JP 2002-366191 A

  When the sound receiving device is mounted on a movable body such as a robot, a person with a high directivity can recognize the surrounding direction depending on the orientation of the face, but it does not move like a navigation device. In the case of a sound receiving device, it is impossible for a person existing in the vicinity to know a direction having high directivity.

  There is a problem even in the technique that indicates a direction having high directivity depending on the orientation of the face. For example, there is a case where it is desired to operate a switch group arranged on the switchboard while instructing the robot by voice. In this case, it is necessary for the robot to continue working with its face facing the switchboard, and the face cannot be directed in a direction with high directivity. In addition to the orientation of the face, there is a need for a technique for displaying whether the directivity of the sound receiving device mounted on the robot is directed toward the instructor or the other direction.

(Invention of Claim 1)
The sound receiving device of the present invention includes a microphone, sound source direction detecting means for detecting a propagation direction of sound received by the microphone with reference to the sound receiving device, and a direction detected by the sound source direction detecting means around the sound receiving device. Display means for displaying in a visible manner.
The sound source direction detection means determines the sound propagation direction when, for example, a sound having a volume equal to or higher than a threshold is received by a microphone. The sound source direction detecting means detects a sound propagation direction with reference to a reference direction fixed to the sound receiving device. The sound source direction detecting means determines the direction in which the sound source exists with reference to the sound receiving device. The sound source direction detection means may be configured by software or hardware. When there are two or more sound sources, the direction of presence of each sound source is detected.
The display means displays the direction of the sound source relative to the reference direction fixed to the sound receiving device. When there are two or more sound sources, the direction of each sound source is displayed.

By using the sound receiving device of the present invention, surrounding people can know the direction of the sound source that the sound receiving device is receiving. If the navigation device is uttering at the driver seat and the passenger seat at the same time, it is possible to know whether both voices are received or only one voice is received. In the latter case, it is possible to know which voice is being received.
When you give a voice to a robot that keeps its face facing a specific direction, you can know whether the robot has directionality to the instructor or other sound source. . Because the directivity is matched to the surrounding noise sources, it is possible to know if the voice of the instructor is not recognized, and know that measures such as reducing noise are effective. it can. Alternatively, it is possible to prevent a voice instruction from being continued to a robot whose directionality is not matched.
Also, when a plurality of people surround the robot and speak at the same time, it becomes clear who the robot is recognizing, and confusion can be avoided.

(Invention of Claim 2)
The display means may display the received sound volume of the microphone for each sound source.
In this case, the surrounding people can know the direction of the sound source detected by the sound receiving device and the sound receiving volume for each direction.

(Invention of Claim 3)
Sound source type discriminating means for discriminating the type of sound source of the received sound is added based on the frequency component of the sound received by the microphone, and the type of the sound source discriminated by the sound source type discriminating means is displayed on the display means. It is preferable.
The sound source type discriminating unit only needs to be able to discriminate between human voices and other sounds, and may be configured by software or hardware.
By using the sound receiving device of the present invention, surrounding people can know the direction of the sound source and the type of sound source recognized by the sound receiving device. For example, if you give a voice to the robot in an environment where the TV or radio is emitting sound, you can know whether the robot is receiving the sound of the TV or radio or only the real voice. .

(Invention of Claim 4)
A plurality of cameras and sound source distance calculation means may be added to the sound receiving device. The plurality of cameras capture an area where the microphone receives sound. The sound source distance calculation means calculates the distance between the sound receiving device and the sound source based on a group of images captured by a plurality of cameras. The sound source distance calculation means may be configured by software or hardware. In this case, the distance calculated by the sound source distance calculating means may be displayed together on the display means.
When the sound receiving device and the sound source are displayed as marks on the display means, the distance between the sound receiving device and the sound source may be displayed according to the length between the marks. Further, the distance between the sound receiving device and the sound source may be displayed as text such as “OOcm”.
By using the sound receiving device of the present invention, surrounding people can know the direction of the sound source recognized by the sound receiving device and the distance to the sound source, making it easier to grasp the sound receiving status of the sound receiving device. .

(Invention of Claim 5)
This invention can also implement | achieve a speech recognition apparatus using the any one of Claims 1-4. The voice recognition apparatus in this case includes the sound receiving apparatus according to any one of claims 1 to 4, a sound source direction fixing means, and a voice recognition means. The sound source direction fixing means fixes the sound receiving direction of the microphone in the direction in which the sound receiving volume is maximized when the sound source direction detecting means detects a plurality of directions.
The “sound receiving direction” of the microphone means a direction having strong directivity. There are a plurality of modes in the mode of “fixing the sound receiving direction”. When the directionality direction is changed by physically rotating the directional microphone, the sound receiving direction can be fixed by physically fixing the microphone. When the directivity is realized by processing the outputs of a plurality of stationary microphones, the sound receiving direction can be fixed by fixing the processing content.
In the prior art, when the person who is speaking cannot know the direction of the sound source recognized by the voice recognition device and cannot obtain the result according to the voice instruction, the cause can be known. It was difficult. According to this device, the sound receiving direction of the sound receiving device is displayed, and when the result according to the voice instruction cannot be obtained, the reason can be easily understood.

(Invention of Claim 6)
It is preferable that the display means also displays the sound receiving direction of the microphone fixed by the sound receiving direction fixing means.
The display means only needs to display at least the direction of sound reception of the microphone, and the direction of sound reception may continue to be displayed while the direction of sound reception is not fixed. For example, a display that changes with time may be displayed while the microphone receives sound while switching the sound receiving direction with time.
Thereby, the user can know the sound receiving direction having a strong directivity. If the displayed sound receiving direction does not face itself, it can be recognized that the user's own sound is not received. It is easy to take necessary measures such as eliminating noise sources that exist in the microphone receiving direction.

(Invention of Claim 7)
The present invention is particularly effective when mounted on a movable body. The movable body here can rotate at least around the vertical axis, and is mounted with the sound receiving device according to any one of claims 1 to 4. The movable body is provided with display means for displaying an angle formed by a reference direction fixed to the movable body and a direction in which the sound source exists.
If the movable body of the present invention is used, the surrounding people can know the direction of the sound source recognized by the sound receiving device mounted on the movable body, and can easily grasp the sound reception status of the sound receiving device. .
Even if the movable body rotates around the vertical axis, the display means can display the angle formed by the reference direction and the direction in which the sound source exists, that is, the relative direction of the sound source with respect to the movable body.

(Invention of Claim 8)
A rotating mechanism for rotating the movable body around the vertical axis with respect to the floor, and a means for fixing the sound receiving direction of the microphone in a direction in which the sound receiving volume becomes maximum when the sound source direction detecting means detects a plurality of directions. It is preferable that voice recognition means for recognizing sound received by a microphone whose sound receiving direction is fixed and control means for controlling the rotation mechanism based on information recognized by the voice recognition means are added.
The user can know whether or not the sound receiving device mounted on the movable body is directed in the direction in which the sound receiving device is located. If the directivity is directed to a sound source other than the user, it can be known. The user can know a measure for keeping the microphone directivity at the user.

(Invention of Claim 9)
The display means included in the movable body preferably displays the sound receiving direction of the microphone fixed by the sound receiving direction fixing means.
Thereby, the user can know the sound receiving direction of the microphone. If the displayed sound receiving direction does not face itself, it is possible to know that its own sound is not received. It becomes easy to deal with a failure in voice input, such as eliminating a noise source present in the sound receiving direction of the microphone.

  If the sound receiving device of the present invention is used, surrounding people can know the direction of the sound source that the sound receiving device is receiving, and can easily understand the sound receiving status of the sound receiving device.

The main features of the embodiments described below are listed.
(First Form) The sound source direction detecting means recognizes the direction in which the volume value received by the microphone while switching the sound receiving direction shows the maximum value as the direction in which the sound source exists.
(2nd form) The means to determine the sound source direction in which the received sound volume value shows the maximum value is added from the sound source direction group recognized by the sound source direction detecting means. The microphone fixes the sound receiving direction to the determined sound source direction. The display device also displays the fixed sound receiving direction.
(Third embodiment) Means for recognizing the language content of the sound received by the microphone from the sound source direction detected by the sound source direction detecting means is added, and the display means also combines the language contents recognized by the language content recognizing means. indicate.
(Fourth Mode) Means for recognizing the language content of the sound received by the microphone from the sound source direction detected by the sound source direction detection means is added, and the display means can recognize the language content by the language content recognition means. The result indicating whether or not it has been completed is also displayed.
(5th form) The means to produce | generate the dialog sound corresponding to the language content recognized by the language content recognition means is added, and a display means displays the dialog sound produced | generated by the dialog production | generation means collectively.
(6th form) The means to generate | occur | produce the dialog sound corresponding to the language content recognized by the language content recognition means is added, and a display means shows whether the dialog generation means was able to generate | occur | produce the dialog sound The result is also displayed.

(First embodiment)
A first embodiment of an interactive robot incorporating a voice input device will be described with reference to FIGS. On the front side of the interactive robot, there is provided a display that can be visually recognized by people located around the robot. The direction of the sound source received by the robot is displayed on the display. The display shows the reference direction fixed to the robot and the direction in which the sound source exists. By looking at it, a person located around the robot can know the direction of the sound source as seen from the robot. In the first embodiment, in addition to the direction in which the sound source is seen from the robot, the type of sound source, the volume level at which sound is received, and the direction at which sound is received by the sound input device (directivity is matched) Direction) is displayed.
FIG. 1 shows an outline of a sound source and a robot that exist in an area where the robot can interact. FIG. 2 is a block diagram showing the configuration of the robot. FIG. 3 is a diagram for explaining the process of specifying the direction in which the sound source exists. FIG. 4 is a flowchart of a program executed by the robot control means. 5 to 8 show examples of screens displayed on the display.

As shown in FIG. 1, sound sources such as humans 2 to 4, a television 5, and a mobile phone 6 exist around a humanoid and interactive robot 1. When the right side of the robot 1 is 0 degrees and the right side in the opposite direction is 180 degrees, the front surface of the robot 1 having a boundary between 0 degrees and 180 degrees is the available angle range that the robot 1 can handle. Here, the human 2 is talking to the robot 1 "Where is the toilet?" Humans 3 and 4 are talking loudly. The TV 5 emits a large volume. The mobile phone 6 emits a ring tone.
A display 10 is provided on the front surface of the robot 1. Above the display 10, there is provided a microphone array 20 (an example of a “microphone” recited in the claims) including six microphones 20 a to 20 f. The microphones 20a to 20f are arranged on a straight line at an interval d on the front surface of the robot 1. A speaker 30 is provided at a portion corresponding to the mouth of the robot 1. The robot 1 is provided with wheel-shaped moving means 40 and can rotate around the vertical axis on the floor surface or move on the floor surface. In addition, the robot 1 has a built-in controller 300. The operation of the robot 1 is controlled by the controller 300.
The robot 1 recognizes and responds to the language content spoken to the interlocutor. When the interrogator asks "Where is the toilet?", The robot 1 can move to the toilet location and provide guidance.

As shown in FIG. 2, the controller 300 includes a control unit 100, a voice input interface 21, a display output interface 11, a voice output interface 31, and a moving unit drive unit 41.
The control means 100 is connected to the microphone array 20 via the audio input interface 21, connected to the display 10 via the display output interface 11, and connected to the speaker 30 via the audio output interface 31. And connected to the moving means 40 via the moving means driving unit 41.
The control unit 100 includes a sound source direction detection unit 101, a sound source type determination unit 102, a directivity control unit 103, a language content recognition unit 104, and a dialogue generation processing unit 105.
The audio input interface 21 is connected to the input side of the sound source direction detection unit 101, the input side of the sound source type determination unit 102, and the input side of the language content recognition unit 104. The output side of the sound source direction detection unit 101 is connected to the input side of the directivity control unit 103 and the display output interface 11. The output side of the sound source type discrimination unit 102 is connected to the input side of the directivity control unit 103 and the display output interface 11. The output side of the directivity control unit 103 is connected to the display output interface 11 and the input side of the language content recognition unit 104. The output side of the language content recognition unit 104 is connected to the input side of the dialogue generation processing unit 105 and the moving means driving unit 41. The output side of the dialogue generation processing unit 105 is connected to the audio output interface 31.

  The sound source direction detection unit 101 detects a sound source direction indicating in which direction the sound source is located with respect to the robot 1 based on the sound signal obtained by the microphone array 20. The conventional method described with reference to FIG. 11 can be used to detect the sound source direction. The sound source direction can be detected by detecting the sound reception time difference Δt from any pair of microphones.

When there is only one sound source, the direction of the sound source can be detected from the sound reception time difference Δt between any pair of microphones. However, when there are a plurality of sound sources, the sound reception time difference Δt can be specified. It becomes difficult.
Therefore, the direction of the sound source is detected using the directivity described with reference to FIGS. The directivity θ is temporally changed between 0 degrees and 180 degrees, and the received sound volume is continuously detected. FIG. 3 illustrates the change in the received sound volume thus obtained. The horizontal axis represents the magnitude of the directivity θ, and the vertical axis represents the volume level received by the directivity. In the case of the graph illustrated in FIG. 3, the maximum values x1 to x4 (dB) of the sound volume are observed in the directions of 5 degrees, 45 degrees, 90 degrees, and 135 degrees when viewed from the robot 1. Illustrated. If the sound volume level threshold that the sound source direction detection unit 101 can recognize as a sound source is A (dB), x1, x3, x4 (dB)> A (dB), and x2 <A (dB). Therefore, the robot 1 detects that some sound source is present in the directions of 5 degrees, 90 degrees, and 135 degrees as viewed from the robot 1. In the case of FIG. 3, the volume of the mobile phone 6 existing at an angle of 45 degrees is low, and the robot 1 does not recognize it as a sound source.

  The sound source type determination unit 102 specifies the type of the sound source based on the frequency component of the received sound signal. The type of sound source is specified by pattern matching using an HMM (Hidden Markov Model) or the like. The sound source type discriminating unit 102 specifies whether the sound source is a human voice, a television voice, or a sound from a CD.

  The directivity control unit 103 is based on the information indicating the direction of each sound source specified by the sound source direction detection unit 101 and the information indicating the type of each sound source specified by the sound source type determination unit 102. Adjust. Here, a sound source that exists in the direction specified by the sound source direction detection unit 101 and is of the human voice and has the highest volume is selected from the sound sources, and the directivity is matched to the sound source. When the sound source direction detection unit 101 obtains the angle-volume level characteristic illustrated in FIG. 3, the directivity is adjusted to the direction of 5 degrees where the human 3 exists.

The language content recognition unit 104 discriminates the language content of the sound emitted by the sound source with the directivity. For example, it recognizes the language content “Did you watch a TV program a while ago?”, Which is a voice uttered by human 3. Since the method for recognizing the language content of speech is a general technique, detailed description thereof is omitted.
The dialogue generation processing unit 105 assembles a sound signal indicating the content of the voice (answer) output from the speaker 30 in accordance with the language content recognized by the language content recognition unit 104. For example, a sound signal that outputs a sound of “I am sorry but I have not seen it” is assembled.
The sound output interface 31 causes the speaker 30 to utter sound by the sound signal input from the dialogue generation processing unit 105.
The display output interface 11 receives the sound source direction input from the sound source direction detection unit 101, the sound source type input from the sound source type determination unit 102, and the directivity input from the directivity control unit 103. Information on the sound direction is output to the display 10. The display 10 displays these pieces of information.
The moving means driving unit 41 drives the moving means 40 in accordance with the voice content recognized by the language content recognition unit 104. If the speech recognized by the language content recognition unit 104 is “Where is the toilet?”, The robot 1 moves to the toilet by the moving means 40 and guides the questioner to the toilet.

  Next, a procedure in which the controller 300 controls the robot 1 will be described with reference to FIG. The flowchart of FIG. 4 shows the processing contents of the program executed by the controller 300. This program is stored in a storage means (not shown) such as a ROM provided in the control means 100 of the controller 300, and a control device such as a CPU provided in the control means 100 (particularly in FIG. (Not shown)) and read out as appropriate.

  In step S10, the controller 300 receives sound while switching the directivity (sound receiving direction) of the microphone array 20 in time (scanning) within an angle range of 0 to 180 degrees. At this time, a search screen 200 as shown in FIG. The search screen 200 displays the available angle range of the robot 1. A position display 201 indicating the position of the robot 1 is displayed in the available angle range. Also, auxiliary lines 202 to 206 indicating 45 degrees, 90 degrees, and 135 degrees directions are displayed, with the lateral direction of the robot 1 being the 0 degree direction and the opposite lateral direction being the 180 degree direction. Further, a search display 207 indicating that a direction having good directivity is being scanned is displayed. The search display 207 is centering on the position display 201 of the robot 1, and the arrow moves clockwise or counterclockwise in the range from the auxiliary line 202 to the auxiliary line 206, and the sound source included in this range is being searched. It shows that.

  The direction in which the sound source exists is detected by the above search. The direction in which the sound source exists is an angle fixed to the robot 1 (for example, a direction of 0 degrees or 180 degrees is fixed to the robot 1, and the direction does not change unless the robot 1 rotates. If it rotates, it rotates with it). If the robot 1 rotates, the direction of the sound source rotates even if the position of the sound source does not move. By executing step S10 of FIG. 4, the sound source direction detection unit 101 shown in FIG. 2 is realized.

  In step S <b> 12 of FIG. 4, the controller 300 specifies the type of sound source in the sound source direction based on the sound signal input from the microphone array 20. When the type of each sound source is specified, as shown in FIG. 6, the display unit 10 displays the direction of the sound source, the type of the sound source, and the received sound volume. On the screen 210 shown in FIG. 6, a human mark 218 indicating the result of detecting that the human 3 is present is displayed in a direction of 5 degrees when viewed from the robot 1. In addition, a human mark 217 indicating the result of detecting that the human 2 is present is displayed in a direction of 90 degrees as viewed from the robot 1. Further, a mark 216 indicating the result of detecting that the television is present is displayed in a direction of 135 degrees as viewed from the robot 1. Each mark is displayed in a size proportional to the volume. In the screen 210 shown in FIG. 8, the volume emitted by the human 3 indicated by the mark 218 is larger than the volume generated by the human 2 indicated by the mark 217. The mark corresponding to the type of the sound source is stored in the storage means (not shown) of the controller 300. When the type of the sound source is determined, the mark corresponding to the type is read from the storage means. Are output to the display 10 via the display output interface 11 and displayed on each screen. By executing step S12 of FIG. 4, the sound source type determination unit 102 shown in FIG. 2 is realized.

In step S14 of FIG. 4, the controller 300 selects a conversation person and matches the sound receiving direction (directivity) with the person who is the conversation person. On the screen 220 shown in FIG. 7, a directional antenna 221 indicating a direction having good directivity is displayed. A state in which the directional antenna 221 faces the direction of 5 degrees and directivity is directed toward the human 3 (refer to FIG. 1 together) existing in the direction of 5 degrees is illustrated. A screen 230 shown in FIG. 8 represents a state in which the directional antenna 231 faces the person 2 positioned in the 90-degree direction.
The controller 300 concentrates the sound signals output from the microphones 20a to 20f by delaying the time signals calculated from the direction of directivity to which the sound signals are directed, and then superimposing the sound signals propagating from that direction. To do. The language content recognizing unit recognizes the content of the language indicated by the voice based on the sound signal received while the directivity is matched to the human. Since the sound propagating from the direction other than the direction in which the directivity is matched is hardly received, the content of the language can be recognized from the voice signal with little noise. By executing step S14 in FIG. 4, the directivity control unit 103 in FIG. 2 is realized.

  Next, the controller 300 proceeds to the process of step S18 in FIG. In step S18, the language content indicated by the voice is recognized. At this time, in order to use the audio signal received with the directivity fixed in step S14, the language content is recognized by noise propagated from the sound source existing in a direction other than the sound source direction for recognizing the language content. Is less likely to be disturbed. By executing step S18, the language content recognition unit 104 of FIG. 2 is realized.

  Next, the controller 300 proceeds to the process of step S20 in FIG. In step S <b> 20, the controller 300 determines the language content to be output from the speaker 30 corresponding to the language content recognized in step S <b> 18, and outputs information for uttering the determined sound to the audio output interface 31. Further, the controller 300 controls the moving means driving unit 41 in accordance with the language content recognized in step S18. By executing step S20 in FIG. 4, the dialog generation processing unit 105 in FIG. 2 is realized.

  Next, the controller 300 proceeds to the process of step S22 in FIG. In step S22, the controller 300 determines whether or not the power of the robot 1 is turned off. If the power is off (Yes in step S22), the process is terminated. If the power is not turned off (No in step S22), the process returns to step S10 and the subsequent processes are repeated.

In the above description, a case has been described in which the human 3 and the human 4 (see FIG. 1) are speaking loudly and the robot 1 is aligned with the human 3 positioned in the direction of 5 degrees. In this case, the human 3 is not interacting with the robot 1, but the human 4 is present behind the robot 1, and therefore recognizes that the robot 1 is speaking to itself and the human 3 exists. The directivity is matched to the direction. Therefore, in response to the human 3 speaking to the human 4 saying “Did you watch the TV program a while ago?”, The voice of “I'm sorry, I haven't watched” is output.
However, it is assumed that the person who desires the dialogue with the robot 1 is actually the person 2 and the person 2 asks the robot 1 “Where is the toilet?”. However, since the robot 1 has replied “I'm sorry, I have not seen it”, the dialogue has not been established. Therefore, when the human 2 looks at the display 10 on the front surface of the robot 1, the screen 220 shown in FIG. 7 is displayed, and the human 3 is loud, so the robot 1 has directivity in the direction of the human 3. You can see that Even if the robot 1 makes a reply that does not correspond to the content of its own utterance, it is less likely to feel uncomfortable because the cause is known. It can be recognized that the robot 1 is not out of order. In order for the robot 1 to recognize his / her utterance, it is possible to take measures such as having the human 3 speak with a low voice.

The series of processes described above are repeatedly executed. When steps S10 and S12 in FIG. 4 are repeated, the controller 300 scans again the available angle range of the robot 1 from 0 degrees to 180 degrees, and again detects the direction of the sound source, the type of the sound source, and the received sound volume. As a result, the display screen of the display 10 changes as shown in FIG. 8, for example. From the screen 230, it can be seen that the volume of the human 3 is low and the directional antenna 231 is directed in the direction of the human 2, so that the robot 1 is aligned with the direction of the human 2. .
When the directivity is fixed in the direction in which the sound volume is maximum, it is preferable to fix the directivity in the sound source direction in which the average sound volume in the past predetermined period is maximum. If the directivity is fixed by the average sound volume, it becomes possible to keep the directivity matched to one speaker.

  According to the robot 1 of the present embodiment, the human 2 determines from the display content of the display 10 the direction of the sound source recognized by the robot 1, the type of sound source recognized by the robot 1, and the robot 1 receiving sound. It is possible to grasp the sound volume. The human 2 can clearly know that the sound receiving direction (directivity) of the microphone is directed in a direction other than his / her own, and which sound source influences it. Moreover, even if the sound receiving direction of the microphone is directed to the user's direction at the present time, it is possible to predict which sound source may become a noise source and block his / her voice input in the future. For example, another person exists in the range of available angles of the robot 1 and the sound volume is low and the microphone receiving direction is not directed, but the robot 1 recognizes the direction of the sound source, When the volume increases, the robot 1 can change the sound receiving direction of the microphone, so that it can be known that the voice input of the human 2 is likely to be rejected. Therefore, the human 2 can clearly know which sound source should be excluded in order to ensure that the sound receiving direction of the microphone of the robot 1 is directed to himself / herself.

(Second embodiment)
In the display of this embodiment, the distance between the robot and the sound source is displayed in addition to the direction of the sound source, the type of the sound source, and the received sound volume. This will be described with reference to FIGS.
FIG. 9 is a block diagram showing the configuration of the robot 1a. FIG. 10 shows an example of a screen displayed on the display.

As shown in FIG. 9, the robot 1a is provided with a stereo camera 50 in addition to the components of the robot 1 shown in FIG. The controller 300a built in the robot 1a is provided with an image input interface 51 in addition to the components of the controller 300 of the robot 1. In addition to the components of the control unit 100 of the robot 1, the sound source distance calculation unit 106 is provided in the control unit 100a of the controller 300a.
The stereo camera 50 is connected to the input side of the sound source distance calculation unit 106 of the control means 100a via the image input interface 51 of the controller 300a. The output side of the sound source distance recognition unit 106 is connected to the input side of the directivity control unit 103 and the display output interface 11. Since the other components of the controller 300a and their connections are the same as those of the controller 300, description thereof will be omitted.

  The stereo camera 50 is composed of two cameras, which are arranged on the right eye and the left eye of the robot 1a. The images picked up by the two cameras are images of the object picked up with a minute angle difference. Each captured image is converted into an image signal and output to the image input interface 51. The sound source distance calculation unit 106 connected to the image input interface 51 calculates the distance between the robot 1a and the sound source based on the input image signal. For example, paying attention to a specific point, the “deviation” when the images are taken by one camera and the other camera is calculated. Then, the distance between the robot 1a and the sound source is calculated from the difference in angle when the object is viewed from each camera causing the “displacement” and the interval (fixed) between the two cameras. Since the method for calculating the distance in this way is a public technique, a detailed description is omitted.

  In the directivity control unit 103a, the direction of each sound source specified by the sound source direction detection unit 101, the information indicating the type of each sound source specified by the sound source type determination unit 102, the information indicating the received sound volume, Based on the information indicating the distance between the robot recognized by the sound source distance calculation unit 106 and each sound source, the direction (sound receiving direction) to match the directivity is specified. Here, a sound source whose type is human voice is selected from the sound source existing directions recognized by the sound source direction detection unit 101, and a sound source closest to the robot 1a is selected. As a result, the person 2 talking to the robot 1a from among the persons 2 to 4 is specified as a dialog person. Then, the directivity is adjusted to the direction of the person 2. If the distance is the same, the sound source with the higher received sound volume is selected, and the directivity is adjusted to the direction of the selected sound source.

The display output interface 11 receives information on the direction of the sound source, the type of the sound source, the received sound volume, the distance between the robot and the sound source, and the direction in which the directivity of the sound receiving device is directed. Is displayed on the display 10. For example, the screen 240 illustrated in FIG. 10 is displayed. The screen 240 is displayed so that a person existing in each sound source can recognize the magnitude of the distance between the robot and the sound source. The outermost semicircle 241 shown in FIG. 12 represents that the distance from the robot is 1.5 m, the semicircle 242 represents that the distance from the robot is 1.0 m, and the semicircle 243 is This indicates that the distance from the robot is 0.5 m.
On the screen 240, a human mark indicating the presence of the human 3 (see FIG. 1) is displayed at a position 5 degrees from the robot 1 and a distance of 1.5 m (d1 = 1.5). In addition, a human mark indicating the presence of the human 2 is displayed at a position of 90 degrees from the robot 1 and a distance of 1 m (d2 = 1). A mark indicating the presence of the television is displayed at a position of 135 m from the robot 1 and at a position of 1.5 m (d3 = 1.5). Since the type of the sound source is a human and the directivity is matched to the direction of the human 2 that is the closest to the robot 1a, the directional antenna is displayed toward the human 2.

  According to the robot 1a of the present embodiment, since the distance between the robot and each sound source can be displayed on the display device 10, a person who looks at the display device can easily grasp the position of the sound source.

The robot 1a may be configured to extract features of a human face based on an image captured by the camera 50 and to distinguish and respond to a person. In this case, the camera 50 is also connected to the input side of the sound source type determination unit 102 via the image input interface 51. The output side of the sound source type determination unit 102 is also connected to the input side of the dialogue generation unit 105. The storage means (not shown) of the control means 100a stores a human personal information database corresponding to facial features. Then, the sound source type discriminating unit 102 extracts the facial features of the captured image, and reads the personal information stored in the personal information database corresponding to the extracted features. The dialog generation unit 105 generates an appropriate dialog based on the read personal information. For example, when a human being speaks to the robot 1a as "Good morning", the robot recognizes who the robot is and responds with "Good morning, Mr. XX".
According to this, even when the robot 1a has the same question, the robot 1a can take a fine response that differs depending on the speaker.

  In the second embodiment, the case where the sound source output direction is recognized by processing the sound signal output from the audio input interface 21 by the sound source direction detection unit 101 has been described, but the image signal output from the image input interface 51 is described. The direction of the sound source may be recognized based on the above. In this case, the image input interface 51 and the input side of the sound source direction detection unit 101 are connected. The sound source direction detection unit 101 specifies the presence direction of an object serving as a sound source from an image.

  In the first and second embodiments, the case where the sound source type is determined by processing the sound signal output from the audio input interface 21 by the sound source type determining unit 102 has been described, but the sound source type is output from the image input interface 51. The type of sound source may be determined from the image signal. In this case, the image input interface 51 and the input side of the sound source type determination unit 101 are connected. Then, the sound source type discriminating unit 102 specifies the type of the object serving as the sound source from the image.

  In the second embodiment, the case where the distance to the sound source is calculated by processing the image signal input from the stereo camera 50 by the sound source distance calculating unit 106 has been described, but the calculation is performed from the sound signal received by the microphone 20. Also good. In this case, the input side of the sound source distance calculation unit 106 is connected to the audio input interface 21. Further, the camera 50 and the image input interface 51 are not necessary.

  In the first and second embodiments, the case where the voice input device of the present invention is applied to an interactive robot has been described. However, the present invention may be applied to other devices. For example, the present invention may be applied to a car navigation system (hereinafter abbreviated as “car navigation”). Car navigation recognizes the sound of a sound source in a limited space in the car. When a plurality of people are on board, it is easily affected by the sound of the noise source. For this reason, the car navigation system easily causes erroneous recognition of the language content and malfunction. If the present invention is applied, the sound source direction recognized by the car navigation system can be displayed on the display device, and the dialog person can grasp this. As a result, when a conversation person speaks to the car navigation system and the voice is not input properly (for example, when there is an inappropriate response), the noise source that is not input properly Can be understood and dealt with. For example, if the sound source position is displayed on the rear seat on the display and people sitting in the rear seat are talking loudly, the voice of the rear seat that is the source of noise is reduced. Can be dealt with.

The display unit of the first and second embodiments may be configured such that the language content recognition unit 104 displays the voice recognition result (for example, “Where is the toilet?”) Produced by the human 2 in text. . According to this, since the conversation person can visually confirm the actually recognized voice, the comfort level of the conversation person is improved.
In addition, the language content recognition unit 104 may be configured to display the result (recognition OK, recognition NG, etc.) indicating whether or not the content of the voice uttered by the person 2 has been recognized. According to this, since the conversation person can visually confirm whether or not the voice is actually recognized, the comfort level of the conversation person is improved.
In addition, the dialog generation processing unit 105 displays a result (dialog generation OK, dialog generation NG, etc.) indicating whether or not the content of the voice (reply) output from the speaker 30 has been determined corresponding to the recognized voice content. You may comprise so that it may display. According to this, since the conversation person can visually confirm whether or not a reply is generated, the comfort level of the conversation person is improved.
The display of the first embodiment and the second embodiment is configured to display the voice (for example, “I will guide you to the toilet”) output from the speaker 30 determined by the dialog generation processing unit 105 as text. Also good. According to this, since the conversation person can confirm a reply by visual observation, the comfort level of the conversation person improves.

In the first and second embodiments, the case where the sound receiving direction of the microphone is switched by processing the outputs of a plurality of stationary microphones has been described. However, the microphone is rotated by physically rotating the microphone. A microphone that switches the sound receiving direction may be used.
The sound source direction detection unit 101, the sound source type determination unit 102, the directivity control unit 103, the language content recognition unit 104, the dialogue generation processing unit 105, and the sound source distance calculation unit 106 may be configured by hardware. It may be configured by software. When configured by software, each of the units 101 to 106 includes each function of the program (sound source direction detection function, sound source type discrimination function, directivity control function, language content recognition processing function, dialog generation processing function, sound source distance calculation. This corresponds to each step of executing the function.

Specific examples of the present invention have been described in detail above, but these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes of the specific examples illustrated above.
In addition, the technical elements described in the present specification or the drawings exhibit technical usefulness alone or in various combinations, and are not limited to the combinations described in the claims at the time of filing. In addition, the technology illustrated in the present specification or the drawings achieves a plurality of objects at the same time, and has technical utility by achieving one of the objects.

An outline of a sound source and a robot that exist in an interactive area of the robot 1 is shown. 1 is a block diagram showing a configuration of a robot 1. FIG. It is a figure explaining the process in which the robot recognizes the direction of a sound source. It is a flowchart figure of the program which the control means of the robot 1 performs. The example of the screen displayed on the indicator 10 is shown. The example of the screen displayed on the indicator 10 is shown. The example of the screen displayed on the indicator 10 is shown. The example of the screen displayed on the indicator 10 is shown. It is a block diagram which shows the structure of the robot 1a. The example of the screen displayed on the indicator 10 is shown. It is a figure explaining the process which detects in which direction the sound source of the sound received with the microphone exists with respect to a sound receiving apparatus. The sound signal of the sound which each microphone 20a-20f received is shown.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,1a Robot 2,3,4 Human 5 Television 6 Mobile phone 10 Display 20 Microphone 30 Speaker 40 Moving means 50 Camera 100, 100a Control means 300, 300a Controller

Claims (9)

A sound receiving device,
With a microphone,
Sound source direction detecting means for detecting the propagation direction of the sound received by the microphone with reference to the sound receiving device;
A sound receiving device comprising: display means for displaying the direction detected by the sound source direction detecting means so as to be visible from the periphery of the sound receiving device.   2. The sound receiving device according to claim 1, wherein the display means displays the sound receiving volume of the microphone for each sound source. A sound source type discriminating means for discriminating the type of the sound source of the received sound based on the frequency component of the sound received by the microphone is added,
The sound receiving device according to claim 1 or 2, wherein the display means also displays the type of the sound source determined by the sound source type determining means. A plurality of cameras that capture the range in which the microphone receives sound;
Sound source distance calculation means for calculating the distance between the sound receiving device and the sound source is added based on the image group captured by the plurality of cameras,
The sound receiving apparatus according to claim 1, wherein the display means displays the distance calculated by the sound source distance calculating means together. A sound receiving device according to any one of claims 1 to 4,
Means for fixing the sound receiving direction of the microphone in the direction in which the sound receiving volume is maximized when the sound source direction detecting means detects a plurality of directions;
A speech recognition apparatus comprising speech recognition means for recognizing a sound received by a microphone having a fixed sound receiving direction.   6. The speech recognition apparatus according to claim 5, wherein the display means also displays the sound receiving direction of the microphone fixed by the sound receiving direction fixing means. It is a movable body that can rotate at least around the vertical axis,
The sound receiving device according to claim 1 is mounted,
A movable body, wherein the display means displays an angle formed by a reference direction fixed to the movable body and a direction in which the sound source exists. A rotation mechanism that rotates a movable body around a vertical axis with respect to the floor surface;
Means for fixing the sound receiving direction of the microphone in the direction in which the sound receiving volume is maximized when the sound source direction detecting means detects a plurality of directions;
Speech recognition means for recognizing sound received by a microphone having a fixed sound receiving direction;
8. The movable body according to claim 7, further comprising control means for controlling the rotation mechanism based on information recognized by the voice recognition means.   9. The movable body according to claim 8, wherein the display means also displays the sound receiving direction of the microphone fixed by the sound receiving direction fixing means.

Images