WO2001057853A1 - Dispositif auditif de robot - Google Patents

Dispositif auditif de robot Download PDF

Info

Publication number
WO2001057853A1
WO2001057853A1 PCT/JP2001/000648 JP0100648W WO0157853A1 WO 2001057853 A1 WO2001057853 A1 WO 2001057853A1 JP 0100648 W JP0100648 W JP 0100648W WO 0157853 A1 WO0157853 A1 WO 0157853A1
Authority
WO
WIPO (PCT)
Prior art keywords
noise
robot
internal
microphone
external
Prior art date
Application number
PCT/JP2001/000648
Other languages
English (en)
Japanese (ja)
Inventor
Kazuhiro Nakadai
Hiroshi Okuno
Hiroaki Kitano
Tatsuya Matsui
Original Assignee
Japan Science And Technology Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2000022678A external-priority patent/JP2001215990A/ja
Priority claimed from JP2000022679A external-priority patent/JP2001215991A/ja
Application filed by Japan Science And Technology Corporation filed Critical Japan Science And Technology Corporation
Publication of WO2001057853A1 publication Critical patent/WO2001057853A1/fr

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/003—Controls for manipulators by means of an audio-responsive input
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02163—Only one microphone
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165—Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal

Definitions

  • the present invention relates to a hearing device for a robot, particularly a humanoid or animal robot. Skill
  • Active perception refers to supporting perception devices that are responsible for perceptions such as robot vision and robot hearing so that they follow the target to be perceived.For example, the head is driven by a drive mechanism. It is.
  • At least the camera which is a sensory device, holds the optical axis of the camera toward the target by controlling the attitude of the drive mechanism, and automatically focuses on the target, zooms in, zooms out, etc. Is performed so that the camera images the target.
  • active vision at least the camera, which is a sensory device, holds the optical axis of the camera toward the target by controlling the attitude of the drive mechanism, and automatically focuses on the target, zooms in, zooms out, etc. Is performed so that the camera images the target.
  • Various studies have been conducted on active vision.
  • the microphone which is a sensory device, maintains its directivity toward the target by controlling the attitude of the drive mechanism, and sounds from the target are collected by the microphone.
  • the disadvantage of active hearing is that the microphone picks up the operating sound of the drive mechanism while the drive mechanism is operating, so that relatively loud noise is mixed into the sound from the target. It may not be possible to recognize the sound from the camera.
  • An object of the present invention is to solve the above-described drawbacks in the conventional technology, and to collect sound from an external target without being affected by noise generated inside a robot such as a drive mechanism, and to actively generate the sound.
  • An object of the present invention is to provide a hearing device of a robot, particularly a humanoid or animal robot, which can perform perception. Disclosure of the invention
  • a robot provided with a noise source inside has a soundproof exterior covering at least a part of a mouth bot, and an exterior provided outside the exterior.
  • the sound signal of the external microphone is operated to cancel the noise of the sound signal from the external microphone.
  • a human or animal robot having a noise source such as a drive mechanism therein is at least a robot.
  • a processing circuit for canceling the sound signal from the external microphone and the sound signal from the internal microphone is at least a robot.
  • the external microphone mainly collects sound from an external target
  • the internal microphone mainly collects noise from a noise generating source such as a drive mechanism inside the robot.
  • the noise signal from the noise source inside the robot is mixed in the acoustic signal collected by the external microphone, and the mixed noise signal is collected by the internal microphone by arithmetic processing in the processing circuit. The noise is significantly reduced by canceling with the noise signal.
  • the noise from the noise generation source such as the drive mechanism inside the robot can be easily and significantly reduced by the arithmetic processing in the processing circuit.
  • the S / N ratio of the sound signal from the external microphone is greatly improved, so that active perception can be further improved.
  • the processing circuit previously stores noise generated by a drive mechanism inside the robot, and performs arithmetic processing on a voice signal from an external microphone and a voice signal from an internal microphone. At this time, noise cancellation processing is performed by referring to the stored noise.
  • the noise cancellation processing can be performed more accurately and more quickly because each noise is accurately recorded in advance.
  • the processing circuit previously stores a noise feature component extracted from noise caused by a driving mechanism inside the robot, and outputs a voice signal from an external microphone and a noise signal from the internal microphone.
  • noise cancellation processing is performed by referring to the stored noise feature components.
  • the storage capacity can be reduced by storing only the noise feature portion.
  • the robot hearing device is preferably configured such that, when the processing circuit performs arithmetic processing on a voice signal from an external microphone and a voice signal from an internal microphone while learning noise due to a driving mechanism inside the robot, Noise cancellation processing is performed by referring to the noise that has been performed.
  • the internal microphone is disposed inside the exterior so as to be movable at least in a lateral direction.
  • the internal microphones are moved in the horizontal direction, the front-rear direction, and / or the vertical direction so as to more appropriately collect noise from a noise source inside the robot, and are paired with each internal microphone.
  • the distance to the corresponding external microphone can be appropriately adjusted, and the noise can be canceled in the most optimal state from the acoustic signal collected by the external microphone.
  • the internal microphone is slidably supported by a support member extending in a lateral direction inside the exterior.
  • the support member includes a slot extending in a lateral direction, and the internal microphone is fixed to a movable member slidably supported along the slot of the support member. Have been.
  • the movable member is screwed to the support member by a fixing screw inserted into the slot.
  • the internal microphone is further disposed so as to be movable in the front-rear direction.
  • the internal microphone is slidably supported on a first support member extending in the lateral direction inside the exterior, and the first support member is provided on the exterior. It is supported slidably with respect to a second support member extending in the front-rear direction on the inside.
  • the internal microphone is further disposed so as to be vertically movable.
  • FIG. 1 is a front view showing the appearance of a humanoid mouthboat incorporating the first embodiment of the robot hearing device according to the present invention.
  • FIG. 2 is a side view of the humanoid robot of FIG.
  • FIG. 3 is a schematic enlarged view showing the configuration of the head in the humanoid robot of FIG.
  • FIG. 4 is a block diagram showing an electrical configuration of the robot hearing device in the humanoid robot and the sort shown in FIG.
  • FIG. 5 is a schematic diagram showing a configuration of a processing circuit in the robot hearing device of FIG.
  • FIG. 6 is a schematic front view showing a mounting structure of an internal microphone in the humanoid robot of FIG.
  • FIG. 7 is an enlarged front view showing a main part of the mounting structure of the internal microphone in FIG.
  • FIG. 8 is an enlarged side view showing a main part of the mounting structure of the internal microphone in FIG.
  • FIG. 9 is an enlarged perspective view showing a main part of the mounting structure of the internal microphone in FIG.
  • FIG. 10 is a schematic diagram showing a configuration of a processing circuit in a second embodiment of the robot hearing device according to the present invention.
  • FIG. 11 is a schematic diagram showing a configuration of a processing circuit in a third embodiment of the robot hearing device according to the present invention.
  • FIG. 12 is a schematic diagram showing a configuration of a processing circuit in a fourth embodiment of the robot hearing device according to the present invention.
  • FIG. 13 is a schematic diagram showing the configuration of a processing circuit in a fifth embodiment of the robot hearing device according to the present invention.
  • FIG. 14 is a schematic diagram showing a configuration of a processing circuit in a sixth embodiment of the robot hearing device according to the present invention.
  • FIG. 15 is a schematic diagram showing a configuration of a processing circuit in a seventh embodiment of the robot hearing device according to the present invention.
  • FIG. 16 is a schematic diagram showing the configuration of a processing circuit in the eighth embodiment of the robot hearing device according to the present invention.
  • FIG. 17 is a schematic diagram showing a configuration of a processing circuit in a ninth embodiment of the robot hearing device according to the present invention.
  • FIG. 18 shows a processing circuit in the tenth embodiment of the robot hearing device according to the present invention.
  • FIG. 3 is a schematic diagram showing the configuration of FIG.
  • FIG. 19 is a schematic diagram showing the configuration of a processing circuit in the eleventh embodiment of the robot hearing device according to the present invention.
  • FIG. 20 is an enlarged perspective view showing a main part of a modified example of the mounting structure of the internal microphone in FIG. BEST MODE FOR CARRYING OUT THE INVENTION
  • 1 to 5 show the overall configuration of an experimental humanoid robot provided with one embodiment of the robot hearing device according to the present invention.
  • the humanoid robot 10 is configured as a robot with 4 DOF (degree of freedom).
  • the humanoid robot 10 includes a base 11, a body 12 supported rotatably around one axis (vertical axis) on the base 11, and a three-axis direction ( And a head 13 that is swingably supported around a vertical axis, a horizontal axis in the horizontal direction, and a horizontal axis in the front-rear direction.
  • the base 11 may be fixedly arranged, and may be operable as a leg of a robot. Further, the base 11 may be placed on a movable cart or the like.
  • the body portion 12 is rotatably supported about a vertical axis with respect to the base 11 as shown by an arrow A in FIG. 1, and is driven to rotate by driving means (not shown). Furthermore, the body 12 is entirely covered with a soundproof exterior.
  • the head 13 is supported by a connecting member 13 a with respect to the body portion 12.
  • a connecting member 13 a Around the horizontal axis in the front-rear direction with respect to the connecting member 13 a, an arrow in FIG. It is supported so as to be able to swing as shown by B, and as shown by arrow C in FIG. 2 around the horizontal axis in the horizontal direction.
  • the connecting member 13a is swingably supported on the body 12 around a horizontal axis in the front-rear direction as shown by an arrow D in FIG.
  • the rotation drive in the directions of arrows A, B, C, and D is performed by a drive (not shown). This is done by moving means.
  • the head 13 is entirely covered with a soundproof exterior 14, as shown in FIG. Further, the head 13 has a camera 15 on the front side as a visual device for robot vision, and a pair of external microphones 16 (16a, 16b) on both sides as hearing devices for robot hearing. Have.
  • the head 13 is provided with a pair of internal microphones 17 (17a, 17b) which are arranged inside the exterior 14 so as to be separated from each other.
  • the exterior 14 is made of a sound absorbing synthetic resin such as a urethane resin.
  • the exterior 14 is configured so that the inside of the head 13 is sound-insulated by almost completely sealing the inside of the head 13.
  • the exterior of the body 12 is similarly made of a sound-absorbing synthetic resin.
  • the camera 15 has a known configuration.
  • a commercially available camera having 3 DOFs (degrees of freedom) of so-called pan, tilt, and zoom can be used.
  • the external microphones 16 are mounted so as to have directivity forward on the side surfaces of the head 13 respectively.
  • each of the left and right external microphones 16a, 16b of the external microphone 16 is, as shown in FIG. 1 and FIG.
  • the sound is shielded by appropriate means so as not to pick up the sound inside the exterior 14.
  • the external microphones 16a and 16b are configured as so-called binaural microphones.
  • the exterior 14 may be formed in the shape of a human outer ear.
  • the internal microphones 17 are respectively positioned inside the exterior 14 in the vicinity of the external microphones 16a and 16b described above, and in the case shown in FIG. It is arranged to be movable in the direction. Note that the internal microphone 17 may be provided at an arbitrary position inside the exterior 14.
  • FIG. 4 shows an electrical configuration for acoustic processing including the external microphone 16 and the internal microphone 17.
  • the sound signals S OL, S OR, SIL and S IR from the external microphones 16 a and 16 b and the internal microphones 17 a and 17 b are respectively processed.
  • the sound signal SIR from the internal microphone 17b is input to the processing circuit 19.
  • the processing circuit 20 includes a filter circuit 21 and a subtraction circuit 22.
  • the filter circuit 21 uses a filter circuit of a known configuration, for example, an inverse filter or an adaptive filter, and receives an audio signal SI (SIL, SIR) from the internal microphone 17 (17a, 17b). You.
  • SI SIL, SIR
  • the subtraction circuit 22 subtracts the signal from the filter circuit 21 from the sound signal SO (SOL, SOR) from the external microphones 16a, 16b.
  • the filter circuit 21 performs a filter process on the sound signal S from the internal microphone 17 to obtain a noise component NO corresponding to the noise component NO included in the sound signal SO from the external microphone 16. Reproduce the signal NI. Then, the subtraction circuit 22 subtracts the noise signal NI from the audio signal SO from the external microphone 16, thereby generating noise sources such as drive mechanisms inside the robot mixed into the audio signal SO from the external microphone 16. Almost completely eliminates the noise signal N 0 from the, and outputs a true acoustic signal SL or SR.
  • the internal microphone 17 is specifically attached to the camera 15 as shown in FIGS. 6 to 9, the internal microphones 17a and 17b are fixed to mounting brackets 40 and 41 as movable members, respectively.
  • the mounting brackets 40, 41 are formed in a substantially U-shape with their upper sides open, and the upper ends thereof are close to each other with the tightening screws 40a, 41a, thereby holding the internal microphones 17a, 17b. And hold it fixed. Further, the mounting brackets 40 and 41 are respectively slidably mounted on the support members 42 and 43 in a lateral direction as described later.
  • the support members 42 and 43 are fixed to the upper surface of the camera 15 by brackets 44, respectively.
  • the support members 42 and 43 are brackets
  • the bracket 44 is formed integrally with the bracket 44, and the bracket 44 is screwed to a mounting base 15a on which the camera 15 is mounted.
  • each support member 42, 43 has a slot 42a, 43a extending in the lateral direction, respectively.
  • Each of the mounting brackets 40 and 41 has a screw hole (not shown) for receiving a fixing screw 45 or 46 passed through each of the slots 42a and 43a. By being tightened, 45 and 46 are fixedly held to the supporting members 42 and 43.
  • the humanoid robot 10 is configured as described above, and the sound from the target to be collected by the external microphones 16a and 16b is collected and noise canceled as follows. .
  • the external microphones 16a and 16b mainly collect external sounds from the target and output acoustic signals S ⁇ L and SOR, respectively.
  • the external microphones 16a and 16b also collect noise from the inside of the robot, but the exterior 14 itself seals the inside of the head 13 and the external microphones 16a and 16b Noise is suppressed to a relatively low level due to the sound insulation to the inside.
  • the internal microphones 17a and 17b mainly collect noise from the inside of the robot, for example, noise from noise generating sources such as the above-described operating sounds of the driving mechanisms and the operating sounds of the cooling fans.
  • the internal microphones 17a and 17b also collect sound from outside, but since the exterior 14 hermetically seals the inside, the level of mixed noise is kept relatively low.
  • the sound signals SOL, S ⁇ R from the external microphones 16a, 16b and the sound signals SIL, SIR from the internal microphones 17a, 17b collected in this manner are sent to the processing circuits 18 and 19, respectively. Is entered.
  • the processing circuits 18 and 19 respectively output the sound signals S ⁇ L and S ⁇ R from the external microphones 16 a and 16 b and the sound signals SIL and SIR from the internal microphones 17 a and 17 b by the filter circuit 20.
  • the subtraction circuit 22 subtracts the noise signals from the noise sources inside the robot from the acoustic signals S ⁇ L and SOR from the external microphones 16a and 16b.
  • the processing circuits 18 and 19 output the true sound signals SL and SR from which noise has been removed to the outside. Power.
  • a control unit (not shown) performs sound recognition to perform active perception.
  • the internal microphones 17a and 17b are fixed and held by the mounting brackets 40 and 41, respectively, and the mounting brackets 40 and 41 are fixed and held by the support members 42 and 43, respectively. Fixedly held. Further, the mounting brackets 40, 41 are screwed to the support members 42, 43 by fixed screws 45, 46 inserted into slots 42a, 43a of the support members 42, 43, respectively. At that time, the mounting brackets 40, 41 and the internal microphones 17a, 17b are mounted on the fixing members 45, 46 while the fixing screws 45, 46 are loosened, and the slots 42a, 43a of the supporting members 42, 43 respectively. The inside position is adjusted as appropriate by sliding inside. After adjustment, the fixing screws 45 and 46 are tightened.
  • the internal microphones 17a and 17b can be moved and adjusted laterally inside the exterior 14 of the head 13 and set to desired positions. That is, the internal microphones 17a and 17b are adjusted to positions at appropriate distances with respect to the corresponding external microphones 16a and 16b, so that the noise cancellation by the processing circuits 18 and 19 described above can be minimized. It can be done effectively.
  • the processing circuits 18 and 19 control the external microphones 16 and 19 with the internal microphones 17 a and 17 b adjusted appropriately in the lateral direction.
  • Noise cancellation is performed based on the acoustic signals SIL and SIR from the internal microphones 17a and 17b from the acoustic signals SOL and SOR from the external microphones 16a and 16a.
  • a sound signal with a good S / N ratio can be obtained while aiming at the direction of the directivity of 16b, and the sound from each sound source can be separated, and more accurate sound signal recognition can be performed.
  • FIG. 10 shows a configuration of a processing circuit in a second embodiment of the robot hearing device according to the present invention.
  • a processing circuit 23 includes a noise sound storage database 24 as a storage unit, an external microphone noise prediction module 25, and a subtraction circuit 22.
  • the noise sound storage data 24 is used to store the sound signals SO and SI collected by the external microphone 16 and the internal microphone 17 for each noise generated from a single sound source in advance.
  • the data is stored as noise data, and the corresponding external noise and internal noise data are labeled and associated with each other.
  • the external microphone noise prediction module 25 compares the input acoustic signal SI from the internal microphone 17 with the internal noise data stored in the noise sound storage database 24, and determines the noise included in the acoustic signal SI. Analyze the type of This matching is performed by combining the internal noise data and predicting the nearest internal noise SI. Then, the external microphone noise prediction module 25 predicts an external noise NI combining the corresponding external noise data based on the analyzed noise type.
  • the subtraction circuit 22 subtracts the external noise NI predicted by the external microphone prediction module 25 from the acoustic signal S O from the external microphone 16.
  • the external microphone noise prediction module 25 stores the internal noise stored in the noise sound storage device 24.
  • the noise signal NI corresponding to the noise component NO included in the acoustic signal SO from the external microphone 16 is predicted with reference to the noise and external noise data.
  • the subtraction circuit 22 subtracts the noise signal NI from the acoustic signal SO from the external microphone 16 to generate noise such as drive mechanisms inside the robot mixed with the acoustic signal SO from the external microphone 16.
  • the noise signal N 0 from the source is almost completely removed, and the true sound signal SL or SR is output.
  • FIG. 11 shows a configuration of a processing circuit in a third embodiment of the robot hearing device according to the present invention.
  • the processing circuit 26 has the same configuration as that of the processing circuit 23 shown in FIG. 10, but has a data format of the internal noise and the external noise stored in the noise sound storage database 24. Is an abstraction (normalization) of the waveform or power spectrum in the time axis, power axis, or frequency axis direction.
  • External microphone noise prediction module 2 5 force Input acoustic signal SI from internal microphone 17
  • the configuration is different in that after performing shift processing in the time, frequency, and power directions, respectively, the matching is performed.
  • the processing circuit 26 having such a configuration, after the external microphone noise prediction module 25 normalizes the sound signal SI from the internal microphone 17, the noise is calculated based on the normalized sound signal SI.
  • the noise signal NI corresponding to the noise component N0 included in the acoustic signal SO from the external microphone 16 is predicted and subtracted.
  • the circuit 22 subtracts the noise signal NI from the acoustic signal SO from the external microphone 16.
  • noise signal N 0 from the noise source such as each drive mechanism inside the robot mixed into the acoustic signal SO from the external microphone 16 is almost completely removed, and the true sound signal SL or SR is obtained.
  • noise data of a plurality of patterns can be combined into one abstract data, so that the data base can be reduced in size.
  • FIG. 12 shows a configuration of a processing circuit in a fourth embodiment of the robot hearing device according to the present invention.
  • the processing circuit 27 has substantially the same configuration as the processing circuit 23 shown in FIG. 10, and is different in the following points. That is, the external microphone noise prediction module 25 compares the input acoustic signal SI from the internal microphone 17 with the internal noise data stored in the noise sound storage database 24, and includes it in the acoustic signal SI. Analyzes the type of noise that is generated. At the time of this collation, the external microphone noise module 25 refers to the motion information 40 of the robot 10, that is, the motor operation information indicating how much the motor of the driving mechanism of the robot 10 has been driven and how much. Analyze the type of noise included in SI.
  • the processing circuit 27 based on the acoustic signal SI from the internal microphone 17, the external microphone noise prediction module 25 and the internal noise and noise stored in the noise noise storage database 24 are stored.
  • the noise signal NI corresponding to the noise component N0 included in the sound signal SO from the external microphone 16 is predicted with reference to the external noise data and the robot operation information 40.
  • FIG. 13 shows the configuration of a processing circuit in a fifth embodiment of the robot hearing device according to the present invention.
  • the processing circuit 28 has the same configuration as the processing circuit 27 shown in FIG. 12, but like the processing circuit 26 shown in FIG.
  • the data format of the internal noise and external noise stored in the memory 24 is an abstraction (normalization) of the waveform or power spectrum in the time axis, power axis or frequency axis direction.
  • 25 has a different configuration in that it performs a shift process on the input acoustic signal SI from the internal microphone 17 in the time, frequency, and power directions and then performs matching.
  • the processing circuit 28 having such a configuration, after the external microphone noise prediction module 25 normalizes the acoustic signal SI from the internal microphone 17, a noise sound is generated based on the normalized acoustic signal SI.
  • the signal NI is predicted, and the subtraction circuit 22 subtracts the noise signal NI from the acoustic signal SO from the external microphone 16.
  • the noise signal N 0 from the noise source such as each drive mechanism inside the robot mixed in the sound signal SO from the external microphone 16 is almost completely removed, and the true sound signal SL or SR is output. .
  • the database can be reduced in size, and the external microphone noise module 25 can use any of the modes when matching. Since it is possible to determine whether or not the evening operation noise is occurring, it is possible to reduce the calculation amount and the calculation time required for the verification.
  • FIG. 14 shows the configuration of a processing circuit in a sixth embodiment of the robot hearing device according to the present invention.
  • the processing circuit 29 has the same configuration as the processing circuit 23 shown in FIG. 10, but instead of the noise sound storage database 24, a noise sound model database 30 is used. It has.
  • the noise sound model database 30 contains the characteristics of the noise extracted from the sound signals SO and SI collected by the external microphone 16 and the internal microphone 17 for each noise generated from a single sound source in advance, for example, Features such as Attack, FM, AM, Harmonics, etc. extracted from the acoustic signal SI are quantified, and the features are stored as a noise model using a distribution expression such as the mean and variance.
  • the noise model stored in the noise sound model database 30 reduces the number of features by expressing each feature in a rectangular coordinate space using principal component analysis or the like for each noise. The amount of calculation at the time of matching may be reduced.
  • the external microphone noise prediction module 25 generates the internal noise model stored in the noise sound model database 30 based on the acoustic signal SI from the internal microphone 17.
  • the closest acoustic signal SI is predicted by combining the internal noise models, or conversely, the acoustic signal SI is analyzed to extract features, and it is determined which internal noise model this feature is closer to. For example, calculation is performed using a least squares method, discriminant analysis, or the like, and the type of noise included in the acoustic signal SO from the external microphone 16 is predicted.
  • the external microphone noise prediction module 25 predicts the noise signal NI by combining the external noise model of the noise sound model database 30 based on the predicted type of noise, and the external microphone 1 Subtract the predicted noise signal NI from the acoustic signal SO from 6.
  • the noise signal N ⁇ from the noise generation source such as each drive mechanism inside the robot mixed into the audio signal S 0 from the external microphone 16 is almost completely removed, and the true audio signal SL or SR is obtained. Output.
  • the noise sound model database 30 stores only the noise feature components extracted from the sound signals SO and SI, the data amount is small, and the entire database can be downsized. The calculation amount and calculation time by the external microphone noise prediction module 25 are reduced.
  • FIG. 15 shows a processing circuit of the seventh embodiment of the robot hearing device according to the present invention. 1 shows the configuration.
  • the processing circuit 31 has substantially the same configuration as the processing circuit 29 shown in FIG. 14, and differs in the following points.
  • the external microphone noise prediction module 25 checks the input sound signal SI from the internal microphone 17 against the internal noise model stored in the noise sound model database 30 to determine the type of noise included in the sound signal SI. Is analyzed.
  • the external microphone noise module 25 refers to the motion information 40 of the robot 10, that is, the motor motion information indicating which motor of which driving mechanism of the robot 10 has been driven and how much. Analyzes the type of noise included in SI. According to the processing circuit 31 having such a configuration, the external microphone noise prediction module 25 stores the internal noise model stored in the noise sound model database 30 based on the acoustic signal SI from the internal microphone 17. Then, referring to the robot motion information 40, the closest acoustic signal SI is predicted by combining the internal noise models, or conversely, the acoustic signal SI is analyzed to extract a feature amount.
  • the external microphone noise prediction module 25 predicts the noise signal NI by combining the noise model data and the external noise model of the evening 30 based on the type of the predicted noise.
  • the predicted noise signal NI is subtracted from the acoustic signal S 0 from the external microphone 16.
  • the noise signal N0 from the noise source such as each drive mechanism inside the robot mixed into the audio signal SO from the external microphone 16 is almost completely removed, and the true audio signal SL or SR is output. I do.
  • FIG. 16 shows the configuration of the processing circuit in the eighth embodiment of the robot hearing device according to the present invention.
  • the processing circuit 32 has the same configuration as the processing circuit 23 shown in FIG. 10, but differs in the following points.
  • the noise sound storage database 24 stores the sound signal S 0 for the sound signal SO collected by the external microphone 16 as the external noise data, and stores the sound collected by the internal microphone 17.
  • the signal SI in the same manner as the noise sound model data base 30 of the processing circuit 11, the characteristics of the noise extracted from the acoustic signal SI for each noise generated in advance from a single sound source, for example, the acoustic signal The features of Attack, FM, AM, Harmonics, etc. extracted from SI are quantified, and the feature values are stored as internal noise models using distribution expressions such as mean value and variance value. Data and the internal noise model are labeled and corresponded.
  • the sound signal SO may be stored as it is, as in the processing circuit 23, and after normalizing the sound signal SO as in the processing circuit 26, It may be stored as data.
  • the external microphone noise prediction module 25 stores the internal noise model stored in the noise sound storage data base 24 based on the acoustic signal SI from the internal microphone 17.
  • the closest acoustic signal SI is predicted by combining the internal noise models, or, conversely, the acoustic signal SI is analyzed to extract features, and it is determined which internal noise model this feature is closer to. For example, calculation is performed using the least squares method, discriminant analysis, or the like, and the type of noise included in the acoustic signal SO from the external microphone 16 is predicted.
  • the external microphone noise prediction module 25 predicts the noise signal NI by combining the external noise data of the noise sound database 24 based on the predicted noise type, and the subtraction circuit 22 predicts the external microphone noise. 16 Subtract the predicted noise signal NI from the acoustic signal SO from 6. As a result, the noise signal N0 from the noise source such as each drive mechanism inside the robot mixed into the acoustic signal SO from the external microphone 16 is almost completely removed, and the true acoustic signal SL or SR is obtained. Output.
  • the noise sound storage database 24 stores the acoustic signal S Since only the characteristic components of the noise extracted from I are stored, the amount of data can be reduced, the entire database can be reduced in size, and the amount of calculation and the calculation time by the external microphone noise prediction module 25 Is reduced.
  • FIG. 17 shows a configuration of a processing circuit in a ninth embodiment of the robot hearing device according to the present invention.
  • the processing circuit 33 has substantially the same configuration as the processing circuit 32 shown in FIG. 16, and differs in the following points.
  • the external microphone noise prediction module 25 compares the input sound signal SI from the internal microphone 17 with the internal noise model stored in the noise sound storage database 24, and determines the type of noise contained in the sound signal SI. Is analyzed. At the time of this collation, the external microphone noise module 25 refers to the operation information 40 of the robot 10, that is, the motor operation information of which drive mechanism of the robot 10 has been driven and how much. Analyzes the type of noise contained in the acoustic signal SI.
  • the external microphone noise prediction module 25 includes the internal noise model stored in the noise sound storage database 24 based on the acoustic signal SI from the internal microphone 17, and the robot Referring to the motion information 40, the nearest acoustic signal SI is predicted by combining the internal noise models, or conversely, the acoustic signal SI is analyzed to extract features, and this feature is applied to any internal noise model. The closeness is calculated using, for example, the least squares method or discriminant analysis, and the type of noise included in the sound signal SO from the external microphone 16 is predicted.
  • the external microphone noise prediction module 25 predicts the noise signal NI based on the predicted noise type by combining the external noise data of the noise sound storage database 24 with the subtraction circuit 22. Then, the predicted noise signal NI is subtracted from the acoustic signal S ⁇ from the external microphone 16. In this manner, the noise signal N 0 from the noise source such as each drive mechanism inside the robot mixed into the acoustic signal S 0 from the external microphone 16 is almost completely removed, and the true acoustic signal SL or Output SR. In this case, since the noise sound storage data base 24 stores only the characteristic components of the noise extracted from the acoustic signal SI with respect to the internal noise, the data amount is small, and the entire database is downsized. And external microphone noise The calculation amount and the calculation time by the measurement module 25 are reduced. Furthermore, since the external microphone noise module 25 can recognize which mode of operation noise is occurring at the time of matching, it is possible to reduce the calculation amount and calculation time required for matching. .
  • FIG. 18 shows the configuration of a processing circuit in a tenth embodiment of the robot hearing device according to the present invention.
  • the processing circuit 34 has substantially the same configuration as the processing circuit 29 shown in FIG. 14, and includes an external microphone noise module 35 instead of the external microphone noise prediction module 25. It has a different configuration in that
  • the external microphone noise prediction module 35 like the external microphone noise prediction module 25, compares the input acoustic signal SI from the internal microphone 17 with the internal noise model stored in the noise sound model database 30. Then, the type of noise included in the acoustic signal SI is analyzed. Further, when the type of the analyzed noise is significantly different from the internal noise model stored in the noise sound model database 30, the external microphone noise prediction module 35 sets the noise as an internal noise model of the unknown noise sound. Stored in the sound model database 30. As a result, unknown noise sounds are learned.
  • the external microphone noise prediction module 35 generates the internal noise model stored in the noise sound model database 30 based on the acoustic signal SI from the internal microphone 17.
  • the closest acoustic signal SI is predicted by combining the internal noise models, or conversely, the acoustic signal SI is analyzed to extract features, and it is determined which internal noise model this feature is closer to. For example, calculation is performed using the least squares method, discriminant analysis, or the like, and the type of noise included in the acoustic signal SO from the external microphone 16 is predicted.
  • the external microphone noise prediction module 35 predicts the noise signal NI by combining the external noise model of the noise sound model database 30 based on the type of the predicted noise, and the external microphone 16
  • the predicted noise signal NI is subtracted from the acoustic signal SO. In this way, the noise signal N ⁇ from the noise generation source of each drive mechanism inside the robot mixed with the sound signal SO from the external microphone 16 is almost completely removed, and the true sound signal SL Or output SR.
  • the external microphone noise module 35 determines the type of the noise as the internal noise of the unknown noise sound.
  • the new internal noise model to be learned is a so-called unsupervised learning, it is preferably determined to be the same noise sound by using a self-organizing method such as a neural network. Noise models in the same category. In this case, it is possible to remove not only a noise sound caused by the motor of the drive mechanism but also a noise sound caused by other factors, for example, a noise sound such as a cover rubbing sound.
  • the noise sound database 24 shown in FIG. 10, FIG. 11, FIG. 16, or FIG. 17 may be used instead of the noise sound model database 30.
  • FIG. 19 shows the configuration of a processing circuit in the eleventh embodiment of the robot hearing device according to the present invention.
  • the processing circuit 36 has substantially the same configuration as the processing circuit 34 shown in FIG. 18 and is different in the following points.
  • the external microphone noise prediction module 35 compares the input acoustic signal SI from the internal microphone 17 with the internal noise model stored in the noise sound model database 30 to determine the noise contained in the acoustic signal SI. Analyze the type of At the time of this collation, the external microphone noise module 35 refers to the motion information 40 of the robot 10, that is, the motor motion information indicating which driving mechanism of which driving mechanism of the robot 10 has been driven and how much. Analyzes the type of noise contained in the acoustic signal SI. Further, when the type of the analyzed noise is significantly different from the noise model stored in the noise sound model database 30, the external microphone noise prediction module 35 calculates the internal noise model of the unknown noise sound. Then, it is stored in the noise sound model database 30 in association with the robot operation information 40. As a result, an unknown noise sound is learned.
  • the external microphone noise prediction module 3 5 is a combination of the internal noise model stored in the noise sound model database 30 based on the acoustic signal SI from the internal microphone 17 and the robot operation information 40, and Predict the signal SI, or conversely, analyze the acoustic signal SI to extract features and calculate which internal noise model this feature is close to using, for example, the least squares method or discriminant analysis.
  • the type of noise included in the acoustic signal S 0 from the external microphone 16 is predicted.
  • the external microphone noise prediction module 35 predicts the noise signal NI by combining the external noise model of the noise sound model database 30 based on the predicted noise type, and the subtraction circuit 22 outputs the acoustic signal NI from the external microphone 16.
  • the noise signal NI predicted from SO.
  • the noise signal N 0 from the noise source such as each drive mechanism inside the robot mixed into the audio signal S 0 from the external microphone 16 is almost completely removed, and the true audio signal SL or SR is output. I do.
  • the external microphone noise prediction module 35 can determine which motor operation noise has occurred during the collation, L, so that the amount and time required for the collation can be reduced. In addition to learning a new internal noise model, it is possible to respond to changes in noise sound due to changes in the environment surrounding the robot.
  • FIG. 20 shows a modification of the mounting structure of the internal microphones 17a and 17b described above.
  • supporting members 42 and 43 for supporting the internal microphones 17a and 17b movably in the lateral direction are further fixed to the upper surface of the camera 15 by a suitable means (not shown). It is slidably attached to members 47 and 48 in the front-rear direction.
  • the second support members 47, 48 have slots 47a, 48a extending in the front-rear direction, respectively.
  • the vertically extending bent portions 42b, 42c (43b, 43c) formed at both ends of the support members 42, 43 force the fixing screws 49, which are inserted into these slots 47a, 48a. It has a screw hole (not shown) for receiving 50, and is fixed and held to the second support members 47, 48 by tightening these fixing screws 49, 50.
  • the internal microphones 17a and 17b are fixedly held by the mounting brackets 40 and 41, respectively.
  • the mounting brackets 40, 41 are fixedly held by the supporting members 42, 43 and the second supporting members 47, 48 so as to be fixedly held to the camera 15.
  • the mounting brackets 40, 41 are respectively screwed to the support members 42, 43 by fixing screws 45, 46 inserted into the slots 42a, 43a of the support members 42, 43, respectively.
  • 42, 43 are screwed to the second support members 47, 48 by fixing screws 49, 50 inserted into the slots 47a, 48a of the second support members 47, 48, respectively.
  • the mounting brackets 40, 41 and the internal microphones 17a, 17b are attached to the fixing members 45, 46 while the fixing screws 45, 46 are loosened. Slide in a to adjust the horizontal position appropriately, and after adjustment, tighten the fixing screws 45 and 46.
  • the fixing members 49, 50 are inserted into the slots 47a, 48a of the second supporting members 47, 48 while the fixing screws 49, 50 are loosened. After sliding, the position in the front-rear direction is adjusted appropriately, and the fixing screws 49 and 50 are tightened after the adjustment.
  • the internal microphones 17a and 17b can be moved and adjusted in the lateral and longitudinal directions inside the exterior 14 of the head 13, and can be set at desired positions. That is, the internal microphones 17a and 17b are adjusted to positions at appropriate distances with respect to the corresponding external microphones 16a and 16b, so that the noise cancellation by the processing circuits 18 and 19 described above is performed. Can be performed more effectively.
  • the humanoid robot 10 is configured to have 4DOF (degree of freedom).
  • the present invention is not limited to this, and the humanoid robot 10 may be configured to perform any operation. It is also possible to incorporate a bot hearing device.
  • the case where the robot hearing device according to the present invention is incorporated in the humanoid mouth boat 10 has been described.
  • the present invention is not limited to this, and various animal-type robots such as dog-type robots and other types of robots Obviously, it is possible to incorporate it into
  • the internal microphone 17 is constituted by a pair of microphones 17a and 17b, but may be a single microphone, and may be a plurality of microphones provided for each noise source. Microphone.
  • the external microphone 16 may be a single microphone, or may be composed of three or more microphones.
  • the internal microphones 17a and 17b are configured to be movable and adjustable in the horizontal direction along the slots 42a and 43a of the support members 42 and 43.
  • the present invention is not limited to this, and it may be configured so that the movement can be adjusted in the horizontal direction by other means.
  • the mounting brackets 40 and 41 may be configured to be movable and adjustable in the lateral direction with respect to the support members 42 and 43 and the force camera 15 by the driving means.
  • the internal microphones 17a and 17b can be adjusted in the lateral direction in the assembled state without removing the exterior 14 from the head 13 of the humanoid robot 10. Noise cancellation can be performed more effectively and more accurately.
  • the internal microphones 17a and 17b are supported so as to be movable in the horizontal direction or in the horizontal direction and in the front-back direction. It may be supported as possible.
  • the internal microphones 17a and 17b are supported by a triaxially movable mechanism having a known configuration.
  • the external microphone mainly collects sound from an external target
  • the internal microphone mainly collects noise from a noise generating source such as a drive mechanism inside the robot.
  • the noise signal from the noise source inside the robot is mixed in the acoustic signal collected by the external microphone, and this mixed noise signal is the noise collected by the internal microphone by the arithmetic processing in the processing circuit. The signal is canceled and significantly reduced.
  • the sound signal from the external microphone is significantly reduced by the noise signal from the noise generation source such as the drive mechanism inside the robot, so that the S / N ratio is greatly improved and the active perception is further improved. Can be performed.
  • the internal microphone is disposed inside the exterior so as to be movable in the horizontal direction, the internal microphone can be moved in the horizontal direction so that noise from a noise source inside the robot is better collected. Move the corresponding microphone to be paired with each internal microphone The distance to the external microphone can be adjusted appropriately, and noise can be better canceled from the acoustic signal collected by the external microphone.
  • the robot hearing device according to the present invention is extremely useful as a hearing system for active hearing in robots, especially human or animal robots.

Abstract

L'invention se rapporte à un dispositif auditif (10) de robot, notamment à un dispositif de robot ayant la forme d'un être humain ou d'un animal, qui est conçu pour pouvoir activement recueillir des sons à partir d'une cible externe sans être influencé par les bruits internes générés, par exemple, par un mécanisme d'entraînement. Ce dispositif comporte un boîtier insonorisé (14) recouvrant au moins en partie la tête (13) du robot, une paire de microphones externes (16) disposés au niveau des oreilles à l'extérieur du boîtier de manière à recueillir principalement des sons externes, une paire de microphones internes (17) disposés à l'intérieur du boîtier de manière à recueillir principalement des sons internes, et un circuit de traitement (18, 19) qui annule les bruits issus des sources de bruits internes dans les signaux sonores en provenance des microphones externes en fonction des signaux sonores reçus des microphones externes et internes. Le circuit de traitement supprime les bruits dans les signaux sonores provenant des microphones externes par manipulation des signaux sonores issus des microphones externes ainsi que des signaux sonores issus des microphones internes.
PCT/JP2001/000648 2000-01-31 2001-01-31 Dispositif auditif de robot WO2001057853A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2000-022678 2000-01-31
JP2000-022679 2000-01-31
JP2000022678A JP2001215990A (ja) 2000-01-31 2000-01-31 ロボット聴覚装置
JP2000022679A JP2001215991A (ja) 2000-01-31 2000-01-31 ロボット聴覚装置

Publications (1)

Publication Number Publication Date
WO2001057853A1 true WO2001057853A1 (fr) 2001-08-09

Family

ID=26584533

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2001/000648 WO2001057853A1 (fr) 2000-01-31 2001-01-31 Dispositif auditif de robot

Country Status (1)

Country Link
WO (1) WO2001057853A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105278759A (zh) * 2014-07-18 2016-01-27 深圳市大疆创新科技有限公司 一种基于飞行器的图像投影方法、装置及飞行器
CN108682428A (zh) * 2018-08-27 2018-10-19 珠海市微半导体有限公司 机器人语音控制系统和机器人对语音信号的处理方法
CN110051289A (zh) * 2019-04-03 2019-07-26 北京石头世纪科技股份有限公司 机器人语音控制方法、装置、机器人和介质
US10597169B2 (en) 2014-07-18 2020-03-24 SZ DJI Technology Co., Ltd. Method of aerial vehicle-based image projection, device and aerial vehicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
OKUNO,H.G. ET AL: "Research Issues of Humanoid Audition", JAPANESE SOCIETY FOR ARTIFICIAL INTELLIGENCE (TECHNICAL REPORT),(PROCEEDINGS OF THE SEVENTH MEETING OF SPECIAL INTEREST GROEP ON AI CHALLENGES),SIG-CHALLENGE-9907-10, 2 November 1999 (1999-11-02), pages 61 - 65, XP002937883 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105278759A (zh) * 2014-07-18 2016-01-27 深圳市大疆创新科技有限公司 一种基于飞行器的图像投影方法、装置及飞行器
US10597169B2 (en) 2014-07-18 2020-03-24 SZ DJI Technology Co., Ltd. Method of aerial vehicle-based image projection, device and aerial vehicle
CN108682428A (zh) * 2018-08-27 2018-10-19 珠海市微半导体有限公司 机器人语音控制系统和机器人对语音信号的处理方法
CN110051289A (zh) * 2019-04-03 2019-07-26 北京石头世纪科技股份有限公司 机器人语音控制方法、装置、机器人和介质
CN110051289B (zh) * 2019-04-03 2022-03-29 北京石头世纪科技股份有限公司 扫地机器人语音控制方法、装置、机器人和介质

Similar Documents

Publication Publication Date Title
JP3780516B2 (ja) ロボット聴覚装置及びロボット聴覚システム
JP4376902B2 (ja) 音声入力システム
JP4897666B2 (ja) 音声妨害を検出および除去する方法および装置
CN103517185B (zh) 对在嘈杂环境中操作的多话筒音频设备的声信号降噪的方法
JP6827561B2 (ja) 音響信号制御装置及び方法、並びにプログラム及び記録媒体
US20070258599A1 (en) Noise removal for electronic device with far field microphone on console
KR101840205B1 (ko) 사운드 제어장치, 차량 및 그 제어방법
RU2004136294A (ru) Диалоговое управление для электрического устройства
JP2001215990A (ja) ロボット聴覚装置
JP4866958B2 (ja) コンソール上にファーフィールドマイクロフォンを有する電子装置におけるノイズ除去
US9392360B2 (en) Steerable sensor array system with video input
WO2001057853A1 (fr) Dispositif auditif de robot
JP3277279B2 (ja) ロボット聴覚装置
CN113099348A (zh) 降噪方法、降噪装置和耳机
JP2001215989A (ja) ロボット聴覚システム
WO2001057852A1 (fr) Systeme auditif de robot
JP4552034B2 (ja) ヘッドセット型マイクロフォンアレイ音声入力装置
JP4660740B2 (ja) 電動車椅子搭載用音声入力装置
JP2001215991A (ja) ロボット聴覚装置
Okuno et al. Real-time sound source localization and separation based on active audio-visual integration
WO2022032636A1 (fr) Procédé d'anc utilisant des accéléromètres comme capteurs de son
JP2011101407A (ja) ロボット、及び収音装置
JPH08293156A (ja) 音声記録装置
Lim et al. Particle filter algorithm for single speaker tracking with audio-video data fusion
CN112788482A (zh) 一种麦克风阵列定位与云台联动的方法、装置

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): US

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
WA Withdrawal of international application