JP2007094104A - Information processing apparatus, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To excellently recognize a speech that a user utters. <P>SOLUTION: In S31, an utterance decision part 14 stands by until a speech is input from a speech processing part 11 and when the speech is input, an advance to S32 is made to decide whether the detection result of a movement pattern of a specified part from a user's picture is supplied. When it is decided that the detection result is supplied, an advance to S33 is made to decide whether the supplied movement pattern is a pattern of movement during the utterance. When it is decided that the supplied movement pattern is the pattern of movement during the utterance in S33, an advance to S34 is made to report the utterance to a speech recognition part. When receiving the report, the speech recognition part starts recognizing the supplied speech. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an information processing apparatus, method, and program, and more particularly, to an information processing apparatus, method, and program that can appropriately recognize a voice uttered by a user.

  In recent years, in the field of robots and home electronics, techniques for recognizing a user's voice and executing predetermined processing have been actively studied.

  By the way, when the collected voice is simply recognized (see Non-Patent Document 1 and Non-Patent Document 2), in addition to the voice uttered by the user, for example, it is output from a television receiver near the user. When voice or the like is included, the user's voice cannot be accurately recognized.

  Thus, for example, a method is conceivable in which the movement of the user's mouth is detected from the video of the user's mouth and the user's voice is recognized based on the detection result (Non-Patent Documents 1 to 6). , Patent Document 1 and Patent Document 2).

L. Rabiner and B. H. Juang, Fundamentals of Speech Recognition, New Jersey, Prentice Hall, Signal Processing Series (1993) Towards improving speech detection robustness for speech recognition in adverse conditions, Speech Communication, Volume 40, Issue 3, May 2003, Pages 261-276, Lamia Karray and Arnaud Martin P. de Cuetos, C. Neti, A.W.Senior.Audio-visual intent-to-speak detection for human-computer interaction, ICASSP June 5-9 2000, pp. 1325-1328, Istanbul, Turkey. G. Iyengar, C. Neti.A vision-based microphone switch for speech intent detection, July 2001, Vancouver, Canada. Large-Vocabulary Audio-Visual Speech Recognition: A Summary of the John Hopkins Summer 2000 Workshop. C. Neti, G. Potaminos, J. Luettin, I. Matthews, H. Glotin and D. Vergyri. ), Cannes, 2001. Audio-Visual Speech Recognition. C. Neti, G. Potamianos, J. Luettin, I. Matthews, H. Glotin, D. Vergyri, J. Sison, A. Mashari and J. Zhou. Technical Report, Workshop 2000, CLSP, Johns Hopkins University, July-August 2000. US20040243416 A1 G10L 20041202 Speech recognition US20030048930 A1 G06K 20030313 KABUSHIKI KAISHA TOSHIBA Image recognition apparatus and method

  As described above, a method of using a user's video in voice recognition has been proposed. However, in this method, it is assumed that the user's voice is included in the input voice, and the user's voice is included in the input voice. There is no method for using the user's video when determining whether or not the audio is included.

  In addition, in the conventional method using the user's movement, usually the movement of the mouth or the movement of the eyes is used separately. There is no way to use the movement of the face (head), the movement of the hand, or the movement of the whole body.

  The present invention has been made in view of such a situation, and enables voice recognition to be performed accurately by executing voice recognition comprehensively using user movements.

  One aspect of the present invention is a voice input means for inputting voice, a video input means for inputting video, and a detection means for detecting movements of a plurality of parts of a speaker's body from the video input by the video input means. And determining means for determining whether or not the movement detected by the detecting means is a movement at the time of speech, and based on the determination result by the determining means, the voice input by the voice input means An information processing apparatus comprising voice recognition means for recognizing.

  One aspect of the present invention is to detect a motion of a plurality of parts of a speaker's body from a voice input step for inputting voice, a video input step for inputting video, and a video input in the processing of the video input step. Based on a detection step, a determination step for determining whether or not the movement detected in the process of the detection step is a movement at the time of speech, and the voice input based on a determination result in the process in the determination step A speech recognition step for recognizing the speech input in the step processing.

  In one aspect of the present invention, audio is input, video is input, motion of a plurality of parts of the speaker's body is detected from the input video, and the detected motion is motion during speech. Whether or not the input voice is recognized based on the determination result.

  As described above, according to one aspect of the present invention, it is possible to accurately perform voice recognition.

  Embodiments of the present invention will be described below. Correspondences between constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment that is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. Conversely, even if an embodiment is described here as corresponding to a configuration requirement, that means that the embodiment does not correspond to a configuration requirement other than the configuration requirement. It's not something to do.

  One aspect of the present invention is an audio input unit (for example, the audio processing unit 11 in FIG. 1) for inputting audio, a video input unit (for example, the image processing unit 12 in FIG. 1) for inputting video, and the video input. Detecting means for detecting movements of a plurality of parts of the speaker's body from the video input by the means (for example, the motion pattern detecting unit 13 in FIG. 1), and the movement detected by the detecting means A determination unit (for example, the utterance determination unit 14 in FIG. 1) that determines whether or not it is a motion, and a voice recognition unit that recognizes the voice input by the voice input unit based on a determination result by the determination unit (For example, the voice recognition unit 15 in FIG. 1).

  One aspect of the present invention is an audio input step for inputting audio (for example, step S31 in FIG. 4), a video input step for inputting video (for example, step S32 in FIG. 4), and processing of the video input step. A detection step (for example, step S32 in FIG. 4) for detecting movements of a plurality of parts of the speaker's body from the input video, and the movement detected in the processing of the detection step is a movement during speech. Voice recognition for recognizing the voice input in the voice input step based on a judgment step (for example, step S33 in FIG. 4) for judging whether or not and a judgment result in the judgment step An information processing method or program including steps (for example, step S34 in FIG. 4).

  FIG. 1 shows a usage example and a configuration example of a speech recognition apparatus 3 to which the present invention is applied.

  The microphone 1 supplies the collected voice signal to the voice recognition device 3.

  The video camera 2 supplies an image signal obtained as a result of shooting to the voice recognition device 3.

  The microphone 1 and the video camera 2 are embedded in, for example, a wall of a room so that the user can know the installation position, and the user can talk to the microphone 1 and the video camera 2. Yes.

  The voice recognition device 3 recognizes the voice uttered by the user from the voice captured by the microphone 1 using the video imaged by the video camera 2 and supplies the recognition result to the processing device 4.

  The processing device 4 performs predetermined processing according to the recognition result of the user voice from the voice recognition device 3. For example, the processing device 4 is a home appliance control device, and performs control such as turning on / off a predetermined home appliance (for example, a television receiver or lighting) in accordance with a user's voice recognition result.

  Next, the configuration of the voice recognition device 3 will be described.

  The voice processing unit 11 of the voice recognition device 3 performs predetermined voice processing on the voice signal supplied from the microphone 1, and supplies the voice data obtained as a result to the utterance determination unit 14 and the voice recognition unit 15.

  The image processing unit 12 performs predetermined image processing on the image signal supplied from the video camera 2, and supplies the image data obtained as a result to the motion pattern detection unit 13.

  The movement pattern detection unit 13 includes a face movement pattern detection unit 21-1 to a hand movement pattern detection unit 21-4.

  The face movement pattern detection unit 21-1 detects the face part of the user from the image data supplied from the image processing unit 12, for example, detects a head part movement pattern, and the detection result is used as an utterance determination unit. 14.

  The eye movement pattern detection unit 21-2 detects the user's eye part from the image data supplied from the image processing unit 12, detects the movement pattern, and sends the detection result to the utterance determination unit 14. Supply.

  The mouth movement pattern detection unit 21-3 detects the mouth portion of the user from the image data supplied from the image processing unit 12, detects the movement pattern, and sends the detection result to the utterance determination unit 14. Supply.

  The hand movement pattern detection unit 21-4 detects the user's hand part from the image data supplied from the image processing unit 12, detects the movement pattern, and sends the detection result to the utterance determination unit 14. Supply.

  The speech determination unit 14 is based on the presence / absence of the audio signal from the audio processing unit 11 and the detection result of the motion pattern from the motion pattern detection unit 13, and the audio signal from the audio processing unit 11 is the audio uttered by the user. (Whether there was an utterance) or not, and the determination result is supplied to the voice recognition unit 15.

The voice recognition unit 15 uses a hidden Markov model (see Literature 1) or the like for the voice supplied from the voice processing unit 11 based on the utterance determination result from the utterance determination unit 14. Processing is performed, and the processing result is supplied to the processing device 4.
Reference 1: L. Rabiner and BH Juang, Fundamentals of Speech Recognition, New Jersey, Prentice Hall, Signal Processing Series (1993)

  When a person speaks, the eyes, head, and hands usually move in addition to the mouth of the speaker (user). For example, when nodding, the head moves back and forth, and moves left and right when denying the content of the story. The eyes and hands move with a predetermined pattern as well.

  That is, in the present invention, voice recognition is performed by detecting movements of a plurality of parts such as a mouth, eyes, heads, hands, etc. from a video of a speaker (user) and comprehensively using the detection results. Is to be done accurately.

  Next, the operation of the face motion pattern detection unit 21-1 of the voice recognition device 3 will be described with reference to the flowchart of FIG.

  In step S11, the face motion pattern detection unit 21-1 inputs image data for one frame from the image processing unit 12.

  In step S12, the face motion pattern detection unit 21-1 detects a skin color region from the input image data, and determines a tracking target (for example, a forehead portion) based on the detection result.

  In step S <b> 13, the face motion pattern detection unit 21-1 inputs image data of the next frame from the image processing unit 12.

  In step S14, the face motion pattern detection unit 21-1 detects the optical flow to be tracked using the image data of the frame input in step S13.

  In step S15, the face motion pattern detection unit 21-1 recognizes the motion pattern of the tracking target (forehead portion) based on the optical flow detected in step S14.

  In step S16, the face motion pattern detection unit 21-1 determines whether or not the motion pattern recognized in step S15 is within the range of face motion assumed in advance, and determines that it is not within the range Now, assuming that the region currently being tracked is not a face image, the process proceeds to step S17.

  In step S17, the face motion pattern detection unit 21-1 performs skin color extraction processing on the tracking target. In step S18, for example, a skin color region of a certain region is recognized as a result, and the tracking target is a face portion and moves. It is determined whether or not the pattern tracking should be continued.

  If it is determined in step S18 that it is not a face portion, the process returns to step S11, and the subsequent processing is performed to detect the tracking target from the beginning.

  If it is determined in step S16 that the recognized motion pattern is within the expected range, or if it is determined in step S18 that tracking of the motion pattern should be continued, the process proceeds to step S19, and the face motion pattern The detection unit 21-1 stores the detected movement of the tracking target and tracks the pattern.

  In step S20, the face motion pattern detection unit 21-1 determines whether data necessary for determining the motion pattern to be tracked has been collected. If it is determined that the data has not yet been collected, the process returns to step S13. The subsequent processing is executed in the same manner.

  If it is determined in step S20 that the data necessary for determining the motion pattern to be tracked has been collected, the process proceeds to step S21, and the face motion pattern detection unit 21-1 performs, for example, a hidden Markov based on the collected data. Using a model or a Bayesian network, a motion pattern of a face (for example, a head portion) is determined and supplied to the utterance determination unit 14.

  Thereafter, the process ends. The above-described processing is repeatedly executed at a predetermined timing, for example, every time image data is supplied from the image processing unit 12.

  Note that the operations of the eye movement pattern detection unit 21-2 to the hand movement pattern detection unit 21-4 are basically the same as the operation of the face movement pattern detection unit 21-1, and thus description thereof is omitted.

  In addition to the processing described above, the detection processing of the motion pattern detection unit 13 includes, for example, processing for specifying a tracking target from image data (processing corresponding to the processing in step S12), and characteristics of the tracking target, as shown in FIG. Can be performed by other methods including a process for extracting the process (a process corresponding to the process of steps S14 and S15) and a process for classifying the extracted feature (a process corresponding to the process of step S21).

  Documents 2 and 3 describe the appearance models that specify the tracking target, and Document 4 describes the optical flow with skin color detection.

Reference 2: Large-Vocabulary Audio-Visual Speech Recognition: A Summary of the John Hopkins Summer 2000 Workshop. C. Neti, G. Potaminos, J. Luettin, I. Matthews, H. Glotin and D. Vergyri. Workshop on Multimedia Signal Processing (MMSP), Cannes, 2001.
Reference 3: Allan D. Jepson, David J. Fleet, and Thomas F El-Maraghi. Robust online appearance models for visual tracking. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, pages 415 {422, Kauai, Hawaii, 2001.
Reference 4: Hoey, J .: Decision Theoretic Learning of Human Facial Displays and Gestures., PhD thesis, University of British Columbia (2004).

  Next, operation | movement of the speech determination part 14 is demonstrated with reference to the flowchart of FIG.

  In step S31, the speech determination unit 14 stands by until a voice is input from the voice processing unit 11. When a voice is input, the utterance determination unit 14 proceeds to step S32.

  In step S32, the speech determination unit 14 determines whether or not a motion pattern detection result is supplied from the motion pattern detection unit 13 at that time. If it is determined that the motion pattern detection result is supplied, the process proceeds to step S33.

  In step S33, the speech determination unit 14 uses a hidden Markov model, a Bayesian network, or the like (see References 5 and 6), and the motion pattern supplied from the motion pattern detection unit 13 is a motion pattern at the time of speech. It is determined whether or not there is an utterance operation.

Reference 5: Bourlard, H. and Dupont, S., 鄭 new ASR approach based on independent processing and recombination of partial frequency bands, Proc. ICSLP, vol. 1, pp. 426429, 1996.
Reference 6: Dynamic Bayesian Networks for Audio-Visual, Speech Recognition, Ara V. Nefian et al. EURASIP Journal on Applied Signal Processing 2002: 11, 115, 2002

  If it is determined in step S33 that the pattern is a movement pattern at the time of utterance (there is an utterance operation), the process proceeds to step S34, and the utterance determination unit 14 notifies the voice recognition unit 15 that an utterance has occurred. . When the voice recognition unit 15 receives the notification from the utterance determination unit 14, the voice recognition unit 15 starts voice recognition of the voice supplied from the voice processing unit 11.

  If it is determined in step S32 that no motion pattern detection result has been input, if it is determined in step S33 that there is no utterance operation, or if the utterance determination result is notified to the voice recognition unit 15 in step S34. Returning to step S31, the subsequent processing is performed in the same manner.

  Next, the operation in the utterance determination unit 14 described above will be specifically described with reference to FIG.

  In the uppermost column of FIG. 5, an example of an image supplied to the motion pattern detection unit 13 is shown.

  The image A is a captured image (moving image) of the entire body when the user is standing and talking with gestures (when the face, eyes, mouth, and hands are moving), and this image A is the image processing unit 12. , The face motion pattern detection unit 21-1 to the hand motion pattern detection unit 21-4 each detect a motion pattern of a predetermined portion and supply the detection result to the utterance determination unit 14.

  In FIG. 5, the ○ marks shown in the “Face”, “Eye”, “Mouth”, and “Hand” items corresponding to the “Movement” column provided below each image column are It indicates that a motion pattern of a part has been detected, and a cross indicates that a motion pattern of that part has not been detected.

  When the image A is supplied to the motion pattern detection unit 13, in this case, a loud voice uttered by the user is supplied to the utterance determination unit 14.

  That is, since the user's voice and the motion pattern detection result for the image A are input to the utterance determination unit 14 (steps S31 and S32), the utterance determination unit 14 proceeds to step S33, and the motion pattern is the utterance time. It is determined whether it is an operation.

  In this case, the utterance determination unit 14 determines that the movements of the head, eyes, mouth, and hand from the movement patterns supplied from the face movement pattern detection unit 21-1 to the hand movement pattern detection unit 21-4 are as follows. (Step S33), and notifies the voice recognition unit 15 that there is an utterance (step S34).

  In FIG. 5, the ○ marks shown in the “Face”, “Eye”, “Mouth”, and “Hand” items corresponding to the “Movement during utterance” column provided below each image column are , Indicates that the movement of the part is a movement at the time of utterance, and a cross indicates that the movement of the part is not a movement at the time of utterance.

  Image B shows the face part when the user covers his mouth with his hand and speaks quietly (when his eyes are moving but his face and hands are not moving and his mouth is not visible) When this image B is supplied from the image processing unit 12, the eye movement pattern detection unit 21-2 detects the eye movement pattern and sends the detection result to the utterance determination unit 14. The face movement pattern detection unit 21-1, the mouth movement pattern detection unit 21-3, and the hand movement pattern detection unit 21-4 do not detect the movement pattern, and thus do not supply the detection result to the utterance determination unit 14.

  When the image B is supplied to the motion pattern detection unit 13, in this case, a small voice uttered by the user is supplied to the utterance determination unit 14.

  That is, since the user's voice and the detection result of the eye movement pattern with respect to the image B are input to the utterance determination unit 14 (steps S31 and S32), the utterance determination unit 14 proceeds to step S33, and the eye movement pattern is the utterance. It is determined whether or not it is an operation at the time.

  In this case, the utterance determination unit 14 determines that the movement of the eye is the movement at the time of utterance from the movement pattern supplied from the eye movement pattern detection unit 21-2 (step S33), and indicates that there is an utterance. The recognition unit 15 is notified (step S34).

  The image C is a captured image (moving image) of the whole body when the user is sitting and sleeping (when the face, eyes, mouth, and hands are not moving), and the image C is supplied from the image processing unit 12. None of the face motion pattern detection unit 21-1 to the hand motion pattern detection unit 21-4 detects a motion pattern, and therefore does not supply the detection result to the utterance determination unit 14.

  When the image C is supplied to the motion pattern detection unit 13, no sound is supplied to the utterance determination unit 14.

  That is, since no voice is input to the utterance determination unit 14 after all (step S31), the utterance determination unit 14 does not notify the voice recognition unit 15 that an utterance has occurred (step S34).

  Image D is a captured image (moving image) when the user is silently sitting in front of the television receiver and playing a video game (when the face and mouth are not moving and the eyes and hands are moving). Yes, when this image D is supplied from the image processing unit 12, the eye movement pattern detection unit 21-2 and the hand movement pattern detection unit 21-4 detect the movement patterns of the eyes and hands, and the detection results are obtained. This is supplied to the speech determination unit 14. Since the face movement pattern detection unit 21-1 and the mouth movement pattern detection unit 21-3 do not detect a movement pattern, the detection result is not supplied to the utterance determination unit 14.

  When the image D is supplied to the motion pattern detection unit 13, the video game sound output from the television receiver is supplied to the utterance determination unit 14.

  That is, the speech pattern determination unit 14 receives the motion pattern detection results from the eye movement pattern detection unit 21-2 and the hand movement pattern detection unit 21-4 for the video game sound and the image D (steps S31 and S32). The utterance determination unit 14 proceeds to step S33, and determines whether or not the movement pattern is an operation at the time of utterance.

  In this case, the utterance determination unit 14 determines that the eye movement and the hand movement are not movements during utterance from the movement patterns supplied from the eye movement pattern detection unit 21-2 and the hand movement pattern detection unit 21-4. (Step S33), the voice recognition unit 15 is not notified that there is an utterance (Step S34).

  The image E is a radio-captured image (image that does not include the user's video) (moving image) that outputs radio broadcast audio. When this image E is supplied from the image processing unit 12, the face motion pattern detection is performed. Since the unit 21-1 to the hand movement pattern detection unit 21-4 do not detect the movement pattern, the detection result is not supplied to the utterance determination unit 14.

  When the image E is supplied to the motion pattern detection unit 13, radio sound is supplied to the speech determination unit 14.

  That is, radio speech is input to the speech determination unit 14 (step S31), but no motion pattern is input (step S32), so the speech determination unit 14 notifies the speech recognition unit 15 that a speech has occurred. No (step S34).

  As described above, since voice recognition is performed by detecting a user's utterance operation from video together with voice input, voice recognition can be performed accurately. For example, as in the example of the image D and the image E in FIG. 5, even if there is a voice input, if the user's speech operation is not detected, the voice recognition is not performed, so that the voice is output from the television receiver or the radio. Is not recognized as the user's voice.

  Moreover, when detecting the user's speech movement, the speech movement is detected using the movements of a plurality of parts of the user's body, so that voice recognition can be performed more accurately. For example, as in the case of the image B, the speech movement can be detected from the movement of the eyes even if the movement of the mouth is not detected, so the user's voice can be detected even if the movement of the mouth is not detected. Voice recognition can be performed.

  In the example of FIG. 1, as shown in FIG. 6A, the speech operation is determined based on the voice input and the operation pattern of the predetermined part, but the operation pattern of each unit is detected as shown in FIG. 6B. When that part is detected as a previous step, the speech action can be determined by further using the detection result.

  Thereby, for example, when the mouth is detected, it is possible to simplify the detection of the utterance operation as if there was an utterance operation regardless of the operation pattern.

  In the example of FIGS. 6A and 6B, the detection result of the target portion and the detection result of the motion pattern are used only for the speech motion detection. However, as shown in FIG. 6C, it can also be used for voice processing. .

  As a result, the user's voice can be extracted more accurately.

  In the above description, the video of the user is used only for the determination of the speech movement. However, the voice recognition unit 15 can also perform voice recognition using, for example, the movement of the mouth.

  Thereby, even if a voice other than the user's voice is included in the voice, the voice can be recognized accurately.

  In the above, the user's body part to be detected is the face, eyes, mouth, and hand, but other parts such as the whole body, ears, arms, legs, etc. may be detected and the detection result may be used. it can.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 7 is a block diagram showing an example of the configuration of a personal computer that executes the above-described series of processing by a program. A CPU (Central Processing Unit) 201 executes various processes according to a program stored in a ROM (Read Only Memory) 202 or a storage unit 208. A RAM (Random Access Memory) 203 appropriately stores programs executed by the CPU 201 and data. The CPU 201, ROM 202, and RAM 203 are connected to each other via a bus 204.

  An input / output interface 205 is also connected to the CPU 201 via the bus 204. Connected to the input / output interface 205 are an input unit 206 made up of a keyboard, mouse, microphone, and the like, and an output unit 207 made up of a display, a speaker, and the like. The CPU 201 executes various processes in response to commands input from the input unit 206. Then, the CPU 201 outputs the processing result to the output unit 207.

  A storage unit 208 connected to the input / output interface 205 includes, for example, a hard disk, and stores programs executed by the CPU 201 and various data. The communication unit 209 communicates with an external device via a network such as the Internet or a local area network.

  Further, a program may be acquired via the communication unit 209 and stored in the storage unit 208.

  The drive 210 connected to the input / output interface 205 drives a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives the programs and data recorded therein. Get etc. The acquired program and data are transferred to and stored in the storage unit 208 as necessary.

  As shown in FIG. 7, a program recording medium for storing a program that is installed in a computer and is ready to be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory), DVD (Digital Versatile Disc), a magneto-optical disk, a removable medium 211 that is a package medium composed of a semiconductor memory, or the like. It is comprised by the hard disk etc. which comprise. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 209 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the step of describing the program stored in the program recording medium is not limited to the processing performed in time series in the order described, but is not necessarily performed in time series. Or the process performed separately is also included.

It is a block diagram which shows the structural example of the speech recognition apparatus 3 to which this invention is applied. It is a flowchart explaining operation | movement of the motion pattern detection part 13 of FIG. It is another figure explaining operation | movement of the motion pattern detection part 13 of FIG. It is a flowchart explaining operation | movement of the speech determination part 14 of FIG. It is another figure explaining operation | movement of the speech determination part 14 of FIG. It is a figure explaining other operation | movement of the speech recognition apparatus. And FIG. 16 is a block diagram illustrating a configuration example of a personal computer.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Microphone, 2 Video camera, 3 Voice recognition apparatus, 4 Processing apparatus, 11 Voice processing part, 12 Image processing part, 13 Motion pattern detection part, 14 Speech determination part, 15 Voice recognition part

Claims (3)

Voice input means for inputting voice;
Video input means for inputting video;
Detecting means for detecting movement of a plurality of parts of a speaker's body from the video input by the video input means;
Determination means for determining whether or not the movement detected by the detection means is a movement during speech;
An information processing apparatus comprising: a voice recognition unit that recognizes the voice input by the voice input unit based on a determination result by the determination unit. A voice input step for inputting voice;
A video input step for inputting video;
A detection step of detecting movements of a plurality of parts of a speaker's body from the video input in the video input step;
A determination step for determining whether or not the movement detected in the process of the detection step is a movement during speech;
A speech recognition step of recognizing the speech input in the speech input step based on a determination result in the determination step. A voice input step for inputting voice;
A video input step for inputting video;
A detection step of detecting movements of a plurality of parts of a speaker's body from the video input in the video input step;
A determination step for determining whether or not the movement detected in the process of the detection step is a movement during speech;
A program that causes a computer to execute a process including a voice recognition step of recognizing the voice input in the voice input step based on a determination result in the process in the determination step.

Images