KR20020068235A - Method and apparatus of recognizing speech using a tooth and lip image - Google Patents

Abstract

PURPOSE: A system and a method of recognizing voices using images of teeth and lips are provided to obtain a high voice recognition rate even in environments having high noise as well as environments having low noise. CONSTITUTION: A voice recognition system using images of teeth and lips includes an image input unit(10) for picking up the image of teeth and lips of a speaker to convert the image into an electric video signal, drivers(18,19,20) for driving at least one load(21,22,23), and a signal processor(14) for extracting teeth/lips parameters based on the video signal. The system further includes a storage(16) for storing the parameters during a learning procedure, and a controller(15) for storing the parameters in the storage during the learning procedure and, during a recognition procedure, compares the teeth/lips parameters of the speaker with the parameters stored in the storage during the learning procedure to find out similar parameters. The controller controls the driver corresponding to the found similar parameters, and provides control signals required for the components of the system.

Description

Speech recognition device and method using tooth and lip image {Method and apparatus of recognizing speech using a tooth and lip image}

With the development of information and communication technology, voice recognition and voice synthesis technology have attracted attention recently. It is realistically possible to operate computer devices and home appliances with human voices and to enable electronic commerce.

As a security technology through biometrics, the speaker authentication technology using the voiceprint, which is in the spotlight along with fingerprint recognition and iris recognition, has become a core technology of the information society as security and security become more important as the Internet and electronic commerce are proliferating.

Speaker Verification is a technology that can authenticate the identity as a voice password by using the unique voice features of each individual. It is an advanced security technology that cannot be used illegally because the voice is used as a password.

The speaker authentication technology is used as a security password function in the home banking service of banknotes, the home trading service of securities companies, computers and various household products. However, these speech recognition technologies not only have a big problem that performance drops sharply in a noisy environment, but also greatly reduce the reliability of the product.

One way to solve this problem is U.S. Patent 4,769,845. This patent is a voice recognition method of the lip image. However, this patent has a problem that does not significantly improve the speech recognition rate due to the ambiguity of the lips.

The present invention relates to a speech recognition apparatus and method using the speaker's teeth and lips image.

In particular, the present invention relates to a speech recognition apparatus and method having a high speech recognition rate under a high noise environment as well as low noise.

The speaker's voice information; It provides a voice recognition method and apparatus that can achieve a higher voice recognition rate by combining the image information of the teeth and lips.

Still another aspect of the present invention relates to speech information of a speaker; Provides a voice recognition method and device that can achieve a higher voice recognition rate by combining with the image information representing the entire structure or part of the speaker's oral cavity by a fluoroscopy device such as an X-ray device or an infrared thermograph device. .

1 is a system block diagram that can recognize the voice and the image of the teeth including the speaker's tongue as well as the speaker's voice signal to enable high voice recognition in a high noise environment.

2 is a temporary embodiment of the present invention using a computer.

Figure 3 is an embodiment of extracting the dental lip parameters from the teeth and lip image signal including the speaker's tongue obtained by the A / D conversion unit.

4A-4C are typical lip image signals.

5A to 5C are tooth and lip image signals including a tongue corresponding to FIGS. 4A to 4C.

Figure 6 is an embodiment showing the change over time of the dental lip parameters of the "Aou Ou" pronunciation of the speaker.

7 is an embodiment of a speaker's voice parameters and dental lip parameters stored in a storage device during the learning process.

8 is a prochart for a voice recognition method using a tooth and lips image.

Figure 9 is an embodiment of the semiconductor chip 100, the speech recognition device and method using the teeth and lips image of the present invention.

Figure 10 is an embodiment showing an image of the oral structure of the speaker by the fluoroscopy device.

An object of the present invention is to use the voice information of the speaker and the teeth and lips including the tongue information as well as the speaker's voice information in order to solve the big problem that the speech recognition performance is sharply dropped in the noise environment to achieve a high voice recognition rate even in a loud noise environment. It is to provide a voice recognition method and apparatus having. It is impossible to have a high voice recognition rate in a noisy environment only by the speaker's voice information. In addition, due to the ambiguity of the lips, it is difficult to improve the voice recognition rate with the image signal of the lips alone. The ambiguity of the lips due to pronunciation can be overcome by using image signals of teeth and lips including the speaker's tongue.

In order to achieve the object of the present invention, the speaker's teeth and lips image while the speaker is speaking, the video camera or CCD camera device or infrared thermograph device or X-ray device or MRI device (magnetic resonance imaging device: Magnetic Resonace After imaging and A / D conversion by the imaging system, it is provided to a computer or a semiconductor chip which can be processed for signal processing. The computer or arithmetic semiconductor chip extracts a dental lip parameter based on the speaker's teeth and lips by phonemes, words, or sentences.

The dental lip parameters based on the teeth and lips images are compared with the dental lip parameters stored for each phoneme, word, or sentence by a pre-learning process to find similar phonemes, words, or sentences, and perform voice recognition. In addition, the present invention A / D conversion of the speech signal of the speaker input from the micro to provide to the computer or arithmetic processing semiconductor chip, the voice parameter and the tooth lip parameters extracted by the computer or arithmetic processing semiconductor chip and Provided are voice recognition methods and apparatus that can achieve higher voice recognition rates by combining information.

Therefore, the present invention is a voice recognition device using the teeth and lips image having a high voice recognition rate in a noisy environment, because the voice recognition by the lip parameters based on the speaker's teeth and lips images as well as the speaker's voice information, and Provide a method.

Hereinafter, described in detail with reference to the accompanying drawings for the voice recognition method and apparatus using the teeth and lips image of the present invention.

FIG. 1 shows a system block diagram that can recognize not only a speaker's voice signal but also a tooth and lip image including the speaker's tongue, thereby enabling high voice recognition even in a high noise environment. Reference numerals 31 and 33 denote the speaker's upper and lower lips, respectively, and reference numerals 32 and 34 denote the speaker's upper and lower teeth, respectively. Reference numeral 36 denotes the speaker's tongue. 1 shows a load section 21, 22, 23 consisting of at least one load; Driving units (18, 19, 20) for driving the load units (21, 22, 23); A video camera or CCD camera device or infrared heat for picking up and converting the images of the teeth 32 and 34 and the lips 31 and 33, including the speaker's tongue 36, into an electrical video signal while the speaker is speaking. An image input unit 10 constituted by a thermograph apparatus or an X-ray apparatus or an MRI apparatus (Magnetic Resonace Imaging System); A voice input unit 11 having a microphone for picking up the speaker's voice signal; An A / D converter 12 for converting a tooth and lip image signal including the speaker's tongue generated by the image input unit 10 into a digital signal; An A / D converter 13 for converting a voice signal generated by the voice input unit 10 into a digital signal; Tooth and lip image signals and voice signals including the tongue converted into digital signals are provided to a signal processor 14 composed of a computer or a semiconductor chip capable of arithmetic processing for signal processing. The signal processor 14 extracts a dental lip parameter based on the teeth and the lip image signal including the speaker's tongue by phonemes, words, or sentences, and extracts a voice parameter based on the speaker's voice signal by phonemes or words. Extract by sentence. A storage device 16 for storing the dental lip parameters and voice parameters during a pre-learning process; During the learning process, the dental lip parameters and the voice parameters are stored in the storage device 16, and during the recognition process, the speaker's dental lip parameters and voice parameters are stored in the storage device 16 during the learning process. Compares the lip parameters and voice parameters to find similar phonemes, words or sentences, controls the drivers 18, 19 and 20 in response to the found similar phonemes, words or sentences, and provides the necessary control signals to the respective parts. It provides a voice recognition device and method using a tooth and lip image, characterized in that it comprises a control unit 15 for.

2 shows a temporary embodiment of the present invention using a computer.

2 shows a load portion 47 consisting of at least one load; A driving unit 48 for driving the load unit 47; A keyboard 46 for inputting various data and commands; A display device 45 constituted by a computer monitor and a TV device for displaying various kinds of information; Video cameras or CCD camera devices or infrared thermograph devices or X-ray devices or MRI devices for picking up teeth and lips including the speaker's tongue while the speaker is speaking (Magnetic Resonance Imaging Device: Magnetic An image input unit 10 constituted by a Resonace Imaging system; A voice input unit 11 having a microphone for picking up the speaker's voice signal; An A / D converter 12 for converting a tooth and lip image signal including the speaker's tongue generated by the image input unit 10 into a digital signal; An A / D converter 13 for converting a voice signal generated by the voice input unit 10 into a digital signal; Tooth and lip image signals and audio signals including the tongue, which are converted into digital signals, are provided to a signal processor 14 including a central processing unit or a semiconductor chip capable of arithmetic processing for signal processing. The signal processor 14 extracts the dental lip parameters based on the teeth and the lips image signal including the speaker's tongue by phonemes, words, or sentences, and extracts voice parameters based on the speaker's voice signal. A storage device 16 for storing the dental lip parameters and voice parameters during a learning process; The dental lip parameters and voice parameters are stored in the storage device 16 during the learning process, and the dental lip parameters and voice parameters of the speaker are stored in the storage device 16 during the learning process during the recognition process. Control unit 15 for finding similar phonemes or words or sentences compared to parameters and voice parameters, controlling the driving unit 47 in response to the found similar phonemes, words or sentences, and providing control signals necessary for the respective units. ); And an interface unit 18 including the controller 15, the A / D converters 12 and 13, the signal processor 14, and the storage device 16, which are necessary for the keyboard 46 and the display device 45. The present invention provides a voice recognition device and method using a tooth and lips image, comprising a computer body 43 having a built-in).

3 illustrates an embodiment of extracting a dental lip parameter from the teeth including the speaker's tongue obtained by the A / D converter 12 and the lip image signal. Reference numerals 31 and 33 denote the speaker's upper and lower lips, respectively, and reference numerals 32 and 34 denote the speaker's upper and lower teeth, respectively. Reference numeral 36 denotes the speaker's tongue.

In one embodiment of the present invention, a dental lip parameter includes a contour shape of a lip; an opening area 49 of a lip; Horizontal lip opening size 50 and vertical lip opening size 51; The ratio of the horizontal lip opening size 50 to the vertical lip opening size 51; Interdental height 52 between upper tooth 32 and lower tooth 34; The exposure size 53 of the upper teeth and the exposure size 54 of the lower teeth; The ratio of the exposed size 53 of the upper tooth to the exposed size 54 of the lower tooth; ; The number of upper teeth exposed; The number of exposed lower teeth; whether the tongue 36 is exposed and the amount of exposure of the tongue 36; It can be a change of time depending on the time of the upper and lower positions and the front and rear positions of the tongue 36.

4A-4C show a typical lip image signal.

5A to 5C illustrate a tooth and lip image signal including a corresponding tongue.

FIG. 4A shows the tooth and lip pattern including the tongue for "ah" pronunciation, FIG. 4B shows the tooth and lip pattern including the tongue for "E" pronunciation, and FIG. 4C shows the tongue and tongue pattern for "this" pronunciation Represents tooth and lips pattern. 4A to 4C are examples of ambiguity of the lips well, and it is difficult to clearly distinguish the pronunciation of "ah", "e", and "yi" only by the lip image. 5A to 5C are dental lip images including the tongues corresponding to FIGS. 4A to 4C, showing that pronunciation can be more clearly distinguished using the dental lip parameters.

Figure 6 is an embodiment showing the change over time of the dental lip parameters for the speaker "Ae Ou" pronunciation.

Fig. 7 shows one embodiment of the speaker's speech parameters and tooth lip parameters stored in storage 16 during the learning process. Is the index of the address of the storage device Indicates the last address index of the storage device. The first address stores voice parameters and dental lip parameters corresponding to "TV on", and the second address stores voice parameters and dental lip parameters corresponding to "TV off".

During the recognition process, the speaker's dental lip parameters and voice parameters are compared with the dental lip parameters and voice parameters stored in the storage device 16 during the learning process to find the corresponding voice spoken by the speaker based on the similarity. One embodiment of the method is as follows.

Similarity between the parameters Is distance Is calculated by the combination of and distance Can be calculated by the sum of the absolute values of the difference between the speaker's parameters and the parameters stored in the storage device 16 as follows.

here, Is the index of the address of the storage device Represents time, Is the total comparison edge time of the parameter. Is a symbol representing the speaker's dental lip parameter or voice parameter as a function of time.

Contour shape of the speaker's lips above; Opening area 49 of the lip; Horizontal lip opening size 50 and vertical lip opening size 51; The ratio of the horizontal lip opening size 50 to the vertical lip opening size 51; Interdental height 52 between upper tooth 32 and lower tooth 34; The exposure size 53 of the upper teeth and the exposure size 54 of the lower teeth; The ratio of the exposed size 53 of the upper tooth to the exposed size 54 of the lower tooth; The number of upper teeth exposed; The number of lower teeth exposed; Whether the tongue 36 is exposed and the amount of exposure of the tongue 36; The change in the time of the vertical position and the front and rear position of the tongue 36 is shown.

Is a symbol representing a dental lip parameter or voice parameter stored in the storage device 16 as a function of time during the learning process,

Contour shape of the lips stored in the storage device 16 during the learning process; Opening area 49 of the lip; Horizontal lip opening size 50 and vertical lip opening size 51; The ratio of the horizontal lip opening size 50 to the vertical lip opening size 51; Interdental height 52 between upper tooth 32 and lower tooth 34; The exposure size 53 of the upper teeth and the exposure size 54 of the lower teeth; The ratio of the exposed size 53 of the upper tooth to the exposed size 54 of the lower tooth; The number of upper teeth exposed; The number of lower teeth exposed; Whether the tongue 36 is exposed; Exposure size of tongue 36; The change in the time of the vertical position and the front and rear position of the tongue 36 is shown.

An index representing the dental lip parameters and voice parameters may be as follows.

= 0: voice parameter

= 1: contour shape of lips

= 2: opening area of the lips

= 3: horizontal lip open size

= 4: vertical lip opening size

= 5: ratio of horizontal lip opening size to vertical lip opening size

= 6: interdental height between upper and lower teeth

= 7: exposed size of upper tooth

= 8: exposed size of lower teeth

= 9: ratio of the exposed size of the upper tooth to the exposed size of the lower tooth

= 10: number of exposed upper teeth

= 11: number of lower teeth exposed

= 12: tongue exposed

= 13: tongue exposure size

= 14: up and down position of tongue

= 15: front and back position of the tongue

The similarity Is the distance as It can be calculated by the weighted sum of.

here Is Is the weight of, In the calculation of It is a variable that determines the importance of.

In one embodiment, in order to increase the proportion of voice parameters under low noise environment, To keep the value large and to reduce the specific gravity of the voice parameters under high noise By setting a small value Calculate

The smaller the value, the greater the similarity, so that the values are sequentially compared with all the dental lip parameters and voice parameters stored in the storage device 16. Calculate the value, with the smallest value The index if it has a value The voice recognition process is completed by finding.

The smallest The value should have a value below a certain threshold.

8 shows a prochart for a voice recognition method using teeth and lips images.

Step 80 is a process of determining whether there is a voice signal from the voice input unit 11, and if there is a voice signal, extracting tooth lip parameters from the teeth and lip images including the tongue from the image input unit 10 (step 81), A voice parameter is extracted from the voice signal from the voice input unit 11 (step 82). After that, it is determined whether the learning process (mode) (step 83). If the learning mode, the dental lip parameters and voice parameters are stored in the storage device (16). In step 83, if it is not the learning mode, that is, the recognition mode (process), the address index is compared with the parameters stored in the storage device 16. Is set to the initial value zero (step 85). Address index Read the previously learned dental lip parameters and voice parameters from the storage device 16 corresponding to the (step 86), the distance (distance) Is calculated (step 87), For calculation, the environment is monitored (checked) and the weight is set (steps 90 and 91). The similarity will then be calculated with the set weight. The address index is then compared with the parameter stored at the next address. The value is increased to the next index value (93) and the above steps 86 to 93 are repeated.

In step 94, it is determined whether the address index is the last. Smallest minimum in Equivalent to Find the value and complete the voice recognition process (step 95). Since found above In operation 96, the corresponding driving unit is driven to execute a voice command corresponding to the value. Thereafter, the process proceeds to step 80 where it is checked whether a voice is input to recognize the next voice.

9 illustrates an embodiment in which the semiconductor chip 100 is designed for a voice recognition device and method using the teeth and lips images of the present invention.

9 shows power terminals pin1 and pin2 for supplying power to the semiconductor chip; At least one driving unit (18, 19, 20) for driving a load connected to the outside of the semiconductor chip (100); Output terminals (pin5, pin6, pin7) for connecting the drive unit (18, 19, 20) and an external load; An input terminal (pin3) for providing a tooth and lip image signal including the tongue of the speaker generated by the image input unit 10 or the A / D conversion to the semiconductor chip 100; An input terminal (pin4) for providing a voice signal generated by the voice input unit 11 or A / D converted to the semiconductor chip; A signal processor 14 for extracting a dental lip parameter and a voice parameter by phonemes, words, or sentences from the teeth and the lip image signal and the voice signal including the tongue;

An external storage device 16 for storing the dental lip parameters and voice parameters during the learning process and an interface terminal 88 therefor; The dental lip parameters and voice parameters are stored in the external storage device 16 during the learning process, and the dental lip parameters and voice parameters of the speaker are stored in the storage device 16 during the learning process during the recognition process. Compares the lip parameters and voice parameters to find similar phonemes, words or sentences, controls the drivers 18, 19 and 20 in response to the found similar phonemes, words or sentences, and provides the necessary control signals to the respective parts. A voice recognition semiconductor chip using a dental lip image in which the control unit 15 is embedded in one semiconductor chip.

As another embodiment of the present invention, the image input unit 10 includes a see-through device including an X-ray apparatus or an MRI apparatus (magnetic resonance imaging apparatus) or an infrared thermography apparatus. The present invention provides a voice recognition method and apparatus capable of achieving a higher voice recognition rate by combining image information representing the entire structure or a partial structure of the speaker's oral cavity with voice information of the voice input unit 11.

Figure 10 is an embodiment showing an image of the oral structure of the speaker by the see-through device.

Various parts of the oral cavity shown in FIG. 10 may be utilized as image information for voice recognition. In this embodiment, the signal processor 14 extracts oral parameters based on the image of the speaker's oral structure by phonemes, words, or sentences. The oral cavity parameters are stored in the storage device 16 during the pre-learning process.

During the recognition process, the controller 15 compares the speaker's oral parameters with the oral parameters stored in the storage device 16 during the learning process to find similar phonemes, words or sentences, and finds the similar phonemes or words or words. The driving units 18, 19, and 20 are controlled in response to a sentence.

Another aspect of the present invention is that the dental lip parameter as a training set during the learning process can be used as an input parameter of the Hidden Markov Model (HMM) and speech recognition is possible by the trained HMM.

Another aspect of the invention is that the dental lip parameter as a training set during the learning process can be used as an input parameter of a neural network and speech recognition is possible by the learned neural network.

Although the present invention has been described in terms of embodiments and examples, it is not intended to limit the invention thereto. In addition, those skilled in the art will recognize that various modifications and variations are possible in addition to those described herein. Such modifications also fall within the scope of the appended claims.

As described above, the present invention provides a speech recognition apparatus and method using an image of a speaker's teeth and lips, and in particular, provides a speech recognition apparatus and method having a high speech recognition rate under a high noise environment as well as low noise. .

Also the speaker's voice information; It provides a voice recognition method and apparatus that can achieve a higher voice recognition rate by combining the image information of the teeth and lips.

Still another aspect of the present invention relates to speech information of a speaker; Provides a voice recognition method and apparatus that can achieve a higher voice recognition rate by combining with the image information representing the entire structure or a partial structure of the speaker's oral cavity by a fluoroscopy device such as an X-ray device or an infrared thermograph device. .

Claims (14)

In the voice recognition device using the teeth and lips image; An image input unit for picking up the speaker's teeth and lips and converting the image into an electrical image signal; A driving unit for driving at least one load; A signal processor for extracting a dental lip parameter based on the speaker's teeth and lips image signal; A storage device for storing the dental lip parameters during a learning process; During the learning process, the dental lip parameters are stored in the storage device, and during the recognition process, similar dental lip parameters are found by comparing the speaker's dental lip parameters with the dental lip parameters stored in the storage device during the learning process. And a control unit for controlling the driving unit in response to the found similar dental lip parameters and providing a control signal required for each unit. In the voice recognition device using the teeth and lips image; An image input unit for picking up the teeth and the lips including the speaker's tongue and converting the images into electrical image signals; A driving unit for driving at least one load; A signal processor for extracting a dental lip parameter based on an image signal of teeth and lips including the speaker's tongue; A storage device for storing the dental lip parameters during a learning process; During the learning process, the dental lip parameter is stored in the storage device, and during the recognition process, similar parameters are found by comparing the speaker's dental lip parameter with the dental lip parameters stored in the storage device during the learning process. And a control unit for controlling the driving unit in response to the found similar parameter and providing a control signal required for each unit. In the voice recognition device using the teeth and lips image; An image input unit for picking up the speaker's teeth and lips and converting the image into an electrical image signal; A driving unit for driving at least one load; A voice input unit having a microphone for picking up the speaker's voice signal; A signal processing unit for extracting a dental lip parameter based on a speaker's teeth and a lip image signal and extracting a voice parameter based on a voice signal; A storage device for storing the dental lip parameters and voice parameters during a learning process; The dental lip parameters and voice parameters are stored in the storage device during the learning process, and the dental lip parameters and voice parameters are stored in the storage device during the learning process during the recognition process. And a control unit for finding a similar parameter and controlling the drive unit in response to the found similar parameter, and providing a control signal required for each unit. In a voice recognition device; An image input unit for picking up the image of the internal structure of the speaker and converting the image into an electrical image signal; A driving unit for driving at least one load; A voice input unit having a microphone for picking up the speaker's voice signal; A signal processor for extracting oral parameters based on an image signal of the internal structure of the speaker and extracting a voice parameter based on the audio signal; A storage device for storing the oral cavity parameters and the voice parameters during the learning process; The oral parameters and voice parameters are stored in the storage device during the learning process, and the oral and voice parameters of the speaker are compared with the oral and voice parameters stored in the storage device during the learning process during the recognition process. And a control unit for finding a similar parameter, controlling the driving unit in response to the found similar parameter, and providing a control signal required for each unit. The speech recognition apparatus according to any one of claims 1 to 4, wherein the image input unit is constituted by a video camera or a CCD camera apparatus or an infrared thermal imaging apparatus or an X-ray apparatus or a magnetic resonance imaging apparatus. The apparatus of claim 1, wherein the dental lip parameter is extracted by phonemes, words, or sentences. 5. The speech recognition apparatus of claim 4, wherein the oral cavity parameter is extracted for each phoneme, word, or sentence. The method of claim 1 or 3, wherein the dental lip parameter is Changes in the shape of the contour of the lips over time, changes in the opening area of the lips over time, changes over time of the horizontal lip opening size, changes over time of the vertical lip opening size, horizontal lip opening size and vertical Changes in the ratio of the size of the lip opening to the change of time, change of the interdental height between the upper and lower teeth, change of the exposure size of the upper tooth over time, change of the exposure size of the lower tooth over time The parameter selected from the trend of the exposure size of the upper teeth and the ratio of the duration of exposure of the lower teeth over time, the trend of the number of exposed upper teeth over time, and the change of the number of exposed lower teeth over time Voice recognition device using a tooth and lips image, characterized in that. The method of claim 2, wherein the dental lip parameters Changes in the shape of the contour of the lips over time, changes in the opening area of the lips over time, changes over time of the horizontal lip opening size, changes over time of the vertical lip opening size, horizontal lip opening size and vertical Changes in the ratio of the size of the lip opening to the change of time, change of the interdental height between the upper and lower teeth, change of the exposure size of the upper tooth over time, change of the exposure size of the lower tooth over time , The change in the ratio of the exposed size of the upper teeth to the exposed size of the lower teeth, the change in time of the number of exposed upper teeth, the change in time of the number of exposed lower teeth, the exposure of the tongue Change of time according to whether or not, the size of the exposure of the tongue over time, change of the time of the upper and lower positions of the tongue, time of the front and rear position of the tongue Voice recognition device using a tooth and lips image, characterized in that the selected parameter among the change trend. A voice recognition semiconductor chip using tooth and lip images; A power supply terminal for supplying power to the semiconductor chip; At least one driver for driving a load connected to the outside of the semiconductor chip; An output terminal for connecting the driving unit and an external load; An input terminal for providing a speaker tooth and lip image signal to the semiconductor chip; An input terminal for providing a speaker's voice signal to the semiconductor chip; A signal processor for extracting a dental lip parameter and a voice parameter by phonemes, words, or sentences from the tooth and lip image signals and voice signals; An interface terminal for connecting the semiconductor chip with an external storage device for storing the dental lip parameters and voice parameters during a learning process; The dental lip parameters and voice parameters are stored in the external storage device during the learning process, and the dental lip parameters and voice parameters are stored in the external storage device during the learning process during the recognition process. A controller for finding similar phonemes or words or sentences compared to parameters, controlling the driver in response to the found similar phonemes, words or sentences, and providing a control signal necessary for each part; And a voice recognition chip using a tooth and a lip image, wherein the driving unit and the signal processing unit are embedded in one semiconductor chip. In the speech recognition method using the teeth and lips image signal; (A) picking up the speaker's teeth and lips and converting them into electrical video signals; (B) picking up the speaker's voice signal; (C) extracting dental lip parameters based on the speaker's teeth and lips image signal, (D) extracting a voice parameter based on the speaker's voice signal; (E) storing the dental lip parameters and voice parameters in a storage device during a learning process; (F) reading the dental lip parameters and voice parameters stored in the storage device during the learning process during the recognition process; (G) finding similar parameters by comparing the parameters read from the storage device and the speaker's parameters during the recognition process; (H) A voice recognition method using the teeth and lips image comprising the step of controlling the drive in response to the found similar parameters. The method of claim 3 or 4, wherein in finding similar parameters during the recognition process, similar parameters are found by varying the weight and the importance of the video signal-based parameter and the voice signal-based parameter according to the noise environment. Voice recognition device. The voice recognition according to claim 1 or 2 or 3 or 4, wherein the dental lip parameters or oral parameters are used as input parameters of the HMM (Hidden Markov Model) as a learning set during the learning process. Device. 5. A speech recognition device according to claim 1 or 2 or 3 or 4, wherein said dental lip parameters or oral parameters are used as input parameters of a neural network as a learning set during a learning process.