JP2002304194A - System, method and program for inputting voice and/or mouth shape information - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system, a method and a program for inputting a voice/ mouth shape information mixture so as to prevent the occurrence of leaking secret information (includes privacy information) that may occur, while inputting the information using only voice, and to provide a highly accurate information inputting. SOLUTION: The system, the method and the program are provided with a voice input means which is used to input voice uttered by a user, a mouth shape information inputting means which is used to input an image of the mouth shape of the user and an input contents determining means which is used to determine the inputted contents of the user, based on the information from the voice input means and the information from the mouth shape information input means. When the strength of the voice inputted into the voice input means is equal to or less than a prescribed level, the input contents determining means determines the inputted contents by the user, based on the information from the mouth shape information input means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system, a method, and a program for allowing a user to input information into a mobile phone, a PDA (Personal Digital Assistant), a personal computer, and the like.

[0002]

2. Description of the Related Art Conventionally, there has been known a method in which a user inputs information by voice to a mobile phone, a PDA, a personal computer, or the like. In addition, a method (lip reading function) in which a user inputs information in a mouth shape (lip shape) to a mobile phone, a PDA, a personal computer, and the like has been proposed, and there are a plurality of patents and patent applications.

[0003]

However, the conventionally proposed method is a method of inputting information only from the input voice at the time of voice input, and the method of inputting information only by lip reading at the time of lip reading input. And an input method that mixes the two has not been proposed yet. In general,
When only voice is input, when inputting secret information such as a password, there is a problem that the secret information is heard by surrounding people and the secret information is leaked. In addition, when only the lip reading function is used, there is a problem that the accuracy of the input content is reduced. SUMMARY OF THE INVENTION The present invention is to solve such a problem of the prior art, and it is possible to prevent leakage of secret information (including privacy information) only by voice input and to enable highly accurate information input. It is an object to provide a system, a method and a program for voice and / or mouth shape input that can be performed.

[0004]

(Explanation of Terms) In the present invention, "secret information" includes privacy information. In the present invention, “mouth shape” includes “lip shape”.

(Content of the present invention) The content of the present invention is as described in the claims of the present specification. Therefore, the contents of the claims of the present specification are incorporated herein (duplicate description by transferring the description of the claims herein is omitted). In the present invention, all ideas of the system (apparatus) described in the claims of this specification are referred to as “methods”.
It can be understood as a "program". That is, in the present invention, all the contents of the invention described in the form of “system (apparatus)” in the claims of the present specification can also be described in the form of “method” and “program”. It can be realized in the form of "method" or "program". That is, in the present invention, in the claims of the present specification, all the inventions in which the letters "system (apparatus)" are described at the end thereof have the letters "method" and "program" at the end. Grasp by
The described invention can be the described invention. That is, the contents of the claims in this specification can all be configured as “method inventions” and “program inventions”, and as such, are configured as “method inventions” and “program inventions”. Invented inventions are also within the scope of the present invention. Embodiments of the “program invention” of the present invention include, for example, AS
When the P (application service provider) transmits the program to the user via a network such as the Internet, the transmission target program is included.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 FIG. FIG. 1 shows a first embodiment of the present invention, and is a schematic block diagram showing a mixed voice / mouth shape input system according to the first embodiment mounted on a mobile phone. In FIG. 1, reference numeral 1 denotes a voice input unit for inputting a user's voice, which is composed of, for example, a microphone. Reference numeral 2 denotes a mouth shape input unit for inputting an image of the shape of the mouth (lips) of the user, and is constituted by, for example, a CCD camera. Reference numeral 3 denotes a voice / character conversion unit for converting the voice input from the voice input unit 1 into a corresponding character by referring to a voice database 3a in which voice and characters are associated and recorded. Reference numeral 4 denotes a mouth shape for converting the mouth (lip) shape input from the mouth shape input unit 2 into a corresponding character by referring to a mouth shape database 4a recorded in association with the mouth (lip) shape and the character. It is a character conversion unit. Reference numeral 5 denotes an input content determining unit for determining the input content from the user based on the output from the voice character converting unit 3 and the output from the mouth-shaped character converting unit 4. Reference numeral 6 denotes a transmission unit (comprising a modem or the like) for transmitting an output from the input content determination unit 5 to another information device (including a recording device) via a network, and 7 denotes the input content determination unit 5 2 is a recording unit (memory) built in the mobile phone of FIG. 1 for recording an output from the mobile phone.

In FIG. 1, reference numeral 8 denotes the loudness (level) of the voice input to the voice input unit 1, and when the level of the input voice is smaller than a predetermined level, This is a sound level determination unit for outputting a signal indicating this to the input content determination unit 5. Now, the input content determining unit 5 determines that the volume of the voice from the user input to the voice input unit 1 is smaller than a predetermined level (for example, the voice of the user). When the input content determination unit 5 receives a signal indicating that the size is of such a small level that the human ear can hardly hear it, the input content determination unit 5 automatically inputs the signal from the mouth shape input unit 2. The input content from the user is determined based only on the mouth shape that has been input (more specifically, based only on the output from the mouth shape character conversion unit 4). That is, in the first embodiment,
When the user's voice input from the voice input unit 1 is lower (lower) than a predetermined level, the input content determination unit 5 receives a signal from the voice level determination unit 8,
The input contents from the user are automatically determined based only on the mouth shape information (character information corresponding to the mouth shape information) input from the mouth shape input unit 2. Therefore, in the first embodiment, when the user is emitting a sound of a predetermined level or more from the mouth (for example, the same size or higher than a size that can be heard by a normal human ear), the user's voice and the shape of the mouth are compared. The input from the user is determined based on both pieces of information, and the user utters a sound from his / her mouth at a predetermined level or more (for example, at least as large as can be heard by a normal human ear). When there is no such information, the input content from the user is determined based on only the information on the mouth shape of the user.

Therefore, according to the first embodiment, the user may normally input a piece of information by uttering a voice of the same size as speaking to a normal person from his / her own mouth (in this case, In this case, the input content from the user is determined by both the voice from the user and the information of the mouth shape. Only in special cases, that is, when inputting secret information such as a password or a credit card number, the user is required to enter a “predetermined voice” such as a voice lower than usual or a voice almost inaudible (at all). Emit a sound at a level lower than the level "(" Speaking a sound at a level lower than the predetermined level "includes" move the mouth without sound at all ")
(In this case, the input content from the user is determined based only on the mouth shape information from the user). As described above, in the first embodiment, the user inputs normal non-confidential information (in this case, the contents of the information may be heard around the user, so that the information can be input by voice input), a password, and the like. (In this case, it is not good to hear the contents of the information in the surroundings), it is necessary to make the sound lower than the predetermined level only for the secret information (speak with a small sound that cannot be heard by the surroundings). In other words, it is possible to perform the operation seamlessly and continuously by performing the same operation as speaking only the shape of the mouth without producing any sound. Therefore, according to the first embodiment, for example, when a user makes an order to purchase a product with a virtual shop on the Internet by voice input, information that is not confidential information such as a product order or a price confirmation. If you enter information with a voice higher than the predetermined level, and enter secret information (including privacy information) such as your password, date of birth, credit card number, etc. Information can be input (either in a quiet voice that is inaudible to the surroundings, or not at all). That is, according to the first embodiment, the user normally emits a voice and inputs the voice, and when it is necessary to input confidential information such as privacy in the middle of the voice, only at that time, makes the voice lower than a predetermined level. (Including not making any sound) to prevent the confidential information from being heard by people around you, And the input can be continued.

Embodiment 2 In the first embodiment, FIG.
Although the voice character conversion unit 3 (and the voice input unit 1) and the mouth shape character conversion unit 4 (and the mouth shape input unit 2) operate in parallel at the same time, in the second embodiment, Normally (i.e., when the user emits a voice with a sound of a predetermined level or more), only the voice character conversion unit 3 (and the voice input unit 1) is operated to perform data input by a voice input method, Only when the voice from the user falls below a predetermined level, the mouth shape character conversion unit 4 (and the mouth shape input unit 2) is automatically activated to input data by the mouth shape input method. Good. That is,
In the second embodiment, unlike the first embodiment, one of the speech character conversion unit 3 (and the speech input unit 1) and the mouth shape character conversion unit 4 (and the mouth shape input unit 2) shown in FIG. It is a "trade-off relationship" that only operates and not both at the same time.

[0010]

According to the present invention, the user normally emits voice and inputs the voice (in the case of the first embodiment, the user mixes both the voice input and the mouth shape input). If you need to enter confidential information such as privacy during the process,
Prevents the secret information from being heard by surrounding people by inputting data only from the mouth shape by making the sound below a predetermined level (including making no sound at all) Then, after the input of the secret information is completed, a normal voice is emitted and the input is continued by the voice input as usual (in the case of the first embodiment, both the voice input and the mouth shape input are performed). It is possible to continue to determine the input contents by mixing the methods). Therefore, according to the present invention, the user can input normal non-secret information (in this case, the contents of the information can be heard around, so that the user can input by voice input) and secret information such as a password. The input of information (in this case, it is bad if the contents of the information are heard by others) means that the sound is lower than the predetermined level only for the secret information (speaking with a small sound that can not be heard by others, (Similar to speaking only the shape of the mouth without any sound) so that it can be performed seamlessly and continuously.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram showing a first embodiment of the present invention.

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] April 8, 2002 (2002.4.8)

[Procedure amendment 1]

[Document name to be amended] Drawing

[Correction target item name] All figures

[Correction method] Change

[Correction contents]

FIG.

Claims (6)

[Claims] 1. A voice input means for inputting voice from a user, a mouth shape input means for inputting an image of a user's mouth shape, information from the voice input means and / or the mouth shape input Means for determining input contents by the user based on information from the means. A system for inputting voice and / or mouth shape, comprising: 2. The apparatus according to claim 1, wherein the input content determining means automatically outputs the voice from the mouth shape input means when the volume of the voice input to the voice input means is smaller than a predetermined level. A system for voice and / or mouth shape input, wherein the input content of a user is determined based on information. 3. Inputting a voice from a user;
Inputting an image of a user's mouth shape, and / or inputting an image of the user's mouth shape; and input content by the user based on the input voice information and / or the input mouth shape information. And determining a speech and / or mouth shape input. 4. The input mouth shape according to claim 3, wherein in the input content determination step, when the volume of the voice input from the user is smaller than a predetermined level, the input mouth shape is automatically set. A method for mixed input of voice and mouth shape, characterized in that the input content by the user is determined based on the information of the user. 5. Inputting a voice from a user,
A function of inputting an image of a user's mouth shape and / or a function of inputting an image of the user's mouth shape based on the input voice information and / or the input mouth shape information; And a computer program for inputting voice and / or mouth shape for realizing the function. 6. The input content determination function according to claim 5, wherein when the volume of the voice input from the user is smaller than a predetermined level,
A computer program for voice and / or mouth shape input, wherein the input content by a user is determined based on the input information on the mouth shape.