JP2001319241A - Image data generator for voice-driven dynamic picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image data generator for voice-driven dynamic picture capable of constituting a telephone equipment or independent image generator, for which only the preparation of a few or one face image is sufficient and it is not necessary to send a large quantity of data at the time of transmission. SOLUTION: This device is provided with a sampling and storage means for previously sampling and storing the typical example of mouth state changes corresponding to the specified voice of a speaker, a read means for reading one image of the face of the speaker, a change adding means for changing the mouth of the image on the basis of the state changes of the mouth stored in the sampling and storage means concerning the mouth of the image read into this read means, and a storage means for storing the state of the mouth changed by this change adding means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data generating apparatus for an audio drive moving image plane, and more particularly to an image data generating apparatus for an audio drive moving image plane suitable for use in a cellular phone, a personal computer communication or the like.

[0002]

2. Description of the Related Art In a technology generally used in a so-called video conference such as a telephone device with an audio drive image or a video conference telephone device, images and voices of both callers are bidirectionally transmitted and received. . With this technology, in order to send images of attendees to video conferences, etc.,
Both need to have cameras. For this reason, it is difficult to use a telephone device with a display device that transmits an image of a caller's face to both without having a camera.

In order to solve this problem, the present inventor has proposed:
Basic facial image data is transmitted from the transmitting device to the receiving device in advance, and the phoneme of the speaker transmitted later is recognized by the voice recognition function, and the movement of the facial image is recognized by the data. A technique has been proposed in which a device on the receiving side can view the program together with audio (registered utility model 3062).
No. 080).

[0004]

In the invention of the registered utility model, a pattern of an image of a face (substantially the entire face) corresponding to at least some phonemes of voice is prepared in advance, and a telephone device or a communication business is prepared. Must be stored in the memory of the user's server, etc. In one of the methods,
An image must be sent for each transmission. As a result, the telephone device or the server needs to have a large-capacity memory, and needs to send a large amount of data at the time of transmission.

Accordingly, it is an object of the present invention to provide an audio drive that can be configured with only a few or one facial image and that does not require a large amount of data to be transmitted during transmission. An object of the present invention is to provide a moving image plane image data generating apparatus.

[0006]

In order to achieve the above-mentioned object, the present invention provides a reading means for reading an image of a face of a speaker when the speaker utters a plurality of specific sounds, and the reading means. A change extracting means for extracting a change in the state of the mouth corresponding to the sound from the image read into the apparatus, and a state storage means for storing the state of the mouth extracted by the change extracting means, And an image data generating device.

The present invention also provides means for reading an image of a face of a speaker when the speaker utters a voice, and image selection for selecting only an image corresponding to a specific voice from the images read by the reading means. Means, change extraction means for extracting a change in the state of the mouth corresponding to the sound from the image selected by the image selection means, and state storage means for storing the state of the mouth extracted by the change extraction means. The present invention employs an image data generating apparatus for an audio drive moving image plane, which is a feature.

The present invention further comprises sampling storage means for pre-sampling and storing a representative example of a mouth state change corresponding to a specific voice of the speaker, and reading means for reading one image of the face of the speaker. A change adding means for making a change to the mouth of the image based on a change in the state of the mouth stored in the sampling storage means for the mouth of the image read by the reading means; And a storage means for storing the state of the mouth.

Here, in the present invention, the change in the state of the mouth is preferably represented by a change in the contour of a lip, and the specific voice is preferably a vowel.

Further, according to the present invention, there is provided a telephone apparatus using the above-described audio drive moving image plane image data generating apparatus, wherein a detecting means for detecting a voice, and a voice for identifying a specific sound detected by the detecting means are provided. A telephone device, comprising: identification means; and display means for displaying an image of the face of the speaker based on a change in the state of the mouth corresponding to the specific sound identified by the voice identification means. It is.

[0011]

Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a flowchart showing the operation flow of the audio drive moving image plane image data generating apparatus according to the first and second embodiments of the present invention. FIG. 2 is a flowchart showing the flow of operation of the audio drive moving image plane image data generating apparatus according to the third embodiment of the present invention. FIG. 3 is a diagram for explaining a sampling process used in the third embodiment. FIG. 4 is a flowchart showing the flow of operation when the audio drive moving image plane image data generating device of the present invention is applied to a telephone device.

First, the basic principle of the present invention will be described. The facial expression changes variously when laughing, angry, crying, talking, and the like. When speaking while looking at the other person's face with the telephone device, it is almost sufficient to know the basic change of the expression of the other person's face when talking, so in the present invention, the expression of the face when speaking is used. Pay particular attention to the part that changes, at least the part of the mouth (and the change in the state of the lips)
This is to generate face image data including a change in the mouth part.

If the vowels (a, i, u, e, o in Japanese) in the voice are common, the change in the mouth part is
Basically, the vowels in the voice are identified and the facial expressions matching the vowels are output, since they change almost in the same manner. However, sounds other than vowels (for example, laughter) may be identified together.

The position of the mouth on the coordinate axis in the image is determined based on the center of the mouth with respect to the contour of the face obtained based on the pattern matching or the projection processing method. Determined for each vowel. Less than,
This will be described in detail with reference to the drawings.

Embodiment 1 First, reference will be made to FIG. A specific sound (for example, a vowel in particular is generated by the user of the telephone device (step S1), and a facial expression at that time is photographed by a camera (in particular, a digital camera) and stored in a memory as image data. (That is, image reading is performed) (step S2).

Next, the contour of the lips is extracted from the image data of the face (step S3), and the contour is stored as data (step S4).

(Embodiment 2) Referring to FIG. 1, a user of the telephone device utters a certain appropriate sentence (step S11), and the facial expression at the time of utterance is read as an image (step S12). ), A specific sound, for example, an image corresponding to a vowel is selected from the images (step S13). Then, the above-mentioned steps 3 and 4
Is performed.

(Embodiment 3) First, in order to obtain a contour of a lip corresponding to a specific voice, a plurality of appropriate persons are selected and uttered, not limited to the user of the voice drive telephone device. Then, the facial expression corresponding to the specific voice is sampled (step S21). Referring now to FIG. 3, a representative sample of the lip contour corresponding to a vowel is shown. Such a sample is obtained, and the sample is read as lip contour data (step S22).

Next, an image of the facial expression of the user of the telephone device is read, and a face image corresponding to a specific voice is created from the lip data obtained in step S22 (step S23) and stored. Is performed (step S24).

Next, the operation flow of the telephone device using the image data generated as described above will be described. Referring to FIG. 4, during a call on the telephone device, after waiting for voice to be detected (step S31), when voice is detected,
The voice is identified (step S32). When the voice is recognized, the data of the lip contour corresponding to the voice (especially, the vowel) is read out (step S33), and displayed as a part of the face image on the display screen of the telephone device based on the data of the lip contour. .

(Other Embodiments) In the above description, a face image is created in a telephone device in accordance with a change state of a mouth, specifically, a contour of a lip. In the image data generation device, a face image is created corresponding to the changing state of the mouth and, more specifically, the contour of the lips, stored in a memory, and the phone device reads out the face image according to a specific voice. May be displayed.

In the above-described embodiment, the basic image of a face image taken by a camera is described. However, an illustration, a portrait, an animation image, a profile, etc., representing the user's face are displayed on the original screen. It may be adopted as one.

The device of the present invention may be provided as a part of a telephone device, or may be used in an image generating device independent of the telephone device.

[0024]

As described above, according to the present invention,
It is sufficient to prepare only a few or one facial image, and an image data generating apparatus for an audio drive moving image plane that can configure a mobile phone so that it is not necessary to send a large amount of data at the time of transmission is obtained.

[Brief description of the drawings]

FIG. 1 is a flowchart showing an operation flow of an audio drive moving image plane image data generating apparatus according to Embodiments 1 and 2 of the present invention.

FIG. 2 is a flowchart illustrating a flow of an operation of an image data generating apparatus for an audio drive moving image plane according to a third embodiment of the present invention.

FIG. 3 is a diagram for explaining a sampling process used in a third embodiment;

FIG. 4 is a flowchart showing a flow of operation when the audio drive moving image plane image data generating device of the present invention is applied to a mobile phone.

Claims (11)

[Claims] 1. A reading means for reading an image of a face of a speaker when the speaker utters a plurality of specific sounds, and a change for extracting a state change of a mouth corresponding to the sound from the image read by the reading means. An audio drive moving image plane image data generating apparatus, comprising: an extraction unit; and a state storage unit that stores a state of the mouth extracted by the change extraction unit. A means for reading an image of the face of the speaker when the speaker utters a voice; an image selecting means for selecting only an image corresponding to a specific voice from the images read by the reading means; A voice comprising: a change extraction unit that extracts a change in the state of a mouth corresponding to a sound from an image selected by an image selection unit; and a state storage unit that stores a state of the mouth extracted by the change extraction unit. Image data generation device for drive moving image plane. 3. Sampling storage means for pre-sampling and storing a representative example of a mouth state change corresponding to a specific voice of a speaker, reading means for reading one image of the face of the speaker, and said reading means A change adding means for making a change to the mouth of the image based on a change in the state of the mouth stored in the sampling storage means for the mouth of the image read in, and a state of the mouth changed by the change adding means. An audio drive moving image plane image data generating apparatus, comprising: a storage unit for storing. 4. The apparatus for generating image data for an audio drive moving image plane according to claim 1, wherein the means for storing the state of the mouth stores the center position of the mouth with respect to the contour of the face on a coordinate axis. An image data generating apparatus for an audio drive moving image plane, wherein the image data is determined from the outline of the mouth and stored. 5. The voice-driven moving image plane image data generating apparatus according to claim 1, wherein the change in the state of the mouth is represented by a change in a contour of a lip. Surface image data generator. 6. The image data generating apparatus according to claim 1, wherein the specific voice is a vowel. apparatus. 7. A telephone device using the audio-drive moving image plane image data generating device according to claim 1, wherein a detecting means for detecting sound, and a specific sound detected by the detecting means. And a display means for displaying an image of the face of the speaker based on a change in the state of the mouth corresponding to the specific sound identified by the voice identification means. Telephone equipment. 8. When the speaker utters a plurality of specific voices, the apparatus has reading means for reading an image of the face of the speaker, and state storage means for storing the image of the face read by the reading means. An image data generating apparatus for an audio drive moving image plane, characterized in that: 9. A means for reading an image of a face of a speaker when a speaker generates a sound, and an image selecting means for selecting only a face image corresponding to a specific sound from the image read by the reading means. And a state storing means for storing an image of the face selected by the image selecting means. 10. The image data generating apparatus according to claim 8, wherein the specific sound is a vowel. 11. A telephone apparatus using the audio drive moving image plane image data generating apparatus according to claim 8, wherein a sensing means for sensing audio is provided.
Voice recognition means for identifying a specific sound detected by the detection means, and display means for displaying an image of a speaker's face corresponding to the specific sound identified by the voice identification means. Telephone equipment.