JPH08307841A - Pseudo moving image video telephone system - Google Patents

Abstract

PURPOSE: To provide the method of displaying a pseudo face moving image of a talker at a receiver side from a voice signal without image transmission from a sender side through the use of a telephone line. CONSTITUTION: A signal sent from a sender side talker via a telephone line 2 is given to a communication means 4, in which a voice signal is outputted and it is converted into a video parameter through linear prediction coding by a voice analysis means 5 to be outputted. On the other hand, a model generating means 7 stores plural mouth model and head models comprising wire frames and outputs mouth and head model data relating to the selected model. A voice parameter is converted into a mouth parameter by a parameter conversion means 8 on the mouth model data to be outputted and an image composite means 9 generates pseudo face moving image of a sender talker on the mouth parameter and the head model data and the image is displayed by a display means 10.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication device using a telephone line or the like, and more particularly to a pseudo moving picture TV telephone device for displaying a pseudo moving picture on the receiving side based on an audio signal from the transmitting side.

[0002]

2. Description of the Related Art Conventionally, a TV telephone has been put into practical use in which a speaker is imaged by a TV camera or the like, image data is compressed and transmitted together with an audio signal, and an image is expanded on the receiving side and displayed on a TV screen or the like. ing. For details, see Osamu Murakami, "Image Media and Communication Revolution" (Sangyo Tosho p61 19).
1984).

[0003]

Problems to be solved by the conventional TV telephone are as follows. First,
The image to be transmitted is the image of the speaker taken by a TV camera or the like, so it depends on the image compression method, but even if the image is compressed, the amount of information is enormous and it takes several seconds to send one screen. It will take about a degree. Also, if the sender does not have a TV telephone device, that is, for the transmission from an ordinary telephone, public telephone, or mobile phone, the TV on the receiver side
There was a problem that it could not function as a telephone device.

An object of the present invention is to provide a TV telephone device capable of displaying a pseudo facial moving image of the speaker on the receiving side from a voice signal without transmitting an image from the transmitting side using a telephone line.

[0005]

In order to solve the above problems, the pseudo moving picture video telephone apparatus of the present invention analyzes the characteristics of the vocal tract characteristic and the radiation characteristic of an audio signal and performs linear predictive coding,
A voice analysis unit that outputs a voice parameter that has been feature-extracted, a model generation unit that stores a plurality of mouth-shaped models and head models configured by wireframes, and outputs mouth-shaped and head model data related to the selected model, Parameter conversion means for inputting the voice parameter and mouth shape model data and converting and outputting the voice parameter into a mouth shape parameter that changes from moment to moment, and based on the mouth shape parameter and head model data, a pseudo face of the transmitting speaker An image synthesizing unit for generating a moving image and a display unit for displaying the pseudo face moving image of the transmitting speaker obtained by the image synthesizing unit are provided.

[0006]

In the present invention, the voice signal sent from the transmitting side speaker, for example, via the telephone line is converted into the voice parameter by the feature extraction based on the linear predictive coding in the voice analysis means. On the other hand, in the model generating means, a plurality of mouth-shaped models and head models composed of wire frames are accumulated,
The mouth shape and head model data relating to the selected model is output. The voice parameter is converted into a mouth shape parameter by the parameter converting means based on the mouth shape model data and output. Then, the image synthesizing means generates a pseudo-face moving image of the transmitting-side speaker based on the mouth shape parameter and the head model data, and displays it by the displaying means.

As described above, according to the present invention, the pseudo face moving image is generated and displayed on the receiving side from the voice signal of the transmitting side speaker, so that the signals actually transmitted are only those related to the voice, and the conventional TV telephone. Compared to the device, the amount of information and transmission time can be reduced significantly, and it can be adapted to transmission from general analog telephone lines, public telephones, mobile phones,
Since the pseudo face moving image is generated and displayed on the receiving side from the voice signal of the transmitting side speaker, it is possible to talk as if talking between videophones.

This is different from the state of the transmitting-side speaker during actual communication in the pseudo-face moving image of the transmitting-side speaker displayed on the receiving side, but at least the pseudo-face of the transmitting-side speaker is displayed. Moreover, since the motion of the mouth that matches the voice is displayed as a moving image, it is possible to make a call as if the user were talking on the sending side on a videophone.

[0009]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram showing a first embodiment of the present invention. In the drawing, only a block diagram of the receiving unit in the pseudo moving picture TV telephone device is shown, and the block diagram of the transmitting unit may be omitted because it may be a known general telephone.

In FIG. 1, reference numeral 1 is a telephone that does not have an image transmission device, such as a home telephone, a public telephone, or a mobile telephone of each home, and 2 is a telephone line. Reference numeral 3 is a pseudo-moving video telephone device of the present invention, and 4 is a communication means for outputting a voice signal sent from a transmitting side speaker via a telephone line. In this case, it is a receiving circuit of the telephone. Reference numeral 5 is a voice analysis means for converting and outputting a voice signal into a voice parameter. Reference numeral 6 is a selection unit that outputs a selection signal for the receiving speaker to select the pseudo face of the transmitting speaker. Reference numeral 7 is a model generating means for outputting the mouth shape model data and the head model data. Reference numeral 8 is a parameter conversion means for inputting a voice parameter and mouth shape model data and converting the voice parameter into a mouth shape parameter for output. 9 is based on mouth shape parameter and head model data,
It is an image synthesizing means for generating a pseudo facial moving image of the transmitting side speaker. Reference numeral 10 is a display unit for displaying a pseudo facial moving image of the transmitting side speaker.

Next, the operation of each section will be described.

When the telephone 1 is on the transmitting side and the receiving operation of the pseudo moving picture TV telephone apparatus 3 is considered, the signal sent from the telephone 1 is input to the pseudo moving picture TV telephone apparatus 3 via the telephone line 2. As the signal, the voice signal a of the transmitting speaker is output from the communication means 4. After that, the voice signal a is input to the voice analysis means 5, the features of the vocal tract characteristic and the radiation characteristic of the voice signal a are analyzed, and linear predictive coding is performed to convert the voice signal a into the voice parameter b from which the feature is extracted. It is output to the conversion means 8.

On the other hand, by listening to the voice of the transmitting speaker, the receiving speaker selects the pseudo face of the transmitting speaker to be displayed by the selecting means 6. Along with this, the selection signal c is output from the selection means 6 and input to the model generation means 7. The model generation means 7 stores a plurality of mouth-shaped models and head models formed of wire frames, and a selection signal c
The model generation means 7 which received the parameter data converts the mouth shape model data d relating to the model selected based on the selection signal c to the parameter conversion means 8 and the head model data f to the image synthesis means 9.
Output each.

Here, the selection of the pseudo face of the transmitting-side speaker by the receiving-side speaker will be described in detail in relation to the operations of the selecting means 6 and the model generating means 7. For example, the receiving speaker images the faces of people who are frequently called, such as relatives, acquaintances, and friends, in advance, and stores them in the model generating means 7 as head models. Also, relatives,
Faces for accommodating people other than acquaintances, friends, and others who frequently call, the face of the standard model is stored as a head model in the model generating means 7 from the beginning, and the receiving speaker The face of the standard model may be created and edited by itself and stored in the model generation means 7 as a head model. When a call is received, the receiving speaker first determines the person from the voice of the transmitting speaker. If the person is a close relative, an acquaintance, a friend, etc., the face is selected by the selection means 6, and the selection signal c is output to the model generation means 7 accordingly. If the person is not a person such as a relative, an acquaintance, or a friend, the face of the standard model is selected by the selection means 6, and the selection signal c is output to the model generation means 7 accordingly. Further, regardless of who is the sender speaker, the selecting unit 6 may select the face of the standard model that the receiver speaker has created and accumulated in advance.

Now, the voice parameter b includes the mouth shape information at the time of pronunciation in the conversation of the transmitting side speaker and is input to the parameter converting means 8. At the same time, the mouth shape model data d is also input to the parameter converting means 8. Here, the voice parameter b is converted and output based on the mouth shape model data d into a mouth shape parameter e that changes from moment to moment. Then, the mouth shape parameter e and the head model data f are input to the image synthesizing means 9. The image synthesizing means 9 transforms a three-dimensional model (wireframe model) composed of triangular polygons on the basis of the mouth shape parameter e and the head model data f, and performs texture mapping processing on each polygon, thereby transmitting side talk. Person's pseudo face moving image is synthesized. still,
FIG. 2 shows an image diagram of a three-dimensional model (wireframe model). Countless triangles forming the three-dimensional model (wireframe model) 90 shown in FIG. 2, that is,
By deforming the triangular polygons and subjecting each of the polygons to texture mapping processing based on the mouth shape parameter e and the head model data f, a pseudo facial moving image of the transmitting speaker can be obtained. Finally, the display unit 10 displays the pseudo-face moving image of the transmitting-side speaker obtained by the image synthesizing unit 9.

As described above, the embodiment of FIG. 1 is characterized in that the pseudo face moving image is generated and displayed on the receiving side from the voice signal of the transmitting side speaker. Volume and transmission time can be much reduced. In addition, even when transmitting from a general telephone, a public telephone, or a mobile telephone, it is possible to make a telephone call as if the telephone calls were between conventional TV telephones. Further, when the pseudo face moving image is generated and displayed on the receiving side from the voice signal of the transmitting side speaker, a method such as converting the audio signal into a parameter and directly connecting the parameter to image synthesis is used. Therefore, there is also an advantage that it is not necessary to use a complicated voice recognition means.

Next, the second embodiment of the present invention will be described in detail with reference to the block diagrams of FIGS. In FIG. 3, reference numeral 100 is a wireless terminal device for mobile communication in a receiving speaker such as an automobile or a mobile phone, and 300 is a pseudo moving image TV phone device of the present invention. Reference numeral 6 is a selection unit that outputs a selection signal for the receiving speaker to select the pseudo face of the transmitting speaker. Reference numeral 7 is a model generating means for outputting the mouth shape model data and the head model data. Reference numeral 8 is a parameter conversion means for inputting a voice parameter and mouth shape model data and converting the voice parameter into a mouth shape parameter for output. Reference numeral 9 is an image synthesizing means for generating a pseudo facial moving image of the transmitting speaker based on the mouth shape parameter and the head model data. 10
Is a display unit for displaying a pseudo facial moving image of the transmitting speaker. Further, a detailed block diagram of the wireless terminal device 100 is shown in FIG. In FIG. 4, 101 is a transmitting / receiving antenna, and 1 is
Reference numeral 02 is a high frequency unit, 103 is a modulation / demodulation unit, 104 is a channel codec, 105 is a voice encoding unit, 106 is a voice synthesizing unit, 107 is a speaker, and 108 is a microphone. For example, a PDC (Personal Digital Ce).
"Digital Car Phone System Standards (RC)
R STD-27B ”.

Next, the operation of each part will be described in detail only about the points different from the first embodiment in FIG.

When a call is made to the wireless terminal device 100, the signal received by the transmitting / receiving antenna 101 is converted into a low frequency signal by the high frequency section 102, and the modulation / demodulation section 1
It is demodulated with 03. Then the channel codec 104
Then, error correction processing is performed and the result is input to the voice processing unit 105. In the voice processing unit 105, the internal voice encoding means 10
In step 6, the features of the vocal tract characteristic and the radiation characteristic of the voice are analyzed, and linear predictive coding is performed, so that the feature-extracted voice parameter b exists. Regarding processing in linear predictive coding, in the case of PDC, VSELP (Vector-Sum) is used.
Excited linear Predictive
Coding) is adopted and similar processing is performed. The voice parameter b includes mouth shape information at the time of pronunciation in the conversation of the transmitting speaker and is input to the parameter converting means 8. The functions and operations of the selecting means 6, the model generating means 7, the parameter converting means 8, the image synthesizing means 9, and the displaying means 10 are the same as those in the first embodiment shown in FIG. As described above, the communication unit 4 is different from the first embodiment in FIG.
That is, the voice analysis unit 5 is included in the wireless terminal device 100.

Next, one modification is shown in FIG. 5 as being essentially the same as the embodiment of FIGS. 3 and 4 but with some modifications. 5 is different from FIGS. 3 and 4 in that the pseudo moving picture TV phone device 300 in FIG. 3 is incorporated in the wireless terminal device 100. The selecting unit 6 in FIG. 3 does not exist in the wireless terminal device 100 in FIG. 5, but can be substituted by operating the keypad 111 or the like via the control unit 110 in a known general wireless terminal device. Similarly, the display means 10 in FIG. 3 does not exist in the wireless terminal device 100 in FIG. 5, but it can be replaced by an LCD 112 or the like that displays a telephone number or the like in a known general wireless terminal device. The specific operation in FIG. 5 is substantially the same as the operation in FIGS.

As described above, the effects of the second embodiment shown in FIGS. 3, 4, and 5 can be obtained in the same manner as the first embodiment. Further, when considering the allowable limit of the transmission rate due to effective use of the frequency, the wireless terminal device is provided with means capable of generating and displaying the pseudo facial moving image from the audio signal on the receiving side without transmitting the image of the transmitting side speaker. The effect given to is great.

[0022]

According to the present invention, since the pseudo face moving image is generated and displayed on the receiving side from the voice signal of the transmitting side speaker, the signals actually transmitted are only those related to the voice, and the conventional TV telephone apparatus. Compared with, the amount of information and transmission time can be greatly reduced, and of course, it can be realized even with general analog telephone lines. Further, even when transmitting from a general telephone, a public telephone, or a mobile telephone, the pseudo face moving image is generated and displayed on the receiving side from the voice signal of the transmitting side speaker, so that the conventional videophones talk with each other. You can talk like that.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of the present invention.

FIG. 2 is a perspective view of a three-dimensional model used in the first embodiment of the present invention.

FIG. 3 is a block diagram showing a second embodiment of the present invention.

FIG. 4 is a block diagram showing a wireless terminal device used in the embodiment of FIG.

FIG. 5 is a block diagram showing a modification of the second embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Telephone, 2 ... Telephone line, 3 ... Pseudo video TV telephone apparatus, 4 ... Communication means, 5 ... Voice analysis means, 6 ... Selection means, 7 ... Model generation means, 8 ... Parameter conversion means, 9 ... Image synthesis means , 10 ... Display means.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Atsushi Yoshioka 292 Yoshida-cho, Totsuka-ku, Yokohama-shi, Kanagawa Ltd.

Claims (2)

[Claims] 1. A voice analysis means for analyzing characteristics of a vocal tract characteristic and a radiation characteristic of a voice signal and outputting a voice parameter having the feature extracted, and a plurality of mouth-shaped models and head models composed of wire frames are accumulated. , Model generation means for outputting mouth shape and head model data relating to the selected model, inputting the voice parameter and the mouth shape model data,
Parameter converting means for converting and outputting the voice parameter into a mouth shape parameter that changes from moment to moment, and image synthesizing means for generating a pseudo face moving image of the transmitting speaker based on the mouth shape parameter and the head model data. And a display unit for displaying the pseudo-face moving image of the transmitting-side speaker obtained by the image synthesizing unit. 2. In a mobile terminal for mobile communication, a speech coding means for analyzing characteristics of vocal tract characteristics and radiation characteristics of speech to perform linear predictive coding, and an analog speech signal from the coded speech data. A plurality of wireframes, each of which is composed of a wire frame, includes a voice synthesizing unit for synthesizing, a communication unit for transmitting and receiving voice information with the encoded voice data, and an output unit for outputting the voice parameter from which the feature is extracted. A model generation unit that stores a mouth shape model and a head model and outputs mouth shape and head model data relating to the selected model, the voice parameter output by the output unit, and the mouth shape model output by the model generation unit. Parameter conversion means for inputting data and converting the voice parameter into a mouth-shaped parameter that changes from moment to moment; An image synthesizing unit for generating the pseudo-face moving image of the transmitting-side speaker based on the head model data, and a display for displaying the pseudo-face dynamic image of the transmitting-side speaker obtained by the image synthesizing unit. And a pseudo moving picture video telephone device.