JPH01162492A - Image transmission system - Google Patents

Abstract

PURPOSE:To reduce information quantity and to economize a transmission by selecting a mouth shape parameter based on a phoneme in voice information transmitted and deforming a mouth shape model image. CONSTITUTION:A corresponding mouth shape parameter is selected from a code book 23 and sent to a memory 24 by a phoneme code in voice information from a transmitter side. The mouth shape model image previously stored is deformed for each parameter in the memory 24, is synthesized by a synthesizing part 25 with a face moving image except the mouth part sent from the transmitter side and the moving image of a face is reproduced. Consequently, since the transmission image is only face image information except the mouth part of a speaker, information quantity can be reduced.

Description

[Detailed Description of the Invention] [Summary] Regarding a method for transmitting a tilted image of a person along with its sound, the present invention aims to realize a transmission method that can further compress the information of the tilted image. A receiving side is provided with a memory that stores a mouth shape model image defined by a set of mouth shape parameters shown in FIG. A corresponding mouth shape parameter is selected from the codebook based on the phoneme code inside, the mouth shape model image in the memory is transformed based on the mouth shape parameter, and the synthesizer combines it with the tilted image other than the mouth part from the transmitting side. In addition to this, it also has a normalized codebook for standard mouth shape images, converts each mouth shape parameter of the normalized codebook according to the first old image from the sending side, and reproduces the code. The configuration includes an initialization device that initializes the book.

[Industrial application field]

The present invention relates to an image transmission method, and particularly to a method for transmitting a tilted image of a person along with the sound thereof.

The goal for videophones, video conferences, etc. is to eventually adopt a transmission method that uses public telephone lines, and for this reason, there is a desire to compress the obtained image information as much as possible. There is.

[Conventional technology]

Images transmitted in videophones, etc. are usually tilted images of people, but as shown in Figure 12, conventional methods have been to transmit such moving image information independently of audio information. be.

That is, an input image is generated as an analog image signal by a TV camera 61 on the transmitting side, and this image signal is converted into a digital signal by an image encoding device 62, encoded, compressed, and sent to the receiving side. On the receiving side, an image decoding device 63 decodes the received image into an original signal and displays it on a display 64 as an output image.

In addition, the input voice is obtained as voice information by the micro 5 on the transmitting side, and is then compressed by voice-specific encoding in the voice encoding device 66.The input voice is then decoded by the voice decoding device 67 on the receiving side, and is then output to the speaker. 68 as output audio.

[Problem that the invention seeks to solve]

Since the amount of information in moving images is large, this traditional method of transmitting moving images that has been commonly used cannot use low bit rate communication lines, resulting in high costs. There was a problem in that it was far from being applicable to TV telephones etc. using public telephone lines.

Therefore, an object of the present invention is to realize a transmission method that can further compress information on tilted images.

[Means for solving problems]

FIG. 1 shows a conceptual diagram of an image transmission method according to the first invention for achieving the above object. A memory 24 that stores a mouth shape model image, and a codebook 23 that stores parameter values of the mouth shape model image for all phoneme codes are provided on the receiving side, and the codebook is created using the phoneme codes in the audio information from the transmitting side. A corresponding mouth shape parameter is selected from 23, the mouth shape model image in the memory 24 is transformed based on the mouth shape parameter, and the synthesizer 25 synthesizes it with the face moving image other than the mouth part from the transmitting side to reproduce a tilted image. do.

In this first invention, the codebook 23 is limited to a specific speaker.

Therefore, in the image transmission system according to the second invention conceptually shown in FIG. An initialization device 30 is provided which initializes the codebook 23 by converting each mouth shape parameter of the normalized codebook 33 according to the initial mouth side image.

[For production]

In the image transmission system according to the first invention shown in FIG.
The phoneme code included in the audio information from the transmitting side (this may be transmitted in the form of a phoneme code, or it may be extracted in the form of a phoneme code by decoding the encoded and transmitted voice) and the codebook. The phoneme code stored in the memory 23 is compared with the phoneme code stored in the memory 23, and a set of parameters of the matching phoneme code is successively extracted and sent to the memory 24. The memory 24 transforms the pre-stored mouth shape model image for each set of parameters, and combines it with the tilted image other than the mouth part sent from the transmitting side in the synthesis unit 25 to reproduce a moving image of the face. .

This eliminates the need to transmit a moving image of the mouth area.

In the image transmission system according to the second invention shown in FIG.
The initialization device 30 added to the first aspect of the present invention first applies each mouth shape parameter of the normalization codebook 33, which is a standard mouth shape image normalized by the basic phoneme code, to the mouth side image transmitted from the transmitting side. The above codebook 23 is initialized by corresponding conversion.

Therefore, the codebook 23 can be used for an unspecified number of speakers.
can be converted.

〔Example〕

Embodiments of the image transmission system according to the present invention will be described below.

FIG. 3 shows an embodiment of the image transmission system according to the first invention.
The transmission section 10 includes an image processing section 11 that performs image processing on tilted image input, and an audio encoding section 12 that encodes human voice input. The receiving unit 20 also includes a voice decoding unit 21 that decodes the voice code encoded by the transmitting unit 10, a voice recognition unit 22 that recognizes the voice signal output from the decoding unit 21, and a voice recognition unit 22 that recognizes the voice signal output from the decoding unit 21. A codebook 23 that sequentially selects a set of mouth shape parameters from phoneme codes (consisting of vowels or consonants, which are the basic constituent units of speech) successively output from the recognition unit 22;
A mouth shape model transformation unit (memory) 24 that transforms a mouth shape model image according to a set of mouth shape parameters sequentially selected in , and a mouth side image generated from this mouth shape model transformation unit 24.
It includes a synthesis section 25 that synthesizes the tilted image other than the mouth portion from the image processing section 11 of the transmission section 10.

In addition, as shown in FIG. 4, the codebook 23 includes parameters j (for example, the width of the mouth), (for example, the thickness of the lips), . . . n (for example, the vertical width of the mouth) is quantified in a table that is stored in advance as the personal information.

Further, the mouth shape model transformation unit 24 stores (maps) one screen (one frame) worth of mouth side image data of the specific speaker as the mouth shape model image in advance as the personal information. This may be done by first sending one screen worth of images of the mouth part from the transmitter 10, but the code book 23 needs to be created in advance.

Next, the operation of the above embodiment will be explained.

The audio input is encoded by the audio encoder 12 and sent to the receiver 20.
This audio code is decoded by the audio decoding section 21 and output as audio. This speech output is sent to the speech recognition section 22, and its phoneme codes are sequentially extracted and sent to the codebook 23. In the codebook 23, a set of parameters I, ■, . . . regarding the corresponding mouth shape are selected from the codebook shown in FIG.
・Select n. Then, based on the selected set of parameters, the mouth shape model deforming section 24 generates an old image obtained by deforming the mouth shape model image stored in advance. As a result, the correspondence relationship between the generated old image and the phonemes extracted by the voice recognition section 22 is as shown in FIG.

The old image deformed and generated in this way is sent to the transmitter 1
A moving image of the entire face is obtained by combining the tilted image information of the outside of the mouth sent from the image processing unit 11 of No. 0 in the combining unit 25.

The transformation of the mouth shape model image by the mouth shape model transformation section 24 is described in IEICE Technical Report IE87-2, Vol. 87, No. 19.

FIG. 6 shows another embodiment of the image transmission system according to the first invention. The difference from the embodiment of FIG. 3 is that the transmitter 10 is provided with a voice recognition section 13, The phonetic code is separated into a phonetic code and other information (intonation, pitch, etc.) and sent to the receiving unit 20.The receiving unit 20 uses the phonetic code as it is in the codebook 23 and synthesizes the phonetic code and information such as intonation into speech. The unit 26 synthesizes the signals and generates audio output. Other configurations and operations are the same as those in FIG. 3.

In the above embodiment, the pre-stored codebook 23 is unique to a predetermined speaker, so when trying to transmit old images of an unspecified number of people, the full-voice code stored in the codebook is 1. Whether to perform rewriting processing to adapt to the speaker each time the speaker changes.1. Alternatively, a huge memory area must be prepared in the codebook in order to record all the codebook information of registered speakers.

Therefore, in the second aspect of the present invention already shown in FIG. 2, the codebook can be used in accordance with unspecified speakers.

That is, as shown in Figure 7, a standard codebook is created by measuring each parameter value of the mouth shape model for the mouth shape of a standard human when pronouncing all phonemes, and each parameter value in this codebook is A normalized codebook is created for each parameter by normalizing (dividing) the parameter value of a predetermined basic phoneme code (for example, silent code) (see FIG. 8).

Then, as shown in Figure 9, a set of parameters is measured from the old image of the individual corresponding to the basic phoneme code, and the total phoneme code of the normalized codebook is obtained for each parameter as shown in Figure 8. By multiplying each parameter for , a personal codebook can be created. That is, for example, if the obtained set of individual oral direct image parameters is b
If I + "' b I n, then in Fig. 8, the normalized code of parameter I with phoneme code ■ is a 21/ a
11 is multiplied by the above parameter bl+ (a
21/ a ++) b z is converted to the code,
Similarly, regarding the parameter (2), the parameter b is multiplied with respect to all phoneme codes.

FIG. 10 shows an embodiment provided with an initialization device 30 for creating such a personal codebook. This reproduces tilted images of a large number of speakers.

The specific configuration of this initialization device 30 is shown in FIG. 11. First, an old image of a basic phoneme code (silence code in this case) in a face image is sent from the image processing unit 11 of the transmitting unit 10. When the old image is received, the feature point extraction unit 31 of the initialization device 30 extracts the feature points of the old image. Then, a parameter calculation unit 32 calculates a set of parameters based on the distance between feature points and the like. This set of parameters is multiplied by a multiplier 34 for each parameter of a normalized codebook prepared in advance in the normalized codebook memory 33 as shown in FIG. is created and stored in the codebook 23.

From now on, it will be referenced when transmitting old images of that individual.

Incidentally, this initialization device 30 is similarly applied to the embodiment shown in FIG.

〔Effect of the invention〕

As described above, according to the image transmission method of the present invention, a mouth shape model image is prepared in advance, mouth shape parameters are selected based on the phoneme in the transmitted audio information, and the shadow parameter Since the mouth shape model image prepared by can be used, making it possible to realize an inexpensive transmission method.

Furthermore, since an initialization device is provided so that the prepared codebook can be updated for each speaker, it becomes possible to easily support an unspecified number of speakers.

[Brief explanation of the drawing]

FIG. 1 is a block diagram conceptually showing an image transmission method according to the first invention, FIG. 2 is a block diagram conceptually showing an image transmission method according to the second invention, and FIG. 3 is a block diagram conceptually showing an image transmission method according to the second invention. A block diagram showing an embodiment of the image transmission method according to the first invention; FIG. 4 is a configuration diagram of a codebook used in the image transmission method according to the invention; FIG. 5 shows mouth images for each phoneme code. The sixth part is a block diagram showing another embodiment of the image transmission method according to the first invention, and FIG. 8 is a configuration diagram of a normalization codebook, FIG. 9 is a diagram showing the procedure for creating a personal codebook in the image transmission system according to the second invention, and FIG. FIG. 11 is a block diagram showing the configuration of an initialization device used in the second invention, FIG. 12 is a system showing a conventional general image transmission method. figure,
It is. 1 and 2, 23... code book, 24... memory, 25... synthesis unit, 30... initialization device, 33... normalization code book. In the figures, the same reference numerals indicate the same or corresponding parts.

Claims (6)

[Claims] (1) A memory (24) that stores a mouth shape model image defined by a set of mouth shape parameters indicating the geometric shape of the mouth part, and a code that stores the parameter values of the mouth shape model image for all phoneme codes. A book (23) is provided on the receiving side, a corresponding mouth shape parameter is selected from the codebook (23) according to the phoneme code in the audio information from the transmitting side, and the data in the memory (24) is selected based on the mouth shape parameter. An image transmission method characterized by deforming a mouth shape model image and combining it with a face moving image other than the mouth part from the transmitting side in a compositing section (25) to reproduce the image. (2) The transmitting side encodes the voice input, and the receiving side decodes and recognizes the voice to provide phoneme codes to the codebook (23).
Image transmission method described in section. (3) The transmitting side recognizes the voice input and transmits it as a phoneme code, and the receiving side provides the phoneme code to the codebook (23). image transmission method. (4) A memory (24) that stores a mouth shape model image defined by a set of mouth shape parameters indicating the geometric shape of the mouth part, and a code that stores the parameter values of the mouth shape model image for all phoneme codes. A book (23) is provided on the receiving side, a corresponding mouth shape parameter is selected from the codebook (23) according to the phoneme code in the audio information from the transmitting side, and the data in the memory (24) is selected based on the mouth shape parameter. This is an image transmission method in which a mouth shape model image is transformed and synthesized with a face moving image other than the mouth part from the transmitting side in a synthesis unit (25) to reproduce the image, and the standard mouth shape image normalization codebook (33) is used. and an initialization device (30) that initializes the codebook (23) by converting each mouth shape parameter of the normalized codebook (33) according to the first mouth image from the transmitting side. An image transmission method featuring: (5) The transmitting side encodes a voice input, and the receiving side decodes and recognizes the voice to provide a phoneme code to the codebook (2). Image transmission method. (6) The transmitting side recognizes the voice input and transmits it as a phoneme code, and the receiving side provides the phoneme code to the codebook (2).
Image transmission method described in section.