JP2003061098A - Image processor, image processing method, recording medium and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor that composites a sign language image together principal moving images and audio signals, so as to fully provide hard of hearing persons with a natural and easily understandable video services. SOLUTION: The image processor is provided for encoding a moving image on an object basis to generate a database for sign language images. The processor is provided with a moving picture entry means for receiving a moving image of people making communication in sign language, an object extracting means that extracts only the people making communication in sign language as objects for capturing moving images of shape information, a natural image encoding means that encodes the moving picture of the people making communication in sign language, a shape information image encoding means that encodes the moving image of the information on the shape, a multiplexer means that multiplexes the natural image after encoding with the shape information image, a text input means that receives the meaning of the sign language image as text data, and a database storage means that stores the database comprising the multiplexed image and the text data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for interpreting from a hearing-impaired person to a hearing-impaired person. The present invention relates to a technology for composite display.

[0002]

Sign language is a language for the hearing impaired that conveys information to the other party by means of the position, direction, moving direction, moving speed, facial expression, etc. of the hand. It has a system different from the natural language used by people. Therefore, when a deaf person has a conversation with a normal hearing person, it is easier and more convenient to have a conversation in sign language than to use a natural language that belongs to the spoken language system to have a conversation or a verbal conversation. Is also fast. Therefore, a system that mixes natural language and sign language is desired.

As a conventional method of generating sign language, there are a method of using CG animation and a method of directly connecting and displaying images taken in word units.

In the method using CG animation, it is difficult to understand detailed information with a simple one, and it becomes difficult to understand the most important hand movement in sign language. On the other hand, a sophisticated development environment is required to create complicated animation.

On the other hand, ISO (International Organizati) is used as a general-purpose next-generation multimedia coding standard that can be used in many fields such as computer, broadcasting and communication.
on for Standardization: International Standardization Organization) and ITU
(International Telecommunication Union) is working on the international standardization of MPEG-4. A major feature of MPEG-4 is the ability to perform object-based coding.

Object-based coding is MPE.
G (Moving Picture Expert Group) -1 and MPEG-2
It is not the method of encoding the entire rectangular image that is adopted in the above, but the image of the shape information generated by some method in advance is used to cut out the person or other object in the image, that is, the image of the image. This is a method of encoding each object. In the following, to distinguish from this shape information image,
An image to be processed is generally called a natural image.

The shape information image is a kind of image having the same number of vertical and horizontal pixels as the natural image to be encoded and representing the shape of the object. The shape information image includes a binary alpha plane in which each pixel is represented by 1 bit and a gray scale alpha plane in which each pixel is represented by 2 bits or more. In the binary alpha plane, an area having a pixel value of "1" usually represents an object area, and an area having a pixel value of "0" represents an area outside the object. In the gray scale alpha plane, an area having a pixel value of “1 or more and 255 or less” represents an object area, and an area of “0” represents an area outside the object.

It should be noted that other detailed contents of MPEG-4 are left to ISO / IEC international standard documents.

[0009]

In order for a conversation in sign language to be conducted smoothly, the image of the signer must be clear, and in order for the shape of the signer's hand to be recognized correctly, Images need to be displayed stereoscopically.

Conventionally, in a method of simply synthesizing an image of a sign language, the image is stereoscopic, but the amount of information to be stored is large. The sign language image dictionary for registering sign language words contains at least about 60 color images per sign language word.
Since more than one word must be stored, a large amount of information is required to be stored per sign language word, which requires a large storage capacity.

An object of the present invention is to efficiently construct a database of sign language images of arbitrary shape by using an object-based coding device and method thereof, and a main AV stream (full-screen moving image and audio). By synthesizing a sign language image, it is possible to provide a means for satisfying a video service that is natural and easy for the deaf person to understand.

[0012]

According to one aspect of the present invention, a database creating means for creating a database of sign language images and a multiplexing means for encoding a main moving image to multiplex with a sign language image in the database. And an image processing device provided with means for decoding a main moving image and a sign language image from the multiplexed image, and combining and outputting.

According to another aspect of the present invention, there is provided an image processing device for performing object-based coding of a moving image to create a database of sign language images, wherein the moving image input means inputs a moving image of a signer. An object extracting means for extracting only a signer as an object to obtain a moving picture of shape information, a natural image coding means for coding the moving picture of the signer, and a moving picture of the shape information. Shape information image encoding means, multiplexing means for multiplexing the encoded natural image and shape information image, text input means for inputting the meaning of the sign language image as text data, and the multiplexed image There is provided an image processing apparatus including: and a database storage unit that stores the text data as a database.

According to still another aspect of the present invention, a coding means for coding a main moving image signal and an audio signal, a voice recognition means for extracting and recognizing voice information from the audio signal, and the above-mentioned recognition. A sign language image searching means for searching a desired sign language image from a database of sign language images using voice data, and a multiplexing means for multiplexing and outputting the encoded main moving image signal, audio signal and the sign language image. An image processing device is provided.

According to still another aspect of the present invention, the input signal is a set of a main moving image signal and an audio signal,
A first distribution means for distributing the sign language image, a first decoding means for decoding the main moving image signal and the audio signal, and the sign language image into a natural image of the sign language and a shape information image of the sign language. A second distributing means for distributing, a second decoding means for decoding the natural image of the signer, a third decoding means for decoding the shape information image of the signer, the main moving image signal and the sign language There is provided an image processing device including a synthesizing unit that synthesizes a main moving image and a sign language image from a person's natural image and a shape information image.

According to still another aspect of the present invention, a database creating step of creating a database of sign language images, a multiplexing step of encoding a main moving image and multiplexing with a sign language image in the database, the multiplexing There is provided an image processing method including a step of decoding a main moving image and a sign language image from the converted image, and synthesizing and outputting.

According to still another aspect of the present invention, there is provided an image processing method for creating a database of sign language images by performing object-based coding of moving images, the moving image input method for inputting a moving image of a signer. A step of extracting only a signer as an object to obtain a moving image of shape information, a natural image encoding step of encoding a moving image of the signer, and encoding a moving image of the shape information Shape information image encoding step, a multiplexing step of multiplexing the encoded natural image and the shape information image, a text input step of inputting the meaning of the sign language image as text data, and the multiplexing There is provided an image processing method including an image and a database storing step of storing the text data as a database.

According to still another aspect of the present invention, a coding step for coding a main moving image signal and an audio signal, a voice recognition step for extracting and recognizing voice information from the audio signal, and the recognition step. A sign language image searching step for searching a desired sign language image from a database of sign language images using voice data; and a multiplexing step for multiplexing and outputting the encoded main moving image signal, audio signal and the sign language image. An image processing method is provided.

According to still another aspect of the present invention, the input signal is a combination of a main moving image signal and an audio signal,
A first distribution step of distributing the sign language image, a first decoding step of decoding the main moving image signal and the audio signal, and the sign language image into a natural image of the sign language and a shape information image of the sign language. A second distributing step of distributing, a second decoding step of decoding a natural image of the signer, a third decoding step of decoding a shape information image of the signer, the main moving image signal and the sign language There is provided an image processing method including a synthesizing step of synthesizing a main moving image and a sign language image from a person's natural image and shape information image.

According to still another aspect of the present invention, a database creation procedure for creating a database of sign language images,
To cause a computer to perform a multiplexing procedure of encoding a main moving image and multiplexing it with a sign language image in the database, and a procedure of decoding the main moving image and the sign language image from the multiplexed image, and synthesizing and outputting. There is provided a computer-readable recording medium recording the program.

According to still another aspect of the present invention, a recording medium in which a program for performing object-based encoding of a moving image and creating a database of sign language images is recorded, and a moving image of a signer is input. A moving image input procedure, an object extracting procedure for extracting only a signer as an object to obtain a moving picture of shape information, a natural image coding procedure for coding a moving picture of the signer, and a moving picture of the shape information. A shape information image encoding procedure for encoding, a multiplexing procedure for multiplexing the encoded natural image and shape information image, a text input procedure for inputting the meaning of the sign language image as text data, A program for causing a computer to execute a multiplexed image and a database storing procedure for storing the text data as a database is recorded. Computer readable recording medium is provided.

According to still another aspect of the present invention, a coding procedure for coding a main moving image signal and an audio signal, a voice recognition procedure for extracting and recognizing voice information from the audio signal, and the above-mentioned recognition. A computer has a sign language image search procedure for searching a desired sign language image from a database of sign language images using voice data, and a multiplexing procedure for multiplexing and outputting the encoded main moving image signal, audio signal, and the sign language image. There is provided a computer-readable recording medium recording a program to be executed by a computer.

According to still another aspect of the present invention, the input signal is a set of a main moving image signal and an audio signal,
A first distribution procedure for distributing the sign language image, a first decoding procedure for decoding the main moving image signal and the audio signal, and the sign language image into a natural image of the sign language and a shape information image of the sign language. A second distribution procedure for distribution, a second decoding procedure for decoding the natural image of the signer, a third decoding procedure for decoding the shape information image of the signer, the main moving image signal, and the sign language. There is provided a computer-readable recording medium in which a program for causing a computer to execute a synthesizing procedure for synthesizing a main moving image and a sign language image from a person's natural image and shape information image is recorded.

According to still another aspect of the present invention, a database creating procedure for creating a database of sign language images,
To cause a computer to perform a multiplexing procedure of encoding a main moving image and multiplexing it with a sign language image in the database, and a procedure of decoding the main moving image and the sign language image from the multiplexed image, and synthesizing and outputting. Program is provided.

According to still another aspect of the present invention, there is provided a program for performing object-based encoding of a moving image and creating a database of sign language images, which comprises a moving image input procedure for inputting a moving image of a signer. , An object extraction procedure for extracting only a signer as an object to obtain a moving picture of shape information, a natural image coding procedure for coding the moving picture of the signer, and a shape for coding the moving picture of the shape information An information image encoding procedure, a multiplexing procedure for multiplexing the encoded natural image and a shape information image, a text input procedure for inputting the meaning of the sign language image as text data, and the multiplexed image A program for causing a computer to execute a database storage procedure for storing the text data in a database and storing the database is provided.

According to still another aspect of the present invention, a coding procedure for coding a main moving image signal and an audio signal, a voice recognition procedure for extracting and recognizing voice information from the audio signal, and the recognized procedure. A computer has a sign language image search procedure for searching a desired sign language image from a database of sign language images using voice data, and a multiplexing procedure for multiplexing and outputting the encoded main moving image signal, audio signal, and the sign language image. A program is provided for execution by.

According to still another aspect of the present invention, the input signal is a set of a main moving image signal and an audio signal,
A first distribution procedure for distributing the sign language image, a first decoding procedure for decoding the main moving image signal and the audio signal, and the sign language image into a natural image of the sign language and a shape information image of the sign language. A second distribution procedure for distribution, a second decoding procedure for decoding the natural image of the signer, a third decoding procedure for decoding the shape information image of the signer, the main moving image signal, and the sign language. There is provided a program for causing a computer to execute a synthesizing procedure for synthesizing a main moving image and a sign language image from a person's natural image and shape information image.

Since the present invention is configured in this way, a database of sign language images of arbitrary shape can be efficiently constructed, and the sign language image can be synthesized with the main moving image and audio signal (full-screen moving image and voice). , It is possible to satisfy video services that are natural and easy for the hearing impaired.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. (First Embodiment) An object-based coding method will be described with reference to FIGS. Figure 7 (a) is a natural image,
FIG. 7B is a shape information image corresponding to FIG. As described above, the object-based encoding method can encode an image with high efficiency because it encodes only the part of the object, not the entire image.

Further, FIG. 8A shows a state in which a main image is displayed, and FIG. 8B shows a state in which a sign language image object is combined with the image. In this way, in the object-based coding, the shape of the object is already clear, and therefore it can be easily combined with the background image.

FIG. 1 is a block diagram showing the configuration of an apparatus for creating a database of sign language images as the first embodiment of the present invention. In this embodiment, an implementation method in MPEG-4 will be described. According to MPEG-4, a moving image based on a natural image is referred to as a texture image, and a moving image based on the shape information of an object to be coded corresponding to this texture is referred to as a shape image. Two types of moving images composed of the texture image and the shape image are input to the apparatus.

The configuration of FIG. 1 will be described. Reference numeral 100 is a camera for photographing the sign language. Reference numeral 101 divides the signer area and the background area from the image of the signer captured by the camera 100,
Binary where the signer area is 1 and the background area is 0
It is a shape generator that generates a moving image composed of an alpha plane. A texture encoder 102 encodes a natural image of a signer from a camera. Reference numeral 103 is a shape encoder that encodes the shape image of the sign language from the shape generator 101. A multiplexer 104 multiplexes the streams from the texture encoder and the shape encoder. Reference numeral 105 denotes a sign language image database for associating and storing the output from the multiplexer and the text corresponding to the separately input sign language image.

Next, the operation of FIG. 1 will be described. First,
The camera 100 captures an image of a signer. For shooting, a background such as a blue background is used to facilitate later composition with other images. The shape generator 101 generates a shape information image of the signer from the photographed image. Next, the photographed image (natural image) of the signer himself is sent to the texture encoder 102, and the shape information image is sent to the shape encoder 103.
And code each object. The respective encoded data are multiplexed by the multiplexer 104 and output as a sign language image stream, which is stored in the database 105 together with the text data which is the meaning of the sign language image.

FIG. 2 is a block diagram showing the overall construction of an apparatus for encoding and multiplexing main audio / video and sign language images as the first embodiment of the present invention. The configuration of FIG. 2 will be described. A main AV encoder 200 inputs and encodes a video signal and an audio signal, which are main contents. A voice recognizer 201 recognizes voice from input audio and outputs the text data. Two
Reference numeral 03 is a sign language image searcher that searches for the corresponding sign language image from the sign language image database using the text data from the voice recognizer 201. A multiplexer 202 multiplexes the main stream from the main AV encoder and the sign language image stream from the sign language image searcher 203.

Next, the operation of FIG. 2 will be described. First,
The main video and audio material signals are encoder 200
Is encoded and output as the main stream. At the same time, the audio material signal is recognized and decoded by the voice recognizer 201. After that, the sign language image database 1 accumulated for the voice decoded by the sign language image searcher 203
Get the matching sign language image stream from 05. Next, the multiplexer 202 multiplexes the main stream and the sign language image stream to output a final bit stream.

FIG. 3 is a block diagram showing the overall construction of an apparatus for decoding and synthesizing main audio / video and sign language images as the first embodiment of the present invention. The configuration of FIG. 3 will be described. Reference numeral 300 is a distributor that inputs the bit stream and distributes it to the main stream and the sign language image stream. A main AV decoder 301 decodes the main stream from the distributor 300 into an audio signal and a video signal.
Reference numeral 302 denotes a distributor which distributes the texture stream of the signer and the shape stream which is the shape information from the sign language image stream. A texture decoder 303 decodes a texture image signal of a signer from the texture stream. A shape decoder 304 decodes a shape image signal of a signer from the shape stream. 305 inputs the image synthesis instruction signal, and when the image synthesis instruction signal is ON, synthesizes the video signal from the main AV decoder 301 and the signal from the sign language image decoder 302 and outputs,
When OFF, the sign language image synthesizer outputs only the audio / video signals from the main AV decoder 301.

Next, the operation of FIG. 3 will be described. First,
The stream is input, and the distributor 300 distributes the main stream and the sign language image stream. Main stream is main AV
It is decoded by the decoder 301 and output as a main video signal and a main audio signal. The sign language image stream is distributed by the sign language image distributor 302 into a texture stream which is a texture image signal of a signer and a shape stream which is a shape image signal of a signer. The texture stream is decoded by a texture decoder 303 into a texture image signal of a signer, and the shape stream is decoded by a shape decoder 304 into a shape image signal of a signer. Next, the main video signal from the main AV decoder 301, the texture image signal from the texture decoder 303, and the shape image signal from the shape decoder 304 are combined by the sign language image synthesizer 305 to obtain a video in which the main video signal and the sign language image signal are synthesized. Output a signal.

FIG. 4 shows the entire stream structure when the main audio / video stream and the sign language image stream are multiplexed using MPEG-4. By using MPEG-4, all streams are packetized in fixed time units, and TS (Time Stamp) indicating the display time is superimposed and then multiplexed, making it possible to easily manage the display of each stream. Become.

FIG. 5 shows an operation flowchart of the sign language image synthesizer 305. First, the main audio / video signal and the sign language image signal (texture image and shape image) are input (400). Next, the main video signal that must be output is output (401). Next, an instruction signal for synthesizing and displaying the sign language image is input (402). When the instruction to synthesize is received, that is, when the synthesis display instruction signal is ON, a video signal in which the sign language image is synthesized with the main video is output (404). When the instruction signal is OFF, the main video signal is output without combining (401) and at the same time the audio signal is output (403).

Another configuration of the first embodiment will be described. In this embodiment, the international standard MPEG-4 is used for the object-based coding method, but it may be applied to other object-based coding methods.
In addition, encoding and decoding other than the object-based encoding method may be other than MPEG-4, for example, MPEG-1, 2 or the like. Although the sign language image synthesizer 305 switches the audio and the sign language image, the sign language image may be synthesized or not output, and the audio may be kept output.

The effects of the first embodiment will be described. The present embodiment provides a means for accumulating a large number of sign language images with a small amount of data in order to construct a sign language image database using an encoded sign language image stream. Since the sign language image is created as a natural image, it is possible to accumulate the detailed facial expressions and movements of the sign language person compared to images such as CG. Since the sign language image is encoded using the object-based encoding, it can be easily combined with other images.

(Second Embodiment) FIG. 6 is a block diagram showing the arrangement of a computer according to the second embodiment. Reference numeral 500 denotes a central processing unit (CPU) that controls the entire computer and performs various kinds of processing, and 501 denotes a memory that provides an operating system (OS) necessary for controlling the computer, software, data, and a storage area necessary for calculation. Is. It is also used as a work area when the CPU 500 performs various processes.

Reference numeral 502 is a bus for connecting various devices and exchanging data and control signals, 503 is a storage device for storing various software, 504 is a storage device for storing moving image data, and 505 is an image or from a computer. It is a monitor that displays system messages.

Reference numeral 507 is a communication interface for transmitting encoded data to the communication circuit, which is connected to a LAN, a public line, a wireless line, a broadcast wave, etc. outside the apparatus. 506
Is a terminal for starting a computer and setting various conditions such as a bit rate.

The memory 501 stores an OS for operating the various computers and software for operating various software, an area for reading image data for encoding, and temporarily storing code data. There is a working area for storing a code area, various calculation parameters, and the like.

The operation of FIG. 6 will be described. In the above configuration, prior to the processing, the storage device 50 is transferred from the terminal 506.
The moving image data to be encoded is selected from the sign language moving image and the corresponding text data stored in 4, and the activation of the computer is instructed. Then, the software stored in the storage device 503 executes the memory 50 via the bus 502.
It is expanded to 1 and the software is started. And CP
The U500 sequentially realizes the processing shown in FIGS. 1, 2, and 3.

The computer in this embodiment is the first
In this embodiment, a database of sign language images of arbitrary shape is efficiently constructed, and the apparatus functions as a device for synthesizing the sign language image with the main AV stream (full-screen moving image and voice).

As is clear from the above description, the first and second embodiments are mainly constructed by efficiently constructing a database of sign language images of arbitrary shapes using the object-based coding device and its method. By synthesizing a sign language image with an AV stream (full-screen moving image and audio), it provides a means to satisfy a video service that is natural and easy for the hearing impaired.

This embodiment can be realized by a computer executing a program. Also,
Means for supplying the program to the computer, for example, a recording medium such as a CD-ROM storing the program or a transmission medium such as the Internet for transmitting the program can be applied as the embodiment of the present invention. The above program, recording medium, and transmission medium are included in the scope of the present invention. As the recording medium, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a non-volatile memory card, a ROM or the like can be used.

It should be noted that the above-mentioned embodiments are merely examples of specific embodiments for carrying out the present invention.
The technical scope of the present invention should not be limitedly interpreted by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

[0051]

As described above, by efficiently constructing a database of sign language images of arbitrary shape and synthesizing the sign language image with the main moving image and audio signal (moving image and voice of full screen), hearing A video service that is natural and easy for people with disabilities to understand can be satisfied.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a sign language image database creation device as a first embodiment of the present invention.

FIG. 2 is a block diagram showing an overall configuration of a device for encoding and multiplexing main audio / video and sign language images as the first embodiment of the present invention.

FIG. 3 is a block diagram showing an overall configuration of a device for decoding and synthesizing main audio / video and sign language images as the first embodiment of the present invention.

FIG. 4 is a diagram showing an overall stream configuration when a main audio / video stream and a sign language image stream are multiplexed.

FIG. 5 is an operation flowchart when synthesizing a sign language image.

FIG. 6 is a block diagram showing a configuration of a computer as a second exemplary embodiment of the present invention.

FIG. 7 is a diagram showing an example of a natural image and a shape information image when a signer is object-encoded.

FIG. 8 is a diagram showing an example of an image when a sign language image is combined with a main image.

[Explanation of symbols]

100 cameras 101 shape generator 102 texture encoder 103 shape encoder 104, 202 multiplexer 105 Sign Language Image Database 200 Main AV encoder 201 voice recognizer 203 Sign Language Image Retrieval Device 300 distributor 301 Main AV decoder 302 Sign Language Image Distributor 303 texture decoder 304 shape decoder 305 Sign Language Image Synthesizer 500 CPU 501 memory 502 bus 503, 504 storage media 505 monitor 506 terminal 507 Communication I / F

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 7/08 G10L 3/00 551G 7/081 551C F term (reference) 5C023 AA06 AA17 AA37 AA38 BA02 BA11 CA01 CA05 DA04 5C052 AA01 AC08 CC11 DD04 DD06 5C059 KK37 MA00 MB03 MB06 MB12 MB23 PP04 PP28 PP29 RB01 RB18 RC19 RC32 SS06 SS12 SS19 SS30 UA02 UA05 5C063 AB07 AC01 AC05 CA11 CA20 CA36 5D015 KK01

Claims (18)

[Claims] 1. A database creating means for creating a database of sign language images, a multiplexing means for encoding a main moving image to multiplex with a sign language image in the database, and a main moving image from the multiplexed images. An image processing apparatus including means for decoding a sign language image, and synthesizing and outputting the image. 2. An image processing apparatus for performing object-based encoding of a moving image to create a database of sign language images, comprising moving image input means for inputting a moving image of a signer, and only the signer as an object. An object extraction unit that extracts a moving image of the shape information, a natural image encoding unit that encodes the moving image of the signer, a shape information image encoding unit that encodes the moving image of the shape information, Multiplexing means for multiplexing the encoded natural image and shape information image, a text input means for inputting the meaning of the sign language image as text data, and a database of the multiplexed image and the text data. An image processing apparatus comprising: a database accumulating means for accumulating. 3. A coding means for coding a main moving image signal and an audio signal, a voice recognition means for extracting and recognizing voice information from the audio signal, and a database of a sign language image using the recognized voice data. An image processing apparatus comprising: a sign language image searching means for searching for a desired sign language image from the above; and a multiplexing means for multiplexing and outputting the encoded main moving image signal, audio signal, and the sign language image. 4. A first distribution means for distributing an input signal into a set of a main moving image signal and an audio signal and a sign language image, and a first decoding means for decoding the main moving image signal and the audio signal. A second distribution unit that distributes a natural image of the signer and a shape information image of the signer from the sign language image; a second decoding unit that decodes the natural image of the signer; and a shape of the signer An image processing apparatus comprising: a third decoding means for decoding an information image; and a synthesizing means for synthesizing a main moving image and a sign language image from the main moving image signal, a natural image of the signer and a shape information image. 5. The input means for inputting an instruction signal for instructing to synthesize the main moving image and the sign language image; and the synthesis means for receiving an instruction signal for instructing synthesis.
The image processing apparatus according to claim 4, further comprising: a switching unit that synthesizes the main moving image and the sign language image and outputs the main moving image without synthesizing the main moving image and the sign language image when no synthesis instruction is given. 6. The switching means outputs a main moving image and a sign language image by synthesizing and outputting the main moving image and the sign signal when the instruction signal instructs the synthesizing, and outputs a main moving image and the audio signal by not activating the synthesizing instruction. Image processing device. 7. A database creating step of creating a database of sign language images, a multiplexing step of encoding a main moving image to multiplex with a sign language image in the database, and a main moving image from the multiplexed image. An image processing method comprising the steps of decoding a sign language image and combining and outputting it. 8. An image processing method for creating a database of sign language images by performing object-based encoding of moving images, comprising: a moving image input step of inputting a moving image of a signer; and only the signer as an object. An object extraction step of obtaining a moving image of the extracted shape information, a natural image encoding step of encoding the moving image of the signer, a shape information image encoding step of encoding the moving image of the shape information, A multiplexing step of multiplexing the encoded natural image and the shape information image, a text input step of inputting the meaning of the sign language image as text data, and a database of the multiplexed image and the text data. An image processing method comprising a database storing step of storing. 9. A coding step of coding a main moving image signal and an audio signal, a voice recognition step of extracting and recognizing voice information from the audio signal, and a database of a sign language image using the recognized voice data. An image processing method comprising: a sign language image searching step of searching for a desired sign language image from the above; and a multiplexing step of multiplexing and outputting the encoded main moving image signal, audio signal, and the sign language image. 10. A first distributing step of distributing an input signal into a set of a main moving image signal and an audio signal and a sign language image, and a first decoding step of decoding the main moving image signal and the audio signal. A second distribution step of distributing the sign language image into a signer's natural image and a signer's shape information image, a second decoding step of decoding the signer's natural image, and the signer's shape An image processing method comprising: a third decoding step of decoding an information image; and a synthesizing step of synthesizing a main moving image and a sign language image from the main moving image signal, a natural image of the signer, and a shape information image. 11. A database creation procedure for creating a database of sign language images, a multiplexing procedure for encoding a main moving image and multiplexing it with a sign language image in the database, and a main moving image from the multiplexed images. A computer-readable recording medium in which a program for causing a computer to execute a procedure of decoding a sign language image and combining and outputting the image is recorded. 12. A recording medium in which a program for performing object-based encoding of a moving image and creating a database of a sign language image is recorded, the moving image input procedure for inputting a moving image of a signer, and a signer only. Object extraction procedure for extracting a moving image of shape information by extracting as a object, a natural image encoding procedure for encoding the moving image of the signer, and a shape information image encoding for encoding the moving image of the shape information. A procedure, a multiplexing procedure for multiplexing the encoded natural image and the shape information image, a text input procedure for inputting the meaning of the sign language image as text data, and the multiplexed image and the text data Computer readable recording program for causing a computer to execute the database storage procedure for storing as a database Recording medium. 13. A coding procedure for coding a main moving image signal and an audio signal, a voice recognition procedure for extracting and recognizing voice information from the audio signal, and a database of sign language images using the recognized voice data. A program for causing a computer to execute a sign language image search procedure for searching for a desired sign language image from the computer, and a multiplexing procedure for multiplexing and outputting the encoded main moving image signal, audio signal, and the sign language image is recorded. Computer-readable recording medium. 14. A first distribution procedure for distributing an input signal into a set of a main moving image signal and an audio signal and a sign language image, and a first decoding procedure for decoding the main moving image signal and the audio signal. A second distribution procedure for distributing the sign language image into a signer's natural image and a signer's shape information image, a second decoding procedure for decoding the signer's natural image, and the signer's shape A program for causing a computer to execute a third decoding procedure for decoding an information image, and a synthesizing procedure for synthesizing a main moving image and a sign language image from the main moving image signal, the natural image of the signer, and a shape information image. A computer-readable recording medium in which is recorded. 15. A database creation procedure for creating a database of sign language images, a multiplexing procedure for encoding a main moving image and multiplexing it with a sign language image in the database, and a main moving image from the multiplexed images. A program for causing a computer to execute a procedure of decoding a sign language image and synthesizing and outputting the image. 16. A program for creating a database of sign language images by performing object-based encoding of moving images, comprising a moving image input procedure for inputting a moving image of a signer and extracting only the signer as an object. An object extraction procedure for obtaining a moving image of shape information, a natural image coding procedure for coding the moving image of the signer, a shape information image coding procedure for coding the moving image of the shape information, and the code A multiplexing procedure for multiplexing the converted natural image and the shape information image, a text input procedure for inputting the meaning of the sign language image as text data, and a database of the multiplexed image and the text data for storage. A program that causes a computer to execute the database storage procedure. 17. A coding procedure for coding a main moving image signal and an audio signal, a voice recognition procedure for extracting and recognizing voice information from the audio signal, and a sign language image database using the recognized voice data. A program for causing a computer to execute a sign language image search procedure for searching for a desired sign language image from the computer and a multiplexing procedure for multiplexing and outputting the encoded main moving image signal, audio signal, and the sign language image. 18. A first distribution procedure for distributing an input signal into a set of a main moving image signal and an audio signal and a sign language image, and a first decoding procedure for decoding the main moving image signal and the audio signal. A second distribution procedure for distributing the sign language image into a signer's natural image and a signer's shape information image, a second decoding procedure for decoding the signer's natural image, and the signer's shape A program for causing a computer to execute a third decoding procedure for decoding an information image, and a synthesizing procedure for synthesizing a main moving image and a sign language image from the main moving image signal, the natural image of the signer, and a shape information image. .