JP2000059794A - Production of image coding data, its program recording medium, image coding data producing device, image communication method and image communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To produce one piece of coding data, with which decoding can be performed by the same decoder in the case of obtaining plural pieces of coding data by embedding information designating a reference image in the coding data of each frame and merging plural pieces of coding data into one piece of coding data. SOLUTION: For instance, a character image inputting part 101 inputs the character image of an announcer and a character image coding part 102 encodes it. A character coding data separating part 103 stores character coding data until coding data for one frame is outputted. A frame changing means 112 selects the coding data to be transmitted under the control of a frame controlling part 113, and a frame number transmitting part 122 adds a frame number. Also, a reference frame number producing part 114 attaches a reference frame number. Furthermore, a display time transmitting part 124 attaches time, when a decoded image is displayed at decoding. A transmission side multiplexing part 116 multiplexes these data and transmits them to a receiving side.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique relating to creation, storage and communication of coded video data, and more particularly to an image communication system for communicating a plurality of coded data as one coded data. is there.

[0002]

2. Description of the Related Art In an object-based moving picture coding system such as MPEG-4, in order to change the quality of an image for each area in one screen, one screen is divided into areas of an arbitrary shape, and each area is divided. In each encoder, encoded data for each area is created. When communicating moving picture coded data using this method, the transmitting side first separates the input image for each region, creates coded data for each region, and sends the coded data to the receiving side for each region. Send. On the receiving side, after receiving the encoded data for each area, the data is decoded by the decoder for each area, and a decoded image of one screen is created from the decoded image for each area. When transmitting the coded data for each area, the coded data sent by the transmitting side and the coded data received by the receiving side are matched by a method such as using a different transmission channel for each area.

[0003] Also, H.S. The H.263 moving image coding method suitable for mobile communication generally uses inter-frame difference coding. In the inter-frame difference encoding, each frame uses either an intra-frame (I-frame) in which the inter-frame difference encoding is not performed or a frame (P-frame) encoding a difference from a previous frame.
H. In the case where the decoder accumulates a plurality of past decoded images as reference images as in H.263AnnexN, the encoder also accumulates the past locally decoded images as reference image candidates, Encode while changing the reference image. The encoder specifies the frame number of the reference image to the decoder by simultaneously transmitting the frame number of the reference image in addition to the encoded data, and the decoder accumulates the image of the received frame number. If there is, decrypt it with reference to it.

As described above, the encoder stores the plurality of locally decoded images and encodes the input image while changing the reference image. The decoding method using the reference image is used for recovery from transmission error propagation in real-time communication.

[0005]

In the prior art, MP
In a system that performs encoding by a moving image encoding method for each region of an arbitrary shape such as EG-4 and communicates encoded data, a decoding side of the encoded data is required on the receiving side in order to decode each encoded data. It was necessary to provide a plurality of decoders corresponding to the number of coded data. Therefore, due to the system configuration,
This resulted in a redundant configuration, and the apparatus had to be expensive.

Further, in general, the time required for decoding varies depending on the content of the encoded data. Therefore, in order to output the decoded image to one screen in synchronization with each region, it is necessary to quickly decode the image in the region that could be decoded quickly. Had to be stored.
That is, an image buffer for accumulating a plurality of frames of decoded images is required for each decoder.

On the other hand, H. By using the H.263 AnnexN, the decoder can decode by referring to the stored frame specified by the transmitting side. When this is used, in the conventional technology,
The transmitting side changes the reference image for one input image in real time with one encoder to create encoded data, or stores the encoded data in which the reference image has been changed in advance. And either sent it. Therefore, according to the conventional technology, when there are a plurality of input images or when there are a plurality of encoded data for each stored area, it is impossible to transmit them as one image encoded data.

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above problems, and has been made in consideration of the above-described problems, and provides a method of encoding an input image and a method of receiving encoded data for a decoder including a reference image accumulation buffer for accumulating a plurality of frames. Coded image data that can create one coded data that can be decoded by the same decoder when a plurality of coded data is obtained by a method of reading coded data from a storage medium or the like. It is an object of the present invention to provide a creation device and an image communication system realized by such an image encoded data creation device and a decoder.

[0009]

Means for Solving the Problems In order to achieve the above-mentioned object, the present invention is characterized by having the following means. (1) A plurality of pieces of encoded data are merged into one piece of encoded data by embedding information designating a reference image in the encoded data of each frame. (2) One of the following is used as reference image designation information.

(A) the frame number of the coded data after merging or the position in the reference image memory; (b) an input data identification signal indicating which input coded data was created; (3) the display order of the decoded image When specifying, display time information is added. (4) The decoding device stores a plurality of decoded images, and decodes based on the stored decoded images in accordance with the designation of the reference image in the merged encoded data.

In particular, in the first invention, a means for acquiring a plurality of encoded data, a means for separating the acquired encoded data into encoded data of each frame, Means for merging the encoded data into one encoded data, and means for designating a reference image of each frame. The receiving side includes means for decoding the encoded data, and an image for the reference image for a plurality of frames. Means for accumulating, and means for receiving a frame number of an image used as a reference image or its memory location.

Further, the transmitting side is provided with means for adding display time information for designating the display time of the decoded image, and the receiving side is provided with
Means for acquiring display time information from the received encoded data may be provided.

When acquiring a plurality of encoded data, for example, the moving image encoded data is received via a transmission path, and the encoded data is acquired. Alternatively, moving image encoded data is read from a storage medium or a device to acquire encoded data. Alternatively, encoded data is generated by encoding the input image, and the encoded data is obtained. Alternatively, the coded image data previously processed is processed to generate new coded data, and the coded data is obtained.

The operation of the present invention is as follows.
Note that the encoded data acquisition process in the following description is
It is positioned as an input to the transmitting side of the system.

In the first aspect of the present invention, a plurality of encoded data obtaining steps are included in the creation of the image encoded data. For example, two encoded data (encoded data A and encoded data B) are encoded. If there is an encoded data acquisition process A and an encoded data acquisition process B, one encoded data C
Is as follows.

It is assumed that encoded data A exists at time 1, time 2, time 4 and time 5, and encoded data B exists at time 3 and time 5. It is assumed that the encoded data A at time 1 and the encoded data B at time 3 do not refer to any frame. For simplicity of description, it is assumed that a separated data storage buffer A and a separated data storage buffer B that separate and store the encoded data of each frame separated by the encoded data separation process at each time are provided.

The coded data A obtained by the coded data acquisition step A is separated into coded data for each frame in the coded data separation step, and is stored in the separated data storage buffer A. Further, the encoded data B obtained by the encoded data acquisition step B is separated into encoded data for each frame in the encoded data separation step, and is stored in the separated data storage buffer B. At each time, the encoded data merging process determines whether the encoded data is stored in the separated data storage buffer A,
It is checked whether or not the frame is stored in the data. If the frame is stored, a frame of the encoded data C is created using the frame.

At time 1, only the coded data A exists, so the coded data merge process uses the coded data A as the coded data C. At the same time, the reference image designation process does not designate a reference image because the frame does not refer to any frame.

At time 2, since only the encoded data A exists, the encoded data merging unit sets the encoded data A to the encoded data C. At the same time, since the reference image specifying step refers to the first frame of the encoded data C, the image of the first frame is specified as the reference image.

At time 3, only encoded data B is present, so the encoded data merging process uses encoded data B as encoded data C. At the same time, the reference image designation process does not refer to any frame, so that no reference image is designated.

At time 4, since only the encoded data A exists, the encoded data merge process uses the encoded data A as the encoded data C. At the same time, since the reference image designation step refers to the second frame of the encoded data C, the image of the second frame is designated as the reference image.

At time 5, encoded data A and encoded data B are present. First, in the encoded data merging process, encoded data A is converted to encoded data C. At the same time, since the reference image specifying step refers to the fourth frame of the encoded data C, the image of the fourth frame is specified as the reference image.
Subsequently, in the encoded data merging process, the encoded data B is set to the encoded data C. At the same time, since the reference image specifying step refers to the third frame of the encoded data C, the image of the third frame is specified as the reference image.

On the receiving side, from the images stored in the image storing process, a reference image selecting process performs
An image to be used as a reference image is selected, and the received encoded data C is decoded in a decoding process with reference to the selected image.

In this embodiment, for the sake of simplicity, the separated data storage buffer A and the separated data storage buffer A which separate and store the encoded data of each frame separated by the encoded data separation process at each time. B, but if there is no separated data storage buffer, the encoded data merging process directly merges the encoded data obtained from the encoded data separating process.

As described above, according to the first aspect, one piece of encoded data can be created from a plurality of pieces of encoded data. Also, the encoded data created on the transmitting side can be decoded on the receiving side by referring to the image specified by the transmitting side.

According to the second aspect of the present invention, on the transmitting side, means for acquiring encoded data, means for separating the acquired encoded data into encoded data for each frame, and encoding for each separated frame Means for merging data into one piece of encoded data; and means for creating a frame identification signal that is a signal for distinguishing from which encoded data acquisition unit the encoded frame is input. The receiving side comprises means for decoding encoded data, means for accumulating a plurality of frames of images to be used as reference images, and means for receiving a frame identification signal. The operation of the present invention is as follows.

According to a second aspect of the present invention, instead of creating a signal for designating an image to be a reference image of each frame of the encoded data created by merging the encoded data in the first aspect of the invention, the encoding is performed. A frame identification signal, which is a signal for discriminating which encoded data acquisition process has been performed by the encoded data created by merging the data, is created. That is, in the first aspect described above, by specifying the reference destination of each frame constituting the coded data to be created, erroneous reference is prevented between the coded data belonging before the creation.
On the other hand, in the second invention, by specifying a signal for distinguishing the coded data belonging to before the creation of each frame constituting the coded data to be created, the encoded data belonging to the coded data belonging before the creation is not specified. To prevent accidental reference.

On the receiving side, a reference image is created for each piece of encoded data that belonged to it before it was created on the transmitting side, a reference image is selected by a frame identification signal, and the selected reference image is used. , And decodes the received encoded data.

In the first aspect of the present invention, the encoded data acquisition section can receive the encoded data to be transmitted. For example, in a case where moving image encoded data is communicated via a relay server using a LAN or the like, if the relay server is provided with the image encoded data creation device according to the present invention, the relay server receives the encoded image data from the encoder. The encoded data is created and transmitted using the encoded data to be transmitted. At the same time, a signal indicating the frame number to be referred to is created and transmitted. The receiving side receives the encoded data created by the relay server, and decodes the encoded data using the image of the frame number designated by the relay server. When the encoded data generated by the relay server is stored in the relay server, the frame number of the referenced image is also stored at the same time. Instead of a frame number,
A reference memory location designation signal may be communicated or stored.
In the second invention, a frame identification signal is communicated or stored instead of the frame number.

Further, in the first invention, the encoded data acquisition section can read the encoded data from a storage medium or a device. For example, new encoded data may be created and stored from encoded video data stored in a CD-ROM or the like. In this case, a signal indicating the frame number to be referred to at the same time is generated and stored. When communicating the encoded data, the transmitting side transmits the stored encoded data and a signal indicating the frame number. The receiving side decodes the received encoded data with reference to the designated frame. Instead of the frame number, a reference memory position designation signal may be communicated or stored. In the second invention, a frame identification signal is communicated or stored instead of the frame number.

Further, in the first invention, the encoded data acquisition section can encode the input image to create encoded data. For example, an image input means such as a camera is provided.
In some cases, encoded data is created and stored using the MPEG-4 image encoding method. In this case, a signal indicating the frame number to be referred to at the same time is generated and stored. When communicating the encoded data, the transmitting side transmits the stored encoded data and a signal indicating the frame number. The receiving side decodes the received encoded data with reference to the designated frame. Instead of the frame number, a reference memory position designation signal may be communicated or stored. In the second invention, a frame identification signal is communicated or stored instead of the frame number.

Further, in the first invention, the encoded data acquisition unit can create encoded data by processing the input encoded data using frame thinning processing, transcoding, or the like. For example, there is a case where encoded data that has been encoded and stored in advance is input, and encoded data subjected to frame thinning processing is generated and stored. The frame thinning processing includes a method of thinning out frames by decoding encoded data once and then changing and encoding a reference image, and a method of thinning out frames without decoding encoded data.

The method of thinning out frames by decoding the encoded data once and then changing and encoding the reference image is to decode a frame from a non-thinned frame to a next non-thinned frame and thin the reference image. This is a method in which encoding is performed again for a frame that does not exist. On the other hand, in a method of thinning out frames without decoding encoded data, an algorithm of an inter-frame differential encoding method is used.

In a general-purpose encoding method such as MPEG-2, inter-frame difference encoding is generally used, and each frame performs inter-frame difference encoding according to the time direction of a frame to be referred to. There are two frames: an intra frame (I frame), a frame that encodes the difference between the previous frame (P frame), and a frame that encodes the difference between the previous frame and the subsequent frame (B frame). Can be classified. Among them, the B frame has the property that the receiving side of the coded data can decode after receiving the data of the frame later than the B frame, and cannot be a reference image of another frame.
The frame thinning process has been performed by simply deleting the data of the B frame itself. On the other hand, since a P frame can serve as a reference image for a subsequent frame, the P frame is not only deleted, but a code that sets a non-deleted frame as a new reference image for a frame that refers to the deleted P frame. Need to process the coded data. Therefore, the DCT of the error information is performed by using the property that the inter-frame difference encoding method encodes the error information between the frames.
A method of thinning out a frame without going through a decoding process such as a method of searching for a motion of a coefficient has also been devised.

In the case of the present embodiment, encoded data is created and stored by performing frame thinning, and at the same time, a signal indicating a frame number to be referred to is generated and stored. When communicating the encoded data, the transmitting side transmits the stored encoded data and a signal indicating the frame number. The receiving side decodes the received encoded data with reference to the designated frame. Instead of the frame number, a reference memory position designation signal may be communicated or stored. In the second invention, a frame identification signal is communicated or stored instead of the frame number.

Of course, it is also possible to apply the above-mentioned coded data acquisition methods in combination. For example, one piece of encoded data can be created from the transmitted encoded data and the accumulated encoded data.

In the first and second aspects of the present invention using the above method, one coded data from a plurality of coded data and a signal specifying a picture to be referred to, or a code belonging to before A signal for identifying the encoded data can be created. Also, a decoder provided with an image encoded data creation device according to the present invention on the transmission side, and capable of storing images for a plurality of frames on the reception side and decoding by referring to the image designated by the transmission side Is provided, the receiving side can decode the encoded data by referring to the frame specified by the transmitting side.

Accordingly, by reducing the number of pieces of encoded data from the transmitting side to the receiving side, a system for performing moving image encoding for each region of an arbitrary shape such as MPEG-4 and communicating the encoded data can be provided. In a system for transmitting and receiving a plurality of encoded data, there is no need to provide a plurality of decoders on the receiving side. Further, in an image communication system provided with a decoder capable of decoding by referring to an image of a designated frame number on the receiving side, the transmitting side stores not only the encoded data encoded in real time but also the accumulated data. The encoded data received and the encoded data received from the transmission path are transmitted and can be decoded on the receiving side.

Each of the above means can be realized by hardware, and a part or all of them can be realized by a combination of a computer and a software program. A program for implementing the above means in combination with a computer can be stored in an appropriate storage medium such as a computer-readable portable medium memory, a semiconductor memory, and a hard disk.

[0040]

DESCRIPTION OF THE PREFERRED EMBODIMENTS [First Embodiment] An example of an image communication system according to a first embodiment of the present invention will be described.

It is assumed that the video is a news video and is composed of a frame composed of an announcer and a background, a field video frame, and a commercial video frame. This system is an image communication system that transmits and receives such images. The transmitting side encodes the announcer and background, receives the on-site video from another on-site relay station, and assumes that the commercial video has been stored in advance. .

In each of the encoded data, the screen size is 176 pixels horizontally and 144 pixels vertically. The encoding method uses MPEG-4, and it is assumed that the first frame is encoded by an I frame and the other frames are encoded by P frames, that is, the immediately preceding frame is used as a reference image. It is assumed that the announcer uses object coding, the shape information of the announcer is also coded, and only the announcer is coded. On the other hand, the background uses an encoding method that does not encode shape information. Similarly, it is assumed that the on-site video and the commercial video use an encoding method in which shape information is not encoded.

In this example, when the background and the frame of the announcer exist at the same time, the frame of the background is arranged first in the encoded data created on the transmitting side, and then the frame of the announcer is arranged. It shall be. On the receiving side, it is assumed that, for a plurality of frames at the same time, those decoded earlier are regarded as the background.

FIG. 1 shows a configuration example of a system according to the present embodiment. In this example, the encoded data acquisition unit inputs a person image input unit 101 for inputting an announcer image, a person image encoding unit 102 for encoding an announcer image, and a background image for a background. A background image input unit 104 and a background image encoding unit 105 for encoding a background image; a site video receiving unit 107 for receiving coded data of a site video for a site video; and a commercial (CM) video for a commercial video CM image storage unit 109 which stores the encoded data of C and reads the stored encoded data.
And an M image reading unit 110.

The coded data separation unit includes a person coded data separation unit 103 for the coded data of the announcer,
Background encoded data separation unit 1 for encoded background data
06, a field coded data separation unit 108 for coded data of a field video, and a CM coded data separation unit 111 for coded data of a commercial video.

The reference frame number creation unit 114 creates a reference frame number for the encoded data of the announcer.
1, F2 for creating a reference frame number for the encoded data of the background, F3 for creating a reference frame number for the encoded data of the field video, and F4 for creating a reference frame number for the encoded data of the commercial video. Be composed.

The coded data merging unit includes a frame changing unit 112 for changing which frame of the coded data of the video is to be input to the coded data transmitting unit 115 using the announcer, the background, the field video, and the commercial. And an encoded data transmission unit 115 for transmitting encoded data.

Further, on the transmitting side, the encoded data transmitting unit 1
15, a frame control unit 113 that counts the number of frames input and controls the frame changing unit 112, a frame number transmission unit 122 that transmits the frame number of the encoded data transmitted by the encoded data transmission unit 115, A display time transmitting unit 124 for transmitting a display time for designating a display time of an image;
15, the reference frame number created by the reference frame number creation unit 114, the frame number transmitted from the frame number transmission unit 122, and the display time transmitted from the display time transmission unit 124. And a transmission-side multiplexing unit 116 for multiplexing.

On the receiving side, a receiving side multiplexing section 117 for separating data received from the transmitting side into encoded data, a reference frame number and a frame number, and a receiving side multiplexing section for dividing the frame number of the encoded data to be received. 117, a display time obtaining unit 126 that obtains the display time of the received encoded data, and a display time match determination that determines whether the display time matches the display time of the previously received frame. Unit 127, an output image storage unit 125 for storing the image decoded by the decoding unit 118, and an image storage unit 119 for storing a plurality of frames of images to be used as reference images.
And In this example, the image storage unit 119 can store four frames of images,
2, M3, M4.

On the transmitting side, each of the encoded data separation units converts the encoded data into each encoded data when the frame control unit 113 changes the frame changing unit 112 and the output of one frame of encoded data is requested. If the data has not been input from the data acquisition unit, the encoded data for one frame is not output, and the frame control unit 113 does not change the frame changing unit 112, and the encoded data is output to the encoded data separation unit. Is input, and held until encoded data for one frame is output. Frame number transmission unit 12
2 counts the number of frames transmitted by the encoded data transmission unit 115 and transmits the number as a frame number. F1, F of the reference frame number creation unit 114
2, F3 and F4, when the frame control unit 113 changes the frame changing unit 112, the frame number transmission unit 12
2. The frame number is obtained from No. 2 and the number is stored. At the time of storage, if a number has already been stored, that number is created as a reference frame number.
It is assumed that the frame number 0 is stored as an initial state. The display time transmitting unit 124 transmits the time displayed on the receiving side.

On the receiving side, the reference frame number receiving unit 12
By changing the reference image changing means 120 so as to refer to the image of the reference frame number received in Step 1,
An image stored in the image storage unit 119 is selected, and the decoding unit 118 uses the selected image as a reference image. When decoding is performed by the decoding unit 118, in this case, since the announcer is coded with an object of an arbitrary shape, the portion where there is no image of the announcer in one screen must be transparent. Is decoded to a signal indicating When the reference frame number is 0, it indicates that nothing is referenced.

When the display time acquired by the display time acquisition unit 126 is different from that of the previous frame, the display time coincidence determination unit 127 transmits a display time mismatch signal to the output image storage unit 125. The output image storage unit 125 creates and stores an output image using the image input from the decoding unit 118 until the display time mismatch signal is received. An image is created by superimposing an image newly input from the decoding unit 118 on an already stored image.

The data multiplexed by the transmitting side multiplexing section 116 is the frame number transmitted by the frame number transmitting section 122, the encoded data transmitted by the encoded data transmitting section 115, and the reference Frame number creation unit 114
, And the display time transmitted from the display time transmitting unit 124. This data format is shown in FIG.

FIG. 3 shows an outline of processing on the transmitting side of the system shown in FIG. When the data to be acquired is a person image of the announcer, the input image is input by the person image input unit 101 and encoded by the person image encoding unit 102 (step S1). The person-encoded data separating unit 103 holds the person-encoded data until one frame of encoded data is output (step S2).

When the data to be acquired is a background image,
The input image is input by the background image input unit 104, and is encoded by the background image encoding unit 105 (step S3). The background coded data separation unit 106 holds the background coded data until one frame of coded data is output (step S4).

If the data to be acquired is field coded data, the coded data of the field video is received by the field video receiver 107 (step S5). The site-encoded data separation unit 108 holds the site-encoded data until one frame of encoded data is output (Step S).
6).

When the data to be acquired is a CM image,
The CM image reading unit 110 reads a CM image from the CM image storage unit 109 (step S7). The CM encoded data separation unit 111 holds the CM encoded data until one frame of encoded data is output (Step S).
8).

Under the control of the frame control unit 113, the coded data to be transmitted is selected by the frame change unit 112, and the frame number is added by the frame number transmission unit 122 (step S9). The reference frame number is added by the reference frame number creation unit 114 (step S
10). Further, the display time transmitting unit 124 adds a time at which the decoded image is displayed at the time of decoding (step S11). The transmitting side multiplexing unit 116 multiplexes these data and transmits the data to the receiving side (step S12).

FIG. 4 shows an outline of processing on the receiving side of the system shown in FIG. On the receiving side, the receiving side multiplexing section 117 receives the encoded data (step S20). The reference frame number receiving unit 121 receives the reference frame number added to the encoded data, and determines whether to refer to an image for decoding the encoded data (Step S21). When referring to an image, the reference image changing unit 120 selects a reference image from the image storage unit 119 (step S22),
The encoded data is decoded by the decoding unit 118 using the reference image (step S23). On the other hand, when the image is not referred to, the decoding unit 118 decodes the encoded data without using the reference image (Step S24).

The decoded image is sent to the frame number receiving unit 12
3 together with the frame number received.
Is stored for reference (step S25). The display time obtaining unit 126 obtains the display time added to the received encoded data, and the display time match determination unit 127 determines whether the display time of the current frame matches the display time of the previous frame ( Step S26). If the display times do not match, the output image stored in the output image storage unit 125 is output (step S27). If the display times match, the output image of the decoding unit 118 is stored in the output image storage unit 125.
The image is accumulated on top of the image accumulated in the image and is not output to the outside (step S28).

Next, an operation example of the system shown in FIG. 1 will be described according to a specific example. In the following example, the announcer and the background video have a total of 3 frames, followed by the on-site video for a total of 2 frames, the commercial video has a total of 2 frames, the on-site video has a total of 2 frames, and the announcer and the The processing performed when there are a total of three frames of background video is shown. Although the announcer and the background video are transmitted twice, it is assumed that the breakdown includes one frame of the background video and two frames of the announcer thereafter. FIG. 5 shows an operation flow on the transmission side and the reception side in the present embodiment. The processing executed when communicating each frame is represented by STEP. In addition, the image of the announcer and the image of the background are superimposed and displayed.
The display order of each video is specified in advance.

In STEP 1, the first frame is
On the transmitting side, the frame control unit 113 changes the frame changing unit 112 to the background coded data separation unit 106 to transmit the coded background data. When the encoded data is input from the background encoded data separation unit 106 to the encoded data transmission unit 115, the encoded data transmission unit 115
The input coded data is transmitted to the transmitting side multiplexing unit 116. The display time of this frame is set to 1. The frame number transmitting section 122 transmits the frame number 1 to the transmitting side multiplexing section 1
16 to be transmitted. The reference frame number creation unit 114
The transmission side multiplexing section 116 stores the frame number 0 stored in
And the frame number 1 is stored in F2. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 0, the reference image changing unit 120
Do not change The decoding unit 118 decodes the encoded data without using the reference image. The decoded image is stored in the image storage unit 119.
And sends it to the output image storage unit 125. The display time acquisition unit 126 receives the display time 1 and sends it to the display time match determination unit 127. Display time match determination unit 127
Does not transmit the display time mismatch signal because the display time is not compared for the first frame. Output image storage unit 1
Reference numeral 25 stores the transmitted image.

In STEP 2, the second frame is
On the transmitting side, the frame control unit 113 transmits the coded data of the announcer.
To the person-encoded data separation unit 103. When the encoded data is input from the encoded person data separating unit 103 to the encoded data transmitting unit 115, the encoded data transmitting unit 1
Reference numeral 15 denotes a transmitting side multiplexing unit 11
Send to 6. The display time of this frame is set to 1. The frame number transmitting section 122 transmits the frame number 2 to the transmitting side multiplexing section 116. Reference frame number creation unit 11
4 transmits the frame number 0 stored in F1 to the transmission side multiplexing unit 116, and stores the frame number 2 in F1.
The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 0, the reference image changing unit 120
Do not change The decoding unit 118 decodes the encoded data without using the reference image. The decoded image is stored in the image storage unit 119.
And transmits it to the output image storage unit 125. The display time acquisition unit 126 receives the display time 1 and sends it to the display time match determination unit 127. Display time match determination unit 127
Compares the display time 1 of the previous frame with the received display time 1 and does not transmit a display time mismatch signal. The output image storage unit 125 creates and stores an image by superimposing the received image on the already stored image.

At STEP 3, the third frame is
On the transmitting side, the frame control unit 113 transmits the coded data of the announcer.
To the person-encoded data separation unit 103. When the encoded data is input from the encoded person data separating unit 103 to the encoded data transmitting unit 115, the encoded data transmitting unit 1
Reference numeral 15 denotes a transmitting side multiplexing unit 11
Send to 6. The display time of this frame is 2. The frame number transmitting unit 122 transmits the frame number 3 to the transmitting side multiplexing unit 116. Reference frame number creation unit 11
4 transmits the frame number 2 stored in F1 to the transmission-side multiplexing unit 116, and stores the frame number 3 in F1.
The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 2, the reference image changing unit 120
Is changed to M2 in which the frame number 2 is stored. The decoding unit 118 decodes the encoded data stored in M2 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M2 of the image storage unit 119 and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 2 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 1 of the previous frame with the received display time 2 and transmits a display time mismatch signal. The output image storage unit 125 outputs the already stored image and stores the received image.

In STEP 4, the fourth frame is
On the transmission side, the frame control unit 113 changes the frame changing unit 112 to the site encoded data separation unit 108 to transmit the encoded data of the site video. When the encoded data is input from the in-situ encoded data separation unit 108 to the encoded data transmission unit 115, the encoded data transmission unit 115
Transmits the input coded data to the transmission side multiplexing unit 116. Assume that the display time of this frame is 3. The frame number transmitting unit 122 transmits the frame number 4 to the transmitting side multiplexing unit 116. Reference frame number creation unit 114
Transmits the frame number 0 stored in F3 to the transmission-side multiplexing unit 116, and stores the frame number 4 in F3. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 0, the reference image changing unit 120
Do not change The decoding unit 118 decodes the encoded data without using the reference image. The decoded image is stored in the image storage unit 119.
And transmits it to the output image storage unit 125. The display time acquisition unit 126 receives the display time 3 and transmits it to the display time match determination unit 127. Display time match determination unit 127
Compares the display time 2 of the previous frame with the received display time 3 and transmits a display time mismatch signal. Output image storage unit 1
Reference numeral 25 outputs the already stored image and stores the received image.

At STEP 5, for the fifth frame,
On the transmission side, the frame control unit 113 changes the frame changing unit 112 to the site encoded data separation unit 108 to transmit the encoded data of the site video. When the encoded data is input from the in-situ encoded data separation unit 108 to the encoded data transmission unit 115, the encoded data transmission unit 115
Transmits the input coded data to the transmission side multiplexing unit 116. The display time of this frame is 4. The frame number transmitting section 122 transmits the frame number 5 to the transmitting side multiplexing section 116. Reference frame number creation unit 114
Transmits the frame number 4 stored in F3 to the transmission-side multiplexing unit 116, and stores the frame number 5 in F3. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 4, the reference image changing unit 120
Is changed to M3 in which the frame number 4 is stored. The decoding unit 118 decodes the encoded data stored in M3 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M3 of the image storage unit 119 and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 4 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 3 of the previous frame with the received display time 4 and transmits a display time mismatch signal. The output image storage unit 125 outputs the already stored image and stores the received image.

At STEP 6, for the sixth frame,
On the transmission side, the frame control unit 113 transmits the encoded data of the commercial video,
12 is changed to the CM encoded data separation unit 111. CM
When the encoded data is input from the encoded data separation unit 111 to the encoded data transmission unit 115, the encoded data transmission unit 115 transmits the input encoded data to the transmission side multiplexing unit 116. Assume that the display time of this frame is 5. The frame number transmitting unit 122 transmits the frame number 6 to the transmitting side multiplexing unit 116. The reference frame number creation unit 114 transmits the frame number 0 stored in F4 to the transmission-side multiplexing unit 116, and stores the frame number 6 in F4. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 0, the reference image changing unit 120
Do not change The decoding unit 118 decodes the encoded data without using the reference image. The decoded image is stored in the image storage unit 119.
And sends it to the output image storage unit 125. The display time acquisition unit 126 receives the display time 5 and transmits it to the display time match determination unit 127. Display time match determination unit 127
Compares the display time 4 of the previous frame with the received display time 5 and transmits a display time mismatch signal. Output image storage unit 1
Reference numeral 25 outputs the already stored image and stores the received image.

At STEP 7, for the seventh frame,
On the transmission side, the frame control unit 113 transmits the encoded data of the commercial video,
12 is changed to the CM encoded data separation unit 111. CM
When the encoded data is input from the encoded data separation unit 111 to the encoded data transmission unit 115, the encoded data transmission unit 115 transmits the input encoded data to the transmission side multiplexing unit 116. Assume that the display time of this frame is 6. The frame number transmitting unit 122 transmits the frame number 7 to the transmitting side multiplexing unit 116. The reference frame number creation unit 114 transmits the frame number 6 stored in F4 to the transmission-side multiplexing unit 116, and stores the frame number 7 in F4. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 6, the reference image changing unit 120
Is changed to M4 in which the frame number 6 is stored. The decoding unit 118 decodes the encoded data stored in M4 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M4 of the image storage unit 119 and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 6 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 5 of the previous frame with the received display time 6 and transmits a display time mismatch signal. The output image storage unit 125 outputs the already stored image and stores the received image.

At STEP 8, for the eighth frame,
On the transmission side, the frame control unit 113 changes the frame changing unit 112 to the site encoded data separation unit 108 to transmit the encoded data of the site video. When the encoded data is input from the in-situ encoded data separation unit 108 to the encoded data transmission unit 115, the encoded data transmission unit 115
Transmits the input coded data to the transmission side multiplexing unit 116. Assume that the display time of this frame is 7. The frame number transmission unit 122 transmits the frame number 8 to the transmission side multiplexing unit 116. Reference frame number creation unit 114
Transmits the frame number 5 stored in F3 to the transmission-side multiplexing unit 116, and stores the frame number 8 in F3. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 5, the reference image changing unit 120
Is changed to M3 in which the frame number 5 is stored. The decoding unit 118 decodes the encoded data stored in M3 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M3 of the image storage unit 119 and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 7 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 6 of the previous frame with the received display time 7 and transmits a display time mismatch signal. The output image storage unit 125 outputs the already stored image and stores the received image.

At STEP 9, for the ninth frame,
On the transmission side, the frame control unit 113 changes the frame changing unit 112 to the site encoded data separation unit 108 to transmit the encoded data of the site video. When the encoded data is input from the in-situ encoded data separation unit 108 to the encoded data transmission unit 115, the encoded data transmission unit 115
Transmits the input coded data to the transmission side multiplexing unit 116. Assume that the display time of this frame is 8. The frame number transmitting unit 122 transmits the frame number 9 to the transmitting side multiplexing unit 116. Reference frame number creation unit 114
Transmits the frame number 8 stored in F3 to the transmission-side multiplexing unit 116, and stores the frame number 9 in F3. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 8, the reference image changing unit 120
Is changed to M3 in which the frame number 8 is stored. The decoding unit 118 decodes the encoded data stored in M3 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M3 of the image storage unit 119 and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 8 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 7 of the previous frame with the received display time 8, and transmits a display time mismatch signal. The output image storage unit 125 outputs the already stored image and stores the received image.

At STEP 10, on the transmitting side, the frame control unit 113 transmits the coded data of the background image to the frame changing unit 112 for the tenth frame.
To the background coded data separation unit 106. When the encoded data is input from the background encoded data separation unit 106 to the encoded data transmission unit 115, the encoded data transmission unit 1
Reference numeral 15 denotes a transmitting side multiplexing unit 11
Send to 6. Assume that the display time of this frame is 9. The frame number transmitting unit 122 transmits the frame number 10 to the transmitting side multiplexing unit 116. Reference frame number creation unit 1
14 transmits the frame number 1 stored in F2 to the transmission-side multiplexing unit 116, and stores the frame number 10 in F2. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 1, the reference image changing unit 120
Is changed to M1 in which the frame number 1 is stored. The decoding unit 118 decodes the encoded data stored in M1 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M1 of the image storage unit 119, and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 9 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 8 of the previous frame with the received display time 9 and transmits a display time mismatch signal. The output image storage unit 125 outputs the already stored image and stores the received image.

At STEP 11, on the transmitting side, the frame control unit 113 changes the frame changing unit 112 to the person-encoded data separating unit 103 for transmitting the coded data of the announcer video for the eleventh frame.
When the encoded data is input to the encoded data transmitting unit 115 from the person encoded data separating unit 103, the encoded data transmitting unit 115 transmits the input encoded data to the transmitting side multiplexing unit 116. Assume that the display time of this frame is 9. The frame number transmitting unit 122 outputs the frame number 11
Is transmitted to the transmission side multiplexing section 116. The reference frame number creation unit 114 transmits the frame number 3 stored in F1 to the transmission side multiplexing unit 116, and stores the frame number 11 in F1.
To accumulate. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 3, the reference image changing unit 120
Is changed to M2 in which the frame number 3 is stored. The decoding unit 118 decodes the encoded data stored in M2 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M2 of the image storage unit 119 and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 9 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 9 of the previous frame with the received display time 9 and does not transmit a display time mismatch signal. The output image storage unit 125 creates and stores an image by superimposing the received image on the already stored background image.

At STEP 12, on the transmitting side, the frame control section 113 changes the frame changing section 112 to the person-encoded data separating section 103 for transmitting the coded data of the announcer video for the twelfth frame.
When the encoded data is input to the encoded data transmitting unit 115 from the person encoded data separating unit 103, the encoded data transmitting unit 115 transmits the input encoded data to the transmitting side multiplexing unit 116. Set the display time of this frame to 10
And The frame number transmitting unit 122 outputs the frame number 1
2 to the transmission-side multiplexing section 116. The reference frame number creation unit 114 stores the frame number 11 stored in F1.
Is transmitted to the transmission-side multiplexing unit 116, and the frame number 12 is stored in F1. The transmitting-side multiplexing unit 116 multiplexes the frame number, the encoded data, the reference frame number, and the display time, and transmits the multiplexed data to the receiving side.

On the receiving side, the receiving side multiplexing section 117 outputs
The frame number receiving section 123 stores the frame number, the reference frame number receiving section 121 stores the reference frame number, and the decoding section 1
Reference numeral 18 denotes the encoded data, and the display time acquisition unit 126 receives the display time. Since the reference frame number receiving unit 121 receives the reference frame number 11, the reference image changing unit 12
0 is changed to M2 in which the frame number 11 is stored. The decoding unit 118 decodes the encoded data stored in M2 of the image storage unit 119 by using the encoded data as a reference image. The decoded image is stored in M2 of the image storage unit 119 and transmitted to the output image storage unit 125. The display time acquisition unit 126 receives the display time 10 and transmits it to the display time match determination unit 127. The display time match determination unit 127 compares the display time 9 of the previous frame with the received display time 10 and transmits a display time mismatch signal. The output image storage unit 125 outputs the already stored image and stores the received image.

In the above, for the sake of simplicity, an example using the MPEG-4 encoding method has been described. It goes without saying that any method can be applied in the same manner.

As described above, according to the present embodiment, the encoded data of the background, the encoded data of the announcer,
The encoded data of the on-site video and the encoded data of the commercial video are communicated as one encoded data, and the receiving side can decode the reference image without error. Furthermore, while the encoded data of the background and the announcer are transmitted as separate frames, they can be output without deteriorating the image quality by combining them at the time of output.

Further, in this example, the frame number of the encoded data is used to specify the reference image. Can perform the same processing.
In this case, on the transmission side, the reference frame number creation unit 114
Instead of storing the frame number, the memory position of the image storage unit 119 on the receiving side is stored and transmitted every time a frame is created. On the receiving side, decoding is performed by referring to the image at the memory location to be received in the image storage unit 119. Note that the transmitting side can acquire the memory position of the image storage unit 119 of the receiving side by having the receiving side transmit the memory position in advance.

[Second Embodiment] Next, a second embodiment of the present invention will be described.
An example of the image communication system according to the embodiment will be described.

The reference data is changed by performing frame thinning-out processing on the stored coded data of the movie to reconstruct the coded data with enhanced error resilience. This is an example of a case where deterioration of image quality is reduced. First, two pieces of encoded data are created from the same encoded data by using frame thinning processing.
At this time, the time of each frame is not made to match in each of the encoded data.

When the receiving side receives such coded data, the receiving side decodes the coded data using the image specified by the frame identification signal among the images stored in the image storage section as a reference image. If an error occurs on the transmission line and a certain frame cannot be decoded correctly, the same frame identification signal is received, and a frame referring to the same cannot be decoded correctly thereafter. However, frames that receive a different frame identification signal and do not reference it can still be decoded correctly. As described above, the error resilience is enhanced by changing the reference image of the encoded data.

In this example, the screen size of the stored encoded data is QCIF. The encoding method is H.264. 2
63, the first frame is encoded by an I frame, and the other frames are encoded by P frames. That is, it is assumed that the immediately preceding frame is used as the reference image.

[0093] Two encoded data A and B are created from the accumulated encoded data. It is assumed that the odd-numbered frames of the encoded data A are thinned out. Encoded data B
Means that even-numbered frames are thinned out.

FIG. 6 shows a configuration example of a system according to the present embodiment. In this example, the coded data acquisition unit performs frame culling of the coded data A by a frame culling unit A4.
02, and a frame thinning unit B406 that thins out frames of the encoded data B. The encoded data separation unit A4 encodes the encoded data A
03 and the encoded data separation unit B for the encoded data B
407. Frame identification signal creation section 411
Is composed of F1 for creating and storing a frame identification signal of the encoded data A, and F2 for creating and storing a frame identification signal of the encoded data B. The coded data merging unit includes a coded frame changing unit 404 that changes which of the coded data A and the coded data B is to be input to the coded data transmitting unit 405;
It comprises an encoded data transmission unit 405 for transmitting encoded data.

Further, the transmission side counts the number of frames input to the input coded data storage section 401 storing the input coded data and the coded data transmission section 405,
A frame control unit 410 for controlling the encoded frame changing unit 404, and a frame thinning signal transmitting unit 409 for transmitting a frame thinning signal for designating frame thinning
, A frame thinning change unit 408 for changing whether the frame thinning signal is transmitted to the frame thinning unit A402 or the frame thinning unit B406, coded data transmitted by the coded data transmitting unit 405, and frame identification. The transmission multiplexing unit 413 multiplexes the frame identification signal created by the signal creation unit 411.

The receiving side multiplexing section 414 for separating the data received from the transmitting side into encoded data and a frame identification signal, and an error detection section for detecting whether or not the received encoded data contains an error. A storage unit 415 and an image storage unit 418 for storing a plurality of frames of images to be used as reference images. In this example, the image storage unit 418 has a reference image buffer that stores images for two frames, and sets them as M1 and M2. It is assumed that M1 stores a decoded image of the encoded data A and uses it as a reference image of the encoded data A. M
2 stores a decoded image of the encoded data B and uses it as a reference image of the encoded data B.

On the transmitting side, the frame thinning unit A 402 and the frame thinning unit B 406 read and store the coded data from the input coded data storage unit 401 frame by frame. When either one receives the frame thinning signal, it performs frame thinning using the stored encoded data. It is assumed that the side receiving the frame thinning signal together with the encoded data performs the process of thinning the frame. That is, the frame thinning unit transmits the next frame to the encoded data separating unit as encoded data obtained by thinning the frame. When thinning out the first I frame, the next frame is to be intra-coded. Each of the encoded data separation units includes a frame control unit 41
0 changes the encoding frame changing means 404,
When output of one frame of encoded data is required,
If the coded data is not input from each frame thinning unit, one frame of coded data is not output.
It is also assumed that the frame control unit 410 does not change the coded frame changing unit 404, and the coded data is input to the coded data separation unit and held until one frame of coded data is output. Frame identification signal creation section 411
It is assumed that A representing the encoded data A is created and stored in F1 of B, and B representing the encoded data B is created and stored in F2. The frame thinning signal transmission unit 409 outputs
It is assumed that a frame thinning signal is being transmitted. In the input coded data storage unit 401, it is assumed that the first frame is an intra-coded I frame and the other frames are coded as P frames.

On the receiving side, the frame identification signal receiving section 41
The image stored in the image storage unit 418 is selected by changing the reference image change unit 417 so as to refer to the image stored in the reference image buffer indicated by the frame identification signal received in Step 2. The decoding unit 416 decodes the encoded data with reference to the image. In addition, the error detection unit 4
When an error is detected in the encoded data received at 15,
When receiving the frame identification signal A in a subsequent frame, the frame identification signal receiving unit 412
A signal indicating that the encoded data is erroneous is transmitted to 15 so that the error detector 415 does not transmit the encoded data to the decoder 416.

The data multiplexed by the transmission-side multiplexing unit 413 becomes coded data transmitted by the coded data transmitting unit 405, and subsequently becomes a frame identification signal transmitted from the frame identification signal creating unit 411. This data format is shown in FIG. As a frame identification number in FIG. 7 which is a frame identification signal, bit 1 is arranged instead of A in the case of encoded data A, and bit 0 is arranged in place of B in the case of encoded data B. I do.

FIG. 8 shows an outline of processing on the transmitting side of the system shown in FIG. The frame thinning unit A402 inputs the coded data of the n-th frame from the input coded data storage unit 410 (Step S30). When the frame thinning signal is transmitted from the frame thinning signal transmitting unit 409 via the frame thinning change unit 408, the process proceeds to the input of the next frame (step S31). If the frame thinning-out signal has not been sent, the previous (n-1) th frame is thinned out (step S32), and the encoded data separating unit A403
Sends the coded data A of the n-th frame to the coded data transmitting unit 405 through the coded frame changing unit 404 (step S33). Similarly, the frame thinning unit B406 and the encoded data separating unit B407 perform frame thinning and transmission of the encoded data B to the encoded data transmitting unit 405 (steps S34 to S37).

Under the control of the frame control unit 410, the frame identification signal creation unit 411 creates the frame identification signal A or B and adds it to the encoded data (step S).
38). The transmission-side multiplexing unit 413 multiplexes the coded data from the coded data transmission unit 405 and the frame identification signal and transmits the multiplexed data to the receiving side (step S39).

FIG. 9 shows an outline of processing on the receiving side of the system shown in FIG. On the receiving side, the receiving side multiplexing unit 414 receives the encoded data (step S40). The error detection unit 415 determines whether the received encoded data has an error (step S41). If an error is detected,
Discard the received encoded data. If no error is detected, it is determined whether to refer to the image for decoding the encoded data (step S42). When referring to an image, the reference image changing unit 417 stores the reference image in the image storage unit 41.
8 (Step S43), and the decoding unit 416 decodes the encoded data using the reference image (Step S44). The selection of the reference image is performed based on the frame identification signal received by the frame identification signal receiving unit 412. On the other hand, when the image is not referred to, the decoding unit 416 decodes the encoded data without using the reference image (Step S45).

The image decoded by the decoding unit 416 is stored in the image storage unit 418 for reference in accordance with the frame identification signal received by the frame identification signal receiving unit 412 (step S46). The image decoded by the decoding unit 416 is output as an output image (step S47).

Next, an operation example of the system shown in FIG. 6 will be described according to a specific example. Hereinafter, an example will be described in which the first to seventh frames of the images stored in the input coded data storage unit 401 are transmitted and received. It is assumed that a transmission error occurs in the fourth frame in the middle. FIG. 10 shows an operation flow of the transmission side and the reception side of the present system. The processing executed when communicating each frame is represented by STEP.

At STEP 1, for the first frame,
On the transmitting side, the frame control unit 410 changes the frame thinning change unit 408 so that the frame thinning signal is transmitted to the frame thinning unit A402. Further, in order to transmit the frame of the coded data B, the coded frame changing unit 404 is changed to a coded data separation unit B407. Frame thinning unit A402 and frame thinning unit B4
Reference numeral 06 reads encoded data for one frame from the input encoded data accumulation unit 401 and accumulates the encoded data. Since the frame thinning unit A402 receives the frame thinning signal at the same time, the frame thinning unit B406 transmits the encoded data to the encoded data separating unit B407. The encoded data transmission unit 405 receives the encoded data of the encoded data B from the encoded data separation unit B407. Encoded data transmission unit 4
In step 05, the encoded data is transmitted to the transmission-side multiplexing section 413, and the frame identification signal creation section 411 transmits the frame identification signal B stored in F2 to the transmission-side multiplexing section 413. The transmission-side multiplexing unit 413 multiplexes the coded data and the frame identification signal B, and transmits the multiplexed data to the reception side.

On the receiving side, the receiving side multiplexing section 414 outputs
The frame identification signal receiving section 412 receives the frame identification signal B
And the error detection unit 415 receives the encoded data. Since the frame identification signal receiving unit 412 receives the frame identification signal B, it refers to M2 of the image storage unit 418,
The reference image changing means 417 is changed. Error detector 415
Does not detect an error, the encoded data is decoded by the decoding unit 416.
Send to Since the encoded data is an I frame, the decoding unit 416 decodes the image without referring to the image in the image storage unit 418. The decoded image is stored in M2, which is a reference image buffer for encoded data B, and output.

In STEP 2, the second frame is
On the transmitting side, the frame control unit 410 changes the frame thinning change unit 408 so that the frame thinning signal is transmitted to the frame thinning unit B406. Further, in order to transmit the frame of the encoded data A, the encoded frame changing unit 404 is changed to the encoded data separating unit A403. Frame thinning unit A402 and frame thinning unit B4
Reference numeral 06 reads encoded data for one frame from the input encoded data accumulation unit 401 and accumulates the encoded data. Since the frame thinning unit B406 receives the frame thinning signal at the same time, the frame thinning unit A402 performs the first frame thinning process. That is, encoded data is generated by intra-encoding the second frame. After creation, the coded data is transmitted to the coded data separation unit A403. The encoded data transmission unit 405 receives the encoded data of the encoded data A from the encoded data separation unit A403. The encoded data transmitting unit 405 transmits the encoded data to the transmitting side multiplexing unit 413.
And the frame identification signal creation unit 411 sends the frame identification signal A stored in F1 to the transmission side multiplexing unit 413. The transmission-side multiplexing unit 413 multiplexes the coded data and the frame identification signal A and transmits the multiplexed data to the reception side.

On the receiving side, the receiving side multiplexing section 414 outputs
The frame identification signal receiving section 412 outputs the frame identification signal A
And the error detection unit 415 receives the encoded data. Since the frame identification signal receiving unit 412 receives the frame identification signal A, it refers to M1 of the image storage unit 418,
The reference image changing means 417 is changed. Error detector 415
Does not detect an error, the encoded data is decoded by the decoding unit 416.
Send to Since the encoded data is an I frame, the decoding unit 416 decodes the image without referring to the image in the image storage unit 418. The decoded image is stored in M1 which is a reference image buffer for encoded data A, and output.

In SETP3, for the third frame,
On the transmitting side, the frame control unit 410 changes the frame thinning change unit 408 so that the frame thinning signal is transmitted to the frame thinning unit A402. Further, in order to transmit the frame of the coded data B, the coded frame changing unit 404 is changed to a coded data separation unit B407. Frame thinning unit A402 and frame thinning unit B4
Reference numeral 06 reads encoded data for one frame from the input encoded data accumulation unit 401 and accumulates the encoded data. Since the frame thinning unit A402 receives the frame thinning signal at the same time, the frame thinning unit B406 performs the second frame thinning process. That is, the third frame referring to the first frame
Create encoded data for the frame. After creation, the coded data is transmitted to the coded data separation unit B407. The encoded data transmission unit 405 receives the encoded data of the encoded data B from the encoded data separation unit B407. The encoded data transmitting unit 405 transmits the encoded data to the transmitting side multiplexing unit 4.
13 and the frame identification signal creation section 411 sends the frame identification signal B stored in F2 to the transmission side multiplexing section 41.
Send to 3. The transmission-side multiplexing unit 413 multiplexes the coded data and the frame identification signal B, and transmits the multiplexed data to the reception side.

On the receiving side, the receiving side multiplexing section 414 outputs
The frame identification signal receiving section 412 receives the frame identification signal B
And the error detection unit 415 receives the encoded data. Since the frame identification signal receiving unit 412 receives the frame identification signal B, it refers to M2 of the image storage unit 418,
The reference image changing means 417 is changed. Error detector 415
Does not detect an error, the encoded data is decoded by the decoding unit 416.
Send to The decoding unit 416 decodes the encoded data with reference to the image stored in M2. The decoded image is stored in M2, which is a reference image buffer for decoded data B, and output.

In STEP 4, for the fourth frame,
On the transmitting side, the frame control unit 410 changes the frame thinning change unit 408 so that the frame thinning signal is transmitted to the frame thinning unit B406. Further, in order to transmit the frame of the encoded data A, the encoded frame changing unit 404 is changed to the encoded data separating unit A403. Frame thinning unit A402 and frame thinning unit B4
Reference numeral 06 reads encoded data for one frame from the input encoded data accumulation unit 401 and accumulates the encoded data. Since the frame thinning unit B406 receives the frame thinning signal at the same time, the frame thinning unit A402 performs the third frame thinning process. That is, the fourth frame with reference to the second frame
Create encoded data for the frame. After creation, the coded data is transmitted to the coded data separation unit A403. The encoded data transmission unit 405 receives the encoded data of the encoded data A from the encoded data separation unit A403. The encoded data transmitting unit 405 transmits the encoded data to the transmitting side multiplexing unit 4.
13 and the frame identification signal creation unit 411 transmits the frame identification signal A stored in F2 to the transmission side multiplexing unit 41.
Send to 3. The transmission-side multiplexing unit 413 multiplexes the coded data and the frame identification signal A and transmits the multiplexed data to the reception side.

On the receiving side, the receiving side multiplexing section 414 outputs
The frame identification signal receiving section 412 outputs the frame identification signal A
And the error detection unit 415 receives the encoded data. Since the frame identification signal receiving unit 412 receives the frame identification signal A, it refers to M1 of the image storage unit 418,
The reference image changing means 417 is changed. Error detector 415
Does not transmit the encoded data to the decoding unit 416 because it detects an error. In the subsequent frames, when receiving the frame identification signal A, the frame identification signal receiving section 412 transmits a signal indicating that the encoded data is erroneous to the error detecting section 415, and sends the signal to the error detecting section 415. The encoded data is prevented from being transmitted to the decoding unit 416. No output image is output.

In STEP 5, for the fifth frame,
On the transmitting side, the frame control unit 410 changes the frame thinning change unit 408 so that the frame thinning signal is transmitted to the frame thinning unit A402. Further, in order to transmit the frame of the coded data B, the coded frame changing unit 404 is changed to a coded data separation unit B407. Frame thinning unit A402 and frame thinning unit B4
Reference numeral 06 reads encoded data for one frame from the input encoded data accumulation unit 401 and accumulates the encoded data. Since the frame thinning unit A402 receives the frame thinning signal at the same time, the frame thinning unit B406 performs the fourth frame thinning process. That is, the fifth frame with reference to the third frame
Create encoded data for the frame. After creation, the coded data is transmitted to the coded data separation unit B407. The encoded data transmission unit 405 receives the encoded data of the encoded data B from the encoded data separation unit B407. The encoded data transmitting unit 405 transmits the encoded data to the transmitting side multiplexing unit 4.
13 and the frame identification signal creation section 411 sends the frame identification signal B stored in F2 to the transmission side multiplexing section 41.
Send to 3. The transmission-side multiplexing unit 413 multiplexes the coded data and the frame identification signal B, and transmits the multiplexed data to the reception side.

On the receiving side, the receiving side multiplexing section 414
The frame identification signal receiving section 412 receives the frame identification signal B
And the error detection unit 415 receives the encoded data. Since the frame identification signal receiving unit 412 receives the frame identification signal B, it refers to M2 of the image storage unit 418,
The reference image changing means 417 is changed. Error detector 415
Does not detect an error, the encoded data is decoded by the decoding unit 416.
Send to The decoding unit 416 decodes the encoded data with reference to the image stored in M2. The decoded image is stored in M2 which is a reference image buffer for encoded data B,
Output.

In STEP 6, for the sixth frame,
On the transmitting side, the frame control unit 410 changes the frame thinning change unit 408 so that the frame thinning signal is transmitted to the frame thinning unit B406. Further, in order to transmit the frame of the encoded data A, the encoded frame changing unit 404 is changed to the encoded data separating unit A403. Frame thinning unit A402 and frame thinning unit B4
Reference numeral 06 reads encoded data for one frame from the input encoded data accumulation unit 401 and accumulates the encoded data. Since the frame thinning unit B406 receives the frame thinning signal at the same time, the frame thinning unit A402 performs the fifth frame thinning process. That is, the sixth frame referring to the fourth frame
Create encoded data for the frame. After creation, the coded data is transmitted to the coded data separation unit A403. The encoded data transmission unit 405 receives the encoded data of the encoded data A from the encoded data separation unit A403. The encoded data transmitting unit 405 transmits the encoded data to the transmitting side multiplexing unit 4.
13 and the frame identification signal creation unit 411 transmits the frame identification signal A stored in F1 to the transmission side multiplexing unit 41.
Send to 3. The transmission-side multiplexing unit 413 multiplexes the coded data and the frame identification signal A and transmits the multiplexed data to the reception side.

On the receiving side, the receiving side multiplexing section 414 outputs
The frame identification signal receiving section 412 outputs the frame identification signal A
And the error detection unit 415 receives the encoded data. Since frame identification signal receiving section 412 receives frame identification signal A, it transmits a signal indicating that the encoded data is erroneous to error detecting section 415, and transmits the encoded data to error detecting section 415 to decoding section 416. Do not let. No output image is output.

At STEP 7, for the seventh frame,
On the transmitting side, the frame control unit 410 changes the frame thinning change unit 408 so that the frame thinning signal is transmitted to the frame thinning unit A402. Further, in order to transmit the frame of the coded data B, the coded frame changing unit 404 is changed to a coded data separation unit B407. Frame thinning unit A402 and frame thinning unit B4
Reference numeral 06 reads encoded data for one frame from the input encoded data accumulation unit 401 and accumulates the encoded data. Since the frame thinning unit A402 receives the frame thinning signal at the same time, the frame thinning unit B406 performs the sixth frame thinning process. That is, the seventh frame referring to the fifth frame
Create encoded data for the frame. After creation, the coded data is transmitted to the coded data separation unit B407. The encoded data transmission unit 405 receives the encoded data of the encoded data B from the encoded data separation unit B407. The encoded data transmitting unit 405 transmits the encoded data to the transmitting side multiplexing unit 4.
13 and the frame identification signal creation section 411 sends the frame identification signal B stored in F2 to the transmission side multiplexing section 41.
Send to 3. The transmission-side multiplexing unit 413 multiplexes the coded data and the frame identification signal B, and transmits the multiplexed data to the reception side.

On the receiving side, the receiving side multiplexing section 414 outputs
The frame identification signal receiving section 412 receives the frame identification signal B
And the error detection unit 415 receives the encoded data. Since the frame identification signal receiving unit 412 receives the frame identification signal B, it refers to M2 of the image storage unit 418,
The reference image changing means 417 is changed. Error detector 415
Does not detect an error, the encoded data is decoded by the decoding unit 416.
Send to The decoding unit 416 decodes the encoded data with reference to the image stored in M2. The decoded image is stored in M2 which is a reference image buffer for encoded data B,
Output.

In the above, for simplicity, H.264 is used in this example.
Although the H.263 coding method was used, the coding method is not limited to this, and it goes without saying that any coding method that performs coding in frame units is similarly applicable.

As described above, according to the present embodiment, frame thinning processing is performed on encoded data to generate two types of encoded data, and these are newly formed into one encoded data. Even if a transmission error appears in one of the encoded data, decoding of the other encoded data can be continued. Thus, encoded data with improved error resilience can be created.

[0121]

According to the conventional method, in a system such as MPEG-4 which performs moving image coding for each area of an arbitrary shape and communicates coded data, and a system which transmits and receives a plurality of coded data, On the side, it was necessary to provide a plurality of decoders for decoding the respective encoded data.

However, according to the present invention, a decoder having a reference image accumulation buffer for accumulating a plurality of frames is provided.
When a plurality of encoded data are obtained by a method of encoding an input image, a method of receiving encoded data, a method of reading from a storage medium, or the like, one encoded data that can be decoded by the same decoder is obtained. And only one decoder is needed. Further, on the receiving side, an image communication device provided with a decoder capable of decoding with reference to an image of a specified frame number or a decoder capable of decoding with reference to an image of a specified frame identification signal In the system, the transmitting side transmits not only the encoded data encoded in real time but also the accumulated encoded data and the encoded data received from the transmission path so that the receiving side can decode the data. Become.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a configuration example of an image communication system according to a first embodiment.

FIG. 2 is a diagram showing a data format of communication data according to the first embodiment.

FIG. 3 is a diagram showing an outline of processing on a transmission side of the system shown in FIG. 1;

FIG. 4 is a diagram showing an outline of processing on a receiving side of the system shown in FIG. 1;

FIG. 5 is a diagram illustrating an operation example of a transmission side and a reception side of the system illustrated in FIG. 1;

FIG. 6 is a diagram illustrating a configuration example of an image communication system according to a second embodiment;

FIG. 7 is a diagram showing a data format of communication data according to a second embodiment.

FIG. 8 is a diagram showing an outline of processing on a transmission side of the system shown in FIG. 6;

FIG. 9 is a diagram showing an outline of processing on a receiving side of the system shown in FIG. 6;

FIG. 10 is a diagram illustrating an operation example of a transmission side and a reception side of the system illustrated in FIG. 6;

[Explanation of symbols]

 Reference Signs List 101 person image input unit 102 person image encoding unit 103 person encoded data separation unit 104 background image input unit 105 background image encoding unit 106 background encoded data separation unit 107 on-site video reception unit 108 on-site encoded data separation unit 109 CM Image storage unit 110 CM image reading unit 111 CM encoded data separation unit 112 Frame changing unit 113 Frame control unit 114 Reference frame number creation unit 115 Encoded data transmission unit 116 Transmission side multiplexing unit 117 Reception side multiplexing unit 118 Decoding unit 119 Image storage unit 120 Reference image change unit 121 Reference frame number receiving unit 122 Frame number transmitting unit 123 Frame number receiving unit 124 Display time transmitting unit 125 Output image storing unit 126 Display time acquiring unit 127 Display time matching determining unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 KK39 LB07 MA00 MA05 MB01 MB21 RB01 RC04 RC11 RC22 SS11 SS20 TA23 TC24 UA02 UA05 UA33 UA35 UA38 UA39 5K022 FF00

Claims (23)

[Claims] 1. An image encoding data creation method for creating moving image encoded data encoded in units of frames using an inter-frame differential encoding method, wherein a plurality of encoded data items are acquired. An acquisition step, an encoded data separation step of separating the encoded data acquired in the encoded data acquisition step into encoded data of each frame, and a code for each frame created in the encoded data separation step. Data merge process for merging encoded data into one encoded data, and a reference image for generating a signal for designating an image to be a reference image of each frame of the encoded data created by the encoded data merge process And a designation process. 2. The method according to claim 1, wherein in the reference image designating step, an image to be a reference image of each frame is designated by a frame number of the merged encoded data. 3. The method according to claim 1, further comprising the step of adding display time information specifying a display time of an image obtained by decoding the encoded data to each frame of the encoded data merged by the encoded data merging unit. 2. The method according to claim 1, wherein: 4. A method for generating encoded video data encoded in units of frames using an inter-frame differential encoding method, wherein the encoded data is obtained by encoding a plurality of encoded data. An acquisition step, an encoded data separation step of separating encoded data acquired by the encoded data acquisition step into encoded data of each frame, and an encoding for each frame created by the encoded data separation step. An encoded data merging process of merging data into one encoded data that can be decoded by one decoder, and a frame merged by the encoded data merging process is input by which encoded data acquiring process. It is a signal that distinguishes whether it is encoded data.
A frame identification signal creating step of creating a frame identification signal. 5. A storage medium storing a program for causing a computer to perform a process of creating encoded video data encoded in units of frames using an inter-frame differential encoding method, comprising: A plurality of encoded data acquisition processes for acquiring encoded data; an encoded data separation process for separating encoded data acquired in the encoded data acquisition process into encoded data of each frame; An encoded data merge process for merging the encoded data for each frame created in the data separation process into one encoded data; and a reference image of each frame of the encoded data created by the encoded data merge process. Characterized in that a program for causing a computer to execute a reference image designation process for creating a signal designating an image is stored. Storage medium for storing program. 6. The storage medium according to claim 5, wherein in the reference image designation processing, an image to be a reference image of each frame is designated by a frame number of the encoded data merged. . 7. The program includes a process of adding display time information specifying a display time of an image obtained by decoding the encoded data created by the encoded data merge process to each frame. An image-encoded data creation program storage medium according to claim 5. 8. A storage medium storing a program for causing a computer to perform processing for creating moving image encoded data encoded in units of frames using an inter-frame differential encoding method. A plurality of encoded data acquisition processes for acquiring encoded data; an encoded data separation process for separating encoded data acquired by the encoded data acquisition process into encoded data of each frame; An encoded data merge process for merging the encoded data for each frame created by the separation process into one encoded data that can be decoded by one decoder, and a frame merged by the encoded data merge process. , A frame for generating a frame identification signal, which is a signal for discriminating which encoded data is input by which encoded data acquisition processing An image encoded data creation program storage medium storing a program for causing a computer to execute an identification signal creation process. 9. A plurality of encoded data sets for acquiring encoded data in an image encoded data creating apparatus for creating moving image encoded data encoded in units of frames using an inter-frame differential encoding method. An acquisition unit, an encoded data separation unit that separates the encoded data acquired by the encoded data acquisition unit into encoded data of each frame, and a code for each frame created by the encoded data separation unit. Encoded data merging unit that merges encoded data into one encoded data, and a reference frame number creating unit that creates a reference frame number that specifies a reference image of each frame. apparatus. 10. A display time transmitting unit for adding display time information specifying a display time of an image obtained by decoding the encoded data to each frame of the encoded data merged by the encoded data merging unit. The image encoded data creation device according to claim 9, characterized in that: 11. A plurality of encoded data acquisition units for acquiring encoded data in an image encoded data creating apparatus for creating moving image encoded data encoded in frame units using inter-frame differential encoding. Unit, an encoded data separation unit that separates the encoded data acquired by the encoded data acquisition unit into encoded data of each frame, and encoding for each frame created by the encoded data separation unit. An encoded data merging unit for merging data into one encoded data that can be decoded by one decoder, and a frame merged in the encoded data merging unit,
What is claimed is: 1. An image encoded data creating apparatus, comprising: a frame identification signal creating unit that creates a frame identification signal, which is a signal for distinguishing from which encoded data acquiring unit the encoded data is input. 12. The plurality of encoded data acquisition units, wherein at least one of the encoded data acquisition units receives moving image encoded data via a transmission path and acquires encoded data. The encoded image data creating device according to claim 9, 10 or 11, wherein: 13. The plurality of encoded data acquisition units, wherein at least one of the encoded data acquisition units reads moving image encoded data from a storage medium or a device and acquires encoded data. The encoded image data creating device according to claim 9, 10 or 11, wherein: 14. The plurality of encoded data acquisition units, wherein at least one of the encoded data acquisition units encodes an input image to generate encoded data, and the encoded data is The encoded image data creating device according to claim 9, wherein the encoded image data is acquired. 15. The plurality of encoded data acquisition units, wherein at least one of the plurality of encoded data acquisition units processes pre-encoded image encoded data to generate new encoded data, 12. The apparatus according to claim 9, wherein the encoded data is obtained. 16. A video communication method for transmitting moving image encoded data encoded using an inter-frame differential encoding system, and a receiving side for encoding a moving image. A plurality of encoded data acquiring processes for acquiring data; an encoded data separating process of separating encoded data acquired in the encoded data acquiring process into encoded data of each frame; An encoded data merging step of merging the encoded data for each frame created in the separation step into one encoded data;
A reference image specifying step of generating a signal specifying an image to be a reference image of each frame of the encoded data created by the encoded data merging step;
An image communication method, comprising: a decoding step of decoding encoded data; an image storing step of storing a plurality of frames of images to be used as reference images; and a reference image selecting step of selecting an image to be used as a reference image. . 17. The method according to claim 1, wherein in the reference image specifying step, an image to be a reference image of each frame is specified by a frame number of the encoded data merged.
6. The image communication method according to 6. 18. A transmitting apparatus, comprising: a step of adding display time information for designating a display time of a decoded image; and a step of displaying a stored decoded image in accordance with the specified display time. 17. The image communication method according to claim 16, wherein: 19. An image communication method for transmitting moving image encoded data encoded using an inter-frame differential encoding method, and a receiving side, comprising: A plurality of encoded data acquiring steps for acquiring data; an encoded data separating step of separating encoded data acquired by the encoded data acquiring step into encoded data of each frame; The encoded data for each frame created by the process is
A coded data merging step of merging into one coded data that can be decoded by one decoder, and a frame merged by the coded data merging step, wherein the coded data input by any coded data acquisition step A frame identification signal creating step of creating a frame identification signal, which is a signal for distinguishing between the frames, a decoding step of decoding encoded data, and accumulating a plurality of frames of an image serving as a reference image on the receiving side. An image communication method, comprising: an image storage step; and a frame identification signal receiving step of receiving a frame identification signal. 20. A transmission side transmits moving image encoded data encoded using an inter-frame differential encoding method, and a reception side transmits an encoded image to a transmission side in an image communication system for decoding the encoded data. A plurality of encoded data acquisition units for acquiring data; an encoded data separation unit for separating encoded data acquired by the encoded data acquisition unit into encoded data of each frame; An encoded data merging unit for merging encoded data for each frame created in the separation unit into one encoded data, and a reference frame number creating unit for creating a reference frame number for designating a reference image of each frame are provided. A decoding unit for decoding encoded data, an image storage unit for storing a plurality of frames of images to be used as reference images, and a frame number of an image to be used as a reference image. An image communication system comprising: a reference frame number receiving unit that receives a signal. 21. A transmitting side transmits moving image encoded data encoded using an inter-frame differential encoding method, and a receiving side transmits an encoded image to a transmitting side in an image communication system for decoding the encoded data. A plurality of encoded data acquisition units for acquiring data; an encoded data separation unit for separating encoded data acquired by the encoded data acquisition unit into encoded data of each frame; A coded data merging unit for merging the coded data for each frame created in the separation unit into one coded data, and a frame merged in the coded data merging unit, A reference memory position specification creating unit that creates a reference memory position specification signal that specifies whether to refer to an image existing at the position, and decodes the encoded data on the receiving side An image communication system, comprising: a decoding unit that performs reference image storage; an image storage unit that stores a plurality of frames of images to be used as reference images; and a reference memory position specification reception unit that receives a reference memory position specification signal. 22. A transmission side, comprising: a display time transmission unit for adding display time information for designating a display time of a decoded image; and a reception time unit for obtaining display time information from received encoded data. 3. The method according to claim 2, further comprising:
22. The image communication system according to claim 21 or claim 22. 23. A transmitting side transmits moving image encoded data encoded by using an inter-frame differential encoding method, and a receiving side transmits an encoded image to a transmitting side in an image communication system for decoding the encoded data. A plurality of encoded data acquisition units for acquiring encoded data; an encoded data separation unit for separating the encoded data acquired by the encoded data acquisition unit into encoded data of each frame; An encoded data merging unit for merging the encoded data for each frame created in the data separating unit into one encoded data decodable by one decoder; and a frame merged in the encoded data merging unit. And a frame identification signal creating unit for creating a frame identification signal, which is a signal for distinguishing from which encoded data input unit the encoded data is input. Image communication comprising: a decoding unit for decoding encoded data; an image storage unit for storing a plurality of frames of images serving as reference images; and a frame identification signal receiving unit for receiving a frame identification signal. system.