JP2644132B2 - Moving image storage / playback processing method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture in an image message storage and exchange system using a center-to-end communication mode for compressing the amount of information of a moving picture, storing the information, and then decoding and reproducing the information. The present invention relates to an image storage / reproduction processing method.

[0002]

2. Description of the Related Art A system configuration of a conventional image storage service based on a center-to-end still image is shown in FIG.
FIG. 9 shows a system configuration of a moving picture videophone, and FIG. 10 shows an image transmission method used in the moving picture videophone.

[0003] In FIG. 8, a center 103 and a terminal 101 are connected via a communication network 102. In response to a connection request from a terminal or a host 110 in the center, a communication path is established via a communication network. When the communication path is established, the data of the guide screen of the still image is transferred from the center to the terminal, displayed on the monitor 106 of the terminal, and the user on the terminal side performs menu selection along the guide screen. When the image storage is selected, the image from the camera 105 and the sound from the handset 107 are respectively transmitted to the image encoding / decoding unit 10.
8. The host 1 is encoded by the audio encoding / decoding unit 109 and stored in the storage device 104 of the center 103 via the communication path.
It is stored under the control of 10. When the reproduction is selected, the coded image / audio data stored in the storage device 104 of the center is read out, and the image is encoded / decoded by the image encoding / decoding unit 108 and the audio encoding / decoding unit 109 of the terminal via the communication path. The image data and the audio data are decoded, and the still image is displayed on the monitor.
Voice is output to the handset. Here, the image is encoded in units of one frame based on a still image.

[0004] On the other hand, the operation of a moving picture videophone is as follows. In FIG. 9, the camera 204 of the local terminal 201
The image input from the image encoding / decoding unit 208a
Performs encoding by inter-frame predictive encoding, and
9a. The audio signal input from handset 206a is encoded by audio encoding / decoding section 210a and input to multiplexing section 209a. Multiplexing unit 209a
Then, under the control of the control unit 207a, the coded image data, the coded audio data, and control data (hereinafter, referred to as image control data) for the image specified by the control unit 207a are multiplexed together. , Via the communication network 203 to the partner terminal 202.

[0005] The multiplexed data sent from the local terminal 201 is received by the multiplexing unit 209b and separated into image control information, coded image data, and coded audio data. The image control information is transmitted to the control unit 207b, the encoded image data is transmitted to the image encoding / decoding unit 208b, and the encoded audio data is transmitted to the audio encoding / decoding unit 210b. The encoded image data that has been subjected to the inter-frame predictive encoding in the image encoding / decoding unit 208b is decoded, reproduced and output to the monitor 205b, and the encoded audio data is output from the audio encoding / decoding unit 21.
0b and output to handset 206b. The image control information is processed by the control unit 207b.

Next, a method of transmitting a moving image will be described with reference to FIG. After the line connection is established, the control unit 207a of the own terminal 201 uses the above-described image control information to send an “update screen transmission request” to the other terminal 202 as a reference screen for inter-frame prediction. put out. Upon receiving this information, the control unit 207b of the partner terminal issues a “screen update instruction” to the image encoding / decoding unit 208b,
The image encoding / decoding unit 208b of the other terminal 202 sends out “updated screen data” to the own terminal 201.
According to such a procedure, a good image can be viewed on the local terminal 201 at the earliest timing after the line connection.

When a reproduced image is disturbed due to a scene change or the like of an input image, a “transmission screen freezing request” is transmitted using control information for the image of the partner terminal 202, and
During the disturbance, the playback image of the local terminal 201 is frozen,
The freeze is released at the timing when the reproduced image becomes somewhat clean.

[0008] When the moving picture videophone employing the inter-frame prediction coding shown in FIGS. 9 and 10 is applied to the system shown in FIG. 8, the stored moving picture stored in the center is reproduced. In this case, depending on the state of the previous message, it is expected that the head portion of the stored moving image cannot be reproduced properly.

[0009]

In the current moving picture videophone, since an inter-frame prediction code is used for image compression, it is necessary to detect and reproduce a reference update screen. However, since the current videophone is premised on end-to-end communication, as can be seen from the sequence shown in FIG. 10, it is not necessary to be conscious of whether the content of the image data is an update screen. It is made. Therefore, when attempting to realize an image storage service using a moving image employing the inter-frame predictive coding method, it is possible to reproduce a high-quality stored moving image even with a video TV telephone terminal that is not aware of the content of image data. It is an important issue to build a simple center system.

SUMMARY OF THE INVENTION It is an object of the present invention to construct a center system capable of reproducing a high-quality stored moving image even with a moving picture videophone terminal that is not aware of the contents of image data.

[0011]

According to the present invention, there is provided an encoding means for encoding an input image, and a first transmission for multiplexing the encoded image with encoded voice and control data and transmitting the multiplexed image. Means, first receiving means for receiving the multiplexed data and separating it into coded images, coded audio, control data, etc., means for decoding the received coded image, and coding means And a first control means for controlling a first transmitting means and a first receiving means, a decoding means, a first transmitting means and a first receiving means.
Second receiving means for receiving the image transmitted from the transmitting means, storing means for storing the encoded image or sound received by the second receiving means, and the stored encoded image or sound And a center provided with a second control means for controlling the storage means and the read means. In a center-to-end type image message storage and exchange system in which a center is connected via a communication network, on the image storage side in the center, screen header information of a moving image from a terminal user is unfrozen. After rewriting to make
The image encoding data starting from the update screen following this is stored together with the image encoding data starting from the update screen, and when the stored image is read out, after the freeze of the previous screen is released by the screen header information, the image encoding data starting from the update screen subsequently transmitted To be decrypted.

[0012]

[Function] By using the moving image storage / reproduction processing method,
Good reproduction of stored moving images can be achieved even by using a moving image encoding / communication method.

[0013]

Embodiments of the present invention will be described below in detail. FIG. 1 shows a system configuration of a moving image accumulation / reproduction processing method according to the present invention. In addition, CCITT Recommendation H. FIG. 2 shows a frame configuration of FIG. 221 and FIG. 4, 5, and 6 show the format of the stored image based on the image data structure defined by H.261, and FIG. 7 shows a method of reproducing the stored moving image.

In FIG. 1, reference numeral 401 denotes a moving picture videophone;
402 is a center and 403 is a communication network. The video videophone 401 includes a camera 404, a monitor 405, a handset 406, a control unit 407, and an image encoding / decoding unit 40.
8, a speech encoding / decoding section 410 and a multiplexing section 409. The center 402 includes a control unit 411, a multiplexing unit 412, and a storage device 413.

In FIG. 2, reference numeral 501 denotes a CCITT Recommendation H.264 assembled by multiplexing sections 409 and 412. 221, a frame synchronization signal (FA)
S) and 503 are bit rate assignment signals (BAS) used for negotiation of terminal capabilities and communication conditions. Reference numeral 504 denotes data in which coded image data, coded audio data, and general data are multiplexed, and 505 denotes a frame configuration when two ISDN basic interfaces are used.

4 to 6, reference numeral 701 denotes a screen header; 702, screen freeze release information; 703a, 703;
b, 703c, 707 are screen header information, 704 is an inter-frame prediction screen or update screen, 705 is an update screen,
Reference numeral 706 denotes an inter-frame prediction screen.

The operation of the embodiment will be described below. FIG.
The basic operation of the moving picture message storage and switching system using a moving picture videophone will be described. After connection is established from the video telephone 401 to the center 402 via the communication network 403 and a communication path is established, the control unit 407 in the video video telephone and the control unit 411 in the center use the BA shown in FIG.
The image communication to the center 402 is performed according to the instruction of the control unit 407 according to the result of the capability exchange of the terminal using S503.

That is, the camera 404 on the videophone 401 side
Are subjected to inter-frame predictive encoding in an image encoding / decoding unit 408. The coded image data is input from the handset 406 and multiplexed by the multiplexing unit 409 together with the coded voice data coded by the voice coding / decoding unit 410 and the image control information from the control unit 407. The data is transmitted to the multiplexing unit 412 in the center 402 via the network 403.

On the center side, the data multiplexed with the image, sound, image control information and the like received by the multiplexing unit 412 is transmitted by the control unit 411 in accordance with the frame structure 50 shown in FIG.
1 or 505 is an appropriate file as it is,
It is stored in the storage device 413.

The stored data is read from the storage device 413 in accordance with an instruction from the control unit 411 that has received a request from the user of the videophone, and the multiplexing unit 41 in the center.
2 The control unit 411 rewrites the image control information stored as it is from the video telephone 401 at the time of storage to new information if necessary, and transmits the new information to the video telephone 401 via the communication network 403.

In the video telephone 401, the multiplexed data of the image, voice, image control information, and the like sent from the center are separated by the multiplexing unit 409 into coded image data, coded audio data, and image control information. The encoded image data is sent to the image encoding / decoding unit 48, the encoded audio data is sent to the audio encoding / decoding unit 410, and the image control information is sent to the control unit 4.
07. The encoded image data is decoded and reproduced by the image encoding / decoding unit 408 in accordance with an instruction from the control unit 407, and is displayed on the monitor 405. The encoded audio data is decoded and reproduced by the audio encoding / decoding unit 410, and is output from the handset 406. In this way, messages from the videophone can be stored in the center, or the stored messages can be searched and read.

Next, the operation of the basic part of the present invention will be described in detail. FIG. 3 shows a moving image storage method. In the figure, an example will be described in which communication is already possible between the moving picture videophone 401 and the center 402, and voice / image messages are stored from the moving picture videophone 401. In accordance with the result of the negotiation between the control unit 407 and the control unit 411, the video video phone 401 and the center 402
H. The image data is transmitted in 221 frames, and is discarded by the multiplexing unit 412 on the center 402 side.

[0023] From the moving picture videophone 401 to the center 402
In order to accumulate a moving image message, first, the control unit 407 of the moving image
An instruction to transmit a “storage start request” of an image using BAS is issued. In the center 402, the multiplexing unit 412 receives the “storage start request” by BAS, and notifies the control unit 411 of the content. The control unit 411 includes a multiplexing unit 412
In response to this, an "accumulation start instruction" is issued. The multiplexing unit 412 notifies the control unit 407 of the moving picture videophone 401 of a “screen update request” serving as a reference screen for inter-frame prediction using the BAS. In the multiplexing unit 412, the H.264 transmitted from the moving picture videophone 401 immediately after receiving the “storage start instruction” from the control unit 411. The image data of the frame 221 is transferred to the control unit 411 without discarding, and the header part of the image data is rewritten to enable the screen freeze release, and then retransmitted to the multiplexing unit 412 and stored. Device 413
To accumulate.

On the other hand, the control unit 4 on the moving picture videophone 401 side
In step 07, in response to the “screen update request” from the center 402, a “screen update instruction” is given to the image encoding / decoding unit 408. The image encoding / decoding unit 408 sends “updated screen data” to the center 402. The multiplexing unit 412 in the center 402 receives the moving image message starting with the update screen, sequentially stores the image data from the multiplexing unit 412 to the storage device 413 without rewriting the header part of the image data, Continue until a "storage termination request" is received. At this time, the format of the image data stored in the storage device 413 is shown in FIGS.

FIG. 4 shows CCITT recommendation H.264. 22 shows the details of the screen header information 701 specified in FIG. Among them, the screen freeze release information 702 indicates that when an image is to be reproduced,
If the previous image is frozen, it will not be displayed even if a new image is sent, so it is controlled. This information is generally set at the time of encoding by the image encoding / decoding unit 408 in the moving picture videophone, but it is arbitrary for the moving picture videophone 401 to control screen freeze release.

FIG. 5 shows the structure of image data transmitted from the moving picture videophone 401 in accordance with the sequence of FIG. 3. The moving picture videophone 401 instructs the center 402 to display an inter-frame prediction screen or an update screen 704. The header information 703a is inserted and transmitted. The subsequent update screen 705 is an update screen transmitted from the moving picture videophone 401 in response to a request from the center 402, and
The screen subsequent to 5 is the inter-frame prediction screen 706. Screen header information 703b and 703c are also inserted in the update screen 705 and the inter-frame prediction screen 706.

FIG. 6 shows a format stored in the storage device 413 of the center 402. The inter-frame prediction screen stored before the update screen 705 or the screen header information 707 inserted before the update screen 704 is stored in the center 402. Side control unit 4
At 11, the screen is rewritten so that the freeze release is effective.

FIG. 7 shows a method of reproducing a stored moving image. The control unit 411 that receives a request from a user of a videophone call
According to the instruction, the stored data is read from the storage device 413, and is transferred to the multiplexing unit 412 as “read data” in the form of the frame configuration 505 or the frame configuration 501 as it is. The multiplexing unit 412 transmits the frame configuration to the moving picture videophone 401. In the moving picture videophone 401, the multiplexing unit 409 separates the frame data, and the contents of the BAS are notified to the control unit 407 as image control information. The encoded image data is sent to the image encoding / decoding unit 408, where the inter-frame prediction code is decoded. In the image encoding / decoding unit 408, FIG.
In accordance with the contents of the header information of the image data, the previous screen is defrosted, the subsequently transmitted update screen 705 is decoded and reproduced, and the subsequent inter-frame prediction screen 706 is decoded and reproduced.

[0029]

As described above, according to the present invention,
Even in the case of using a moving picture videophone, a moving picture storage and exchange system can be realized that can always reproduce and display from the beginning of the stored moving picture message regardless of the state of the previous screen.

[Brief description of the drawings]

FIG. 1 shows a system configuration of a moving image accumulation / reproduction processing method of the present invention.

FIG. 2 shows a configuration of a stored image frame according to the present invention.

FIG. 3 shows a moving image storage method of the present invention.

FIG. 4. CCITT Recommendation H. 26 shows details of screen header information in the form of a stored image according to the image data structure defined by H.261.

FIG. 5: CCITT Recommendation H. 26 shows transmission data from a moving picture videophone in the form of a stored image according to the image data structure defined by H.261.

FIG. 6: CCITT Recommendation H. 26 shows the stored data at the center in the form of a stored image according to the image data structure defined by H.261.

FIG. 7 shows a method of reproducing a stored moving image.

FIG. 8 shows a system configuration of a conventional image storage service based on still images.

FIG. 9 shows a system configuration of a conventional moving picture videophone.

FIG. 10 shows an image transmission method used in a conventional moving picture videophone.

[Explanation of symbols]

 401 video videophone 402 center 403 communication network 404 camera 405 monitor 406 handset 407 control unit 408 image encoding / decoding unit 409 multiplexing unit 410 audio encoding / decoding unit 411 control unit 412 multiplexing unit 413 storage device

Claims (1)

(57) [Claims] An encoding unit for encoding an input image, a first transmission unit for multiplexing the encoded image with encoded voice or control data and transmitting the multiplexed image, and receiving the multiplexed data. Receiving means for separating the coded image, coded sound and control data into data, means for decoding the received coded image, coding means, decoding means and first transmitting means A terminal having first control means for controlling the first receiving means; and a second receiving means for receiving the image transmitted from the first transmitting means and a second receiving means for receiving the image transmitted from the first transmitting means. Storage means for storing the coded image and sound; reading means for reading the stored coded image and sound; second transmitting means for transmitting the read coded image and sound; Controls the storage means and readout means That includes a center and a second control means, the terminal and the center and is in the picture message store and forward system of the center-to-end connected via a communication network, the second receiving means of said center The terminal information received at
The moving image encoded data from the user is transmitted to the second control unit.
When storing in the storage means according to these instructions, the second control
Control means to store the screen header information of the first moving image to be stored.
Information so that screen freeze release is enabled.
After that, start with the top image data and the following update screen
The data is transferred to the storage means together with the encoded image data and stored.
And when the terminal requests a readout of the stored image,
Is transferred to the terminal by the second transmitting means of the
According to the screen header information of the top screen of the stored image,
Before reading the stored image, the image displayed on the terminal
After the freeze of the screen display is released,
A moving image storage / reproduction processing method comprising decoding and displaying encoded image data starting from a new screen .