JPH0750838A - Motion picture coder - Google Patents

Abstract

PURPOSE:To facilitate a frame skip for synchronization reproduction. CONSTITUTION:Video data are coded by a coding circuit 80 and stored tentatively in a code buffer 82. A counter 86 counts in parallel a code data quantity generated by the coding circuit 80. When coding processing is finished by one frame, an insertion circuit 84 inserts a code representing the code data quantity from the counter 86 before code data from the code buffer 82. That is, the insertion circuit 84 outputs the code data after a code representing generated code quantity data. Frame skip processing in the case of synchronization reproduction is facilitated by a code representing the code data quantity at a video receiver side.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture coding apparatus, and more particularly to a moving picture coding apparatus in an image communication system.

[0002]

2. Description of the Related Art A TV is a low bit rate image communication system.
I have a phone. The video telephone system is basically based on real-time communication and requires hardware such as a dedicated terminal and a dedicated expansion board. On the other hand, a system for exchanging video files on a computer such as a general-purpose workstation is known. In the method using such a video file, the image communication /
It has become possible to perform processing such as playback display by software.

A video / audio transmission system constructed by using a general-purpose workstation has the following restrictions.
That is, the frame memory of the display system is usually 8 bits,
It is common to use 256 of the 16.4 million colors. The colors to be displayed can be selected by a look-up table method. In addition, 8-bit color
When a black-and-white image and a color image are mixed on a monitor, the black-and-white image can express 128 gradations (gray scale), and the color image is dithered to allocate 2 bits to each color of R, G and B, and 64 colors Can be expressed.

In the communication and storage system, both audio data and video data are compression-coded to reduce the data amount. The compression method includes an inter-frame coding (INTER) method, an intra-frame coding (INTRA) method, and a method using these in combination. In the compression method that uses both the inter-frame coding method and the intra-frame coding method, in order to prevent transmission of transmission error and facilitate special reproduction, the intra-coded frames are adaptively or uniformly spaced at appropriate intervals. It is usual to place in. Discrete cosine transform (DCT), differential pulse coding (DPCM), or the like is used as an individual coding method.

Interframe coding may be frequently used to increase the compression rate, but there is a drawback that the data of the previous frame is required for decoding.

In a moving image, it is necessary to synchronize the reproduction of video and the reproduction of audio. However, when the processing capacity of hardware is low or the processing capacity of the reproduction processing is low, it is necessary to skip an appropriate frame. There is.

[0007]

A conventional image compression method,
In particular, in an image compression method that employs a variable-length coding method in which the code data amount changes for each frame, very complicated processing is often required to calculate the code data amount for one frame. There was a problem that skipping became difficult. That is, although the frame skip is performed for the purpose of reducing the load of the reproduction process, a new burden (calculation of the code data amount for one frame) is imposed due to the frame skip. .

It is an object of the present invention to provide a moving picture coding apparatus which solves such a problem.

[0009]

A moving picture coding apparatus according to the present invention comprises a compression coding means for compressing and coding a moving picture,
Counting means for counting the generated code amount for a predetermined screen by the compression encoding means, storage means for storing code data for the predetermined screen, and inserting a code indicating the code amount of the screen for each screen of the code data. And a code inserting means for performing the same.

[0010]

By the above means, the code data amount of one frame can be known in advance by the reproduction process, so that the frame skip becomes very simple.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

First, a video file communication system which is a premise of the present invention will be described. FIG. 1 shows a schematic block diagram thereof. In FIG. 1, reference numeral 10 is a file server, which stores a video database in the mass storage device 12. Reference numeral 14 is a workstation or a small computer which is a client, and is connected to the file server 10 via the network 16. Client 1
For the monitor display system of No. 4, the X window system well known in the workstation is used.

FIG. 2 shows a monitor screen of the client 14. On the monitor screen of the client 14, an index window 20 for displaying index images of a video database stored in the storage device 12 of the server 10, a file box for displaying a list of video file names transferred from the video database to the client 14, There is a window 22 and a playback display window 24 for displaying a video file designated from the video files displayed in the file box window 22.

Further, in the index window 20, ge for instructing the transfer of the designated video file to the own device is generated.
There is a t button 26. The reproduction display window 24 has a reproduction button 28 for instructing reproduction start and a reproduction stop button 30, and may additionally have a pause button for instructing reproduction interruption and forward and reverse frame advance buttons. .

FIG. 3 shows an operational flowchart of the system shown in FIGS. Index window 2 of the index image of the video database on the server 10
It is displayed at 0 (S1). The index image is, for example,
It is a still image of the first or representative screen of a video file. A video file to be reproduced is selected according to the displayed index image (S2). When the get button 26 is pressed, the selected video file is transferred from the server 10 to the client 14 via the network 16 (S
3). The client 14 saves the received video file in a storage device (not shown). For the video files transferred to the client 14, the video file names are displayed as a list in the file box window 22.

When one of the video file names listed in the file box window 22 is designated, the first image is displayed in the playback display window 24. When the play button 28 is operated, the client 14
Decodes the compression code of the specified video file (S
4), color conversion from YUV system to RGB system (S5), dither processing (S6), video display on the playback display window 24, and playback voice output as voice (S7).

FIG. 4 is a schematic block diagram of a circuit for converting a video to be a video file into data. The video signal from a video camera or video tape recorder is A /
It is digitized by the D converter 40 and converted into a luminance signal Y and a color difference signal U / V by the NTSC decoder 42.

The dither circuit 44 includes the NTSC decoder 42.
The luminance signal Y from is converted into an 8-bit lookup table number. Further, the color conversion circuit 46 receives the luminance signal Y and the color difference signal U / V from the NTSC decoder 42 as RG.
Converted into a B signal, the dither circuit 48 converts the RGB signal into an 8-bit lookup table number. The color conversion circuit 46 also supplies the luminance signal Y and the color difference signal U / V from the NTSC decoder 42 to the mass storage device 52 via the interface 50 as they are. A digital audio signal is also supplied to the large-capacity storage device 52 from an audio processing system (not shown) and is stored in the large-capacity storage device 52 together with the video signal.

The output of the dither circuit 44 is stored in the memory area 54A of the frame memory 54 corresponding to the screen displaying a black and white gradation image, and the output of the dither circuit 48 is
Memory area 54B corresponding to the screen displaying a color image
Stored in. The 8-bit data stored in the frame memory 54 is sequentially read and applied to the video display device 58 via the look-up table 56. The image in the memory area 54A is displayed in the window 58A in the display screen of the video display device 58, and the image in the memory area 54B is displayed in the window 58B.

FIG. 5 shows a flowchart of a process for storing raw video and audio data stored in the mass storage device 52 as a video file in the mass storage device 12 of the server 10. Raw video and audio data is read from the storage device 52 (S11), compression encoded (S12), stored in the mass storage device 12 of the server 10, and registered as a video file (S13).

FIG. 6 shows an example of a 256-color look-up table in the case of displaying a grayscale monochrome image and a color image in a mixed manner. Table numbers 0 to 127 are for black and white and are represented by 7-bit 128 gradations. Table number 128-1
Reference numeral 91 is for color and represents 64 colors of color dither. Table numbers 192-255 are for unused or other applications.

FIG. 7 shows a detailed circuit block diagram of the dither circuit 48. The color conversion circuit 46 uses YUV according to the following formula.
Convert the system to the RGB system. That is, R = Y + (V-128) /0.713 G = Y + (U-128) /0.564 B = (Y-0.299R-0.114B) /0.587 4-value dither circuits 48a and 48b. , 48c respectively convert the R, G, B data from the color conversion circuit 46 into 2 bits,
The conversion circuit 48d is a four-value dither circuit 48a, 48b, 4
A total of 6-bit data from 8c is assigned to the table numbers 128 to 191 shown in FIG.

As a video signal compression method, as described above, there is known a compression coding method in which interframe coding (INTER) and intraframe coding (INTRA) are arranged in a fixed order or adaptively. There is. Considering frame skipping, it is necessary to arrange intraframe coding at a certain interval, and the distribution of intraframe coding and interframe coding is determined by the required compression ratio, image quality, playback processing capacity, etc. To be done. In one embodiment of the present invention, intra-frame coding and inter-frame coding are arranged for each frame. This allows fine control of frame skip.

FIG. 8 shows a schematic block diagram of a video signal compression encoding apparatus. The frame memory 60 delays the input image data by one frame, and its output is applied to the a contact of the switch 62. At the b contact of the switch 62,
0'is applied. The selected value of the switch 62 is applied to the negative input of the subtractor 64, and the input image data is applied to the positive input of the subtractor 64.

In response to the INTRA / INTER switching signal, the switch 62 is connected to the a contact during interframe coding and to the b contact during intraframe coding. Therefore, the subtractor 64 outputs the input image data as it is in the intra-frame coding, and outputs the difference value from the previous frame in the inter-frame coding. The encoding circuit 66 compression-codes the output of the subtractor 64 and outputs compressed code data.

FIG. 9 is a block diagram showing the configuration of the decoding device corresponding to FIG. The decoding circuit 68 corresponds to the encoding circuit 66 and decodes the compressed code data. The output is applied to the adder 70. The output of the adder 70 is delayed by one frame in the frame memory 72, and the switch 74
Is applied to the a contact of the. For switch b contact b '
0'is applied. The selected value of the switch 74 is applied to the adder 70.

The switch 74, like the switch 62,
By the INTRA / INTER switching signal, the inter-frame coding is connected to the a contact, and the intra-frame coding is connected to the b contact. IN for switch 74
The TRA / INTER switching signal is formed from the intra-frame coding / inter-frame coding identification information included in the header of the compressed image data.

By switching the switch 74, the adder 7
In the case of 0, the output of the decoding circuit 68 is output as it is in an intra-frame encoded frame, and in the case of an inter-frame encoded frame, the output of the previous circuit is added to the output of the decoding circuit 68 and output. . The output of the adder 70 becomes the restored image data.

FIG. 10 is an explanatory diagram of a sequence in which intraframe coding and interframe coding are switched for each frame. One second is composed of four frames. When the frame is skipped because the reproduction and display process is not in time, the frame for inter-frame encoding is skipped and the frame for intra-frame encoding is reproduced and displayed.

The encoded video data is stored and transmitted in the format as shown in FIG. There is a fixed-length header in front of the encoded data, and the header includes identification information indicating the compression method, intraframe encoding or interframe encoding, and information indicating the amount of encoded data.
When it is desired to skip the frame, the code amount of the inter-frame coded frame can be known by reading the header of each frame, and the frame can be skipped without performing a complicated operation.

FIG. 12 shows the structure of a video file.

In the videophone, the video data and the audio data for each unit time are alternately transmitted in packets.
In the case of file communication, voice and video can be communicated independently.

For speech, a well-known general coding method can be used.

Since the audio data is generally smaller than the video data, it is possible to use the hierarchical structure such that the audio data is sent first and only the audio is reproduced when the transfer of this portion is completed.

FIG. 13 shows a flow chart of the synchronous control. An example will be described in which frames are displayed at Δt intervals from time t0. t is the current time obtained by referring to the internal clock.

The variable n is initialized and the variable t0 is set to the current time t.
Initialize with (S41). The current time t is substituted into the variable t1 (S42). Read the compressed data from the file,
The previously described processing such as decompression, color conversion, and dither processing is performed to develop the image data in a displayable format in the memory (S43). The value obtained by subtracting t1 from the current time t is substituted into the variable s (S44). s indicates the time required to execute step S43.

It is judged whether or not the current time has passed the display scheduled time of the nth frame (S45), and if it has passed, the process proceeds to the next S46, and if it has not passed, it waits for it to pass. This loop prevents synchronization deviation due to the display advancing too fast.

The image data in the memory is displayed (S4
6), the frame number n is incremented (S47),
The number of the frame to be displayed next is calculated in consideration of the time s required to execute S42, and is substituted for the variable m (S48). m and n are compared (S49). If m is larger than n, the process jumps to S52, and if not, the process proceeds to S50.

In S50, the data for (mn) sheets is skipped. Specifically, a seek is performed by the code data amount of the next frame obtained by reproducing the data indicating the code amount recorded by the method of the present embodiment from the position of the seek pointer to the data file at that time.・
By advancing the pointer, one frame of data can be skipped. This is repeated (mn) times. Then, m is substituted for the frame number n (S51). That is,
Correct the gap between the frame number and seek pointer to the file due to data skip.

By the above-described processing of S48 to S51, the synchronization shift when the display is not in time is prevented.

In S52, it is determined whether or not the reproduction of the moving image is ended. The criteria for determination include the case where the seek pointer to the file reaches the end, that is, the moving image is reproduced to the end, and the case where the reproduction stop button or the pause button is operated. When the reproduction ends, this routine ends. Otherwise, S
It returns to 42 and repeats S42 and subsequent steps.

FIG. 14 shows a schematic block diagram of a circuit for inserting a code indicating the code data amount into the code data.
80 is an encoding circuit, 82 is a code buffer, 84 is an insertion circuit for inserting a code indicating the code data amount into the code data,
A counting circuit 86 counts the generated code data amount.

The video data is coded by the coding circuit 80 and then temporarily stored in the code buffer 82. Meanwhile, in parallel, the counter 86 counts the code data amount generated by the encoding circuit 80. When the encoding process is completed for one frame, the inserting circuit 84 inserts the code indicating the code data amount from the counter 86 before the code data from the code buffer 82. That is, the insertion circuit 84 outputs the code data after the code indicating the generated code amount data.

On the video receiving side, the code indicating the amount of coded data facilitates frame skip processing at the time of synchronous reproduction.

In the existing coding method, it is not possible to arbitrarily insert a code indicating the code data amount. But,
For example, in a standard encoding method such as MPEG, a "user area" that a user can freely use to some extent
It is usual that the same effect can be obtained by writing the information of the code data amount of one frame in this area. That is, in FIG. 14, instead of inserting the code indicating the code data amount by the inserting circuit 84, the code indicating the code data amount for one frame may be written in the “user area”. The other processing is exactly the same as that of the above-mentioned embodiment, and therefore its explanation is omitted.

In this embodiment, one frame, that is,
Although the data indicating the code amount is inserted for each screen, the present invention is not limited to this, and the code amount data may be inserted for each unit such as every two screens or every five screens.

It is obvious that the processing of the above embodiment can be partially or entirely realized by either hardware or software.

[0048]

As can be easily understood from the above description, according to the present invention, frame skip processing is simplified at the time of reproduction.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of a video / audio transmission system that is a premise of the present invention.

FIG. 2 is an explanatory diagram of an operation display screen of the client 14.

FIG. 3 is an operation flowchart of FIG.

FIG. 4 is a schematic block diagram of a device for converting video information into data.

FIG. 5 is a flowchart for converting the converted video data into a video file.

FIG. 6 is a lookup table of an image in which black and white and color are mixed.

FIG. 7 is a detailed circuit block diagram of the dither circuit 48.

FIG. 8 is a schematic block diagram of a compression encoding device for video information.

9 is a schematic block diagram of a decoding device corresponding to FIG.

FIG. 10 is a sequence diagram of an example in which intra-frame coding and inter-frame coding are alternately arranged.

FIG. 11 is a recording format of encoded video information.

FIG. 12 is a structural diagram of a video file according to an embodiment of the present invention.

FIG. 13 is a flowchart of synchronized playback of video.

FIG. 14 is a schematic block diagram of a circuit that inserts a code indicating a code data amount into code data.

[Explanation of symbols]

10: File server 12: Mass storage device 1
4: client 16: network 20: index window 22: file box window 24: playback display window 26: get button 28: playback button 30: playback stop button 40: A / D converter 42: N
TSC decoder 44: Dither circuit 46: Color conversion circuit 48: Dither circuit 48a, 48b, 48c: 4-value dither circuit 48d: Conversion circuit 50: Interface 52: Mass storage device 54: Frame memory 5
4A, 54B: Memory area 56: Look-up table 58: Video display device 58A: Window 58
B: Window 60: Frame memory 62: Switch 64: Subtractor 66: Encoding circuit 68: Decoding circuit 70: Adder 72: Frame memory 74:
Switch 80: Encoding circuit 82: Code buffer 8
4: Insertion circuit 86: Code data amount counting circuit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical indication H04N 5/92 7/15 7251-5C 7734-5C H04N 5/92 H

Claims (1)

[Claims] 1. A compression encoding means for compression encoding a moving image, a counting means for counting the generated code amount for a predetermined screen by the compression encoding means, and a storage means for storing code data for a predetermined screen. , A code inserting means for inserting a code indicating a code amount of the screen for each screen of the code data.