JP2776401B2 - Image / audio compression device and playback device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image / audio compression apparatus for recording an image code and a compressed audio code on a recording medium such as a CD-ROM or a hard disk, and a recorded image code and an audio code. The present invention relates to an image / sound playback device used when playing back.

[0002]

2. Description of the Related Art An image / sound reproducing apparatus reads an image and a sound code recorded on a recording medium such as a CD-ROM or a hard disk and reproduces the read codes on a display or a speaker. The codes of the image and the sound reproduced by the image and sound reproducing device are recorded by being compressed and encoded by the image and sound compressing device.

[0003] A conventional image compression apparatus using an inter-frame difference encoding method will be described with reference to FIG. FIG. 1 is a diagram for explaining a conventional inter-frame differential encoding method.
In the inter-frame difference method, a difference value between frames of an image is 0.
The difference value is compression-encoded using the concentration in the vicinity. As shown in FIG. 1, the image is separated into luminance and color difference, and the difference between the pixel value of the current frame and the pixel value of the previous frame is calculated and compression-encoded. A plurality of frames are compressed by continuously performing the compression encoding of the difference value. Further, since the difference value between frames of an image may increase due to a scene change, a frame (key frame) for encoding a pixel value instead of a difference at a certain interval is inserted.

[0004] In such a system, a device that can edit a large amount of information such as images and sounds at high speed, compress it, and record it is required. To do so, it is necessary to take the reproduction timing (synchronization). In particular, the recording / reproducing speed of an image may be lower than the recording / reproducing speed of audio depending on the performance of the apparatus, the size of the compression code of the image, and the like. As a result, the image and the audio are not synchronized at the time of reproduction, and the image looks unnatural. For example, the processing for displaying the decoded image is 5
If it takes milliseconds, decoding of one frame needs to be performed in about 28 milliseconds in order to perform processing at 30 frames / second.
If the decoding processing time takes about 50 milliseconds, the sound is reproduced about 22 milliseconds ahead while reproducing the image of one frame, so that the moving image and the audio cannot be synchronized.

[0005] Therefore, by synchronizing the image and the audio at the time of reproduction by performing frame thinning of the image, in this case, while the moving image and the audio are synchronized, the image is not synchronized while the frame is thinned. A problem that the motion of the moving image looks unnatural because it does not change occurs. For example, when four frames are thinned out to synchronize the moving image and the audio, the image does not change for about 133 milliseconds.

As a conventional example for achieving the above-mentioned synchronization, there is JP-A-61-274480. In this method, image compression / reproduction is speeded up by detecting changed pixels and encoding images that are sequentially sharpened from coarse images. Also, Japanese Patent Application Laid-Open No. 2-117290 proposes a method for speeding up image compression / reproduction by dividing an image into a plurality of regions and encoding only a changed region.

[0007]

In a conventional image / sound compression apparatus or image / sound reproduction apparatus, image compression / image reproduction is speeded up by simplifying image processing by thinning out images. However, since only an area having a large number of changed pixels is encoded, an area with a large motion can be normally reproduced, but an area with a small motion is not reproduced. Therefore, there is a problem that the motion of the entire screen looks unnatural. An object of the present invention is to prepare a plurality of types of parameters corresponding to an image processing speed in an image thinning process, and to perform the thinning process based on the parameters, thereby synchronizing the image with the audio and synthesizing the image. An object of the present invention is to provide an image / sound compression device and an image / sound reproduction device which can prevent deterioration.

[0008]

According to the present invention, an image of a current frame is separated into luminance and chrominance, and an inter-frame difference between the current frame and the previous frame of luminance and chrominance is compression-coded and compressed. In an image / audio compression apparatus for recording an image signal and an audio signal in synchronization with a recording medium, a decimating information storing means for storing chrominance decimating information predetermined according to an information amount of an image signal to be processed; Compression information extracting means for extracting the thinning information from the thinning information storage means, and compression encoding means for thinning out the color difference according to the thinning information extracted from the compressed information extracting means and compression-encoding the inter-frame difference. Thus, an image / sound compression device characterized by having the above is obtained.

Further, according to the present invention, there is provided means for extracting the thinning-out information from a compressed code of an image and a sound provided with the thinning-out information of the color difference recorded in the image / sound compressing device, and a color difference according to the thinning-out information. Decoding means for interpolating and decoding the compression code, thereby obtaining an image / sound reproducing apparatus.

[0010]

According to the present invention, since the image / sound compression apparatus adapts the processing speed to reduce the color difference and simplifies the image processing to speed up the image compression, the image and the sound are synchronized and recorded. Can be. Also, in the image / sound playback device,
Since the thinned color difference is interpolated to simplify the image processing and speed up the image reproduction, the image and the sound can be reproduced in synchronization.

Further, when compressing an image by frame thinning, a difference value is determined and a frame having a small difference value is thinned, so that compression can be performed without deteriorating image quality. Further, when an image is reproduced by frame thinning, the thinned frame is interpolated, so that the image can be reproduced without deteriorating the image quality.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, the present invention will be described with reference to the drawings.

FIG. 2 is a block diagram of an image / sound compression apparatus of an inter-frame difference encoding system showing an embodiment of the present invention. The image / audio compression apparatus shown in FIG. 2 includes a control unit 7, a hard disk 1, a frame memory 2, a frame buffer 3, an audio encoder 4, a memory 5, a keyboard 6, and a thinning information memory 11. The control unit 7 shown in FIG. 3 includes a device control unit 8, an encoding unit 9, and a compressed information extracting unit 10. The processing of the image / sound compression device is designated by an input from the keyboard 6, and the control unit 7 controls the entire device. The image signal stored in the frame memory 2 is separated into a luminance signal and a color difference signal and stored in the frame buffer 3. The difference between the image data stored in the frame buffer 3 and the image data stored in the frame memory is compression-encoded by the encoding means 9 and stored in the memory 5. The audio data is encoded by the audio encoder 4 and stored in the memory 5. The encoded compressed image / sound data is recorded on the hard disk 1.

The thinning information for each processing speed is stored in the thinning information memory 11, and is recorded in synchronism with the voice by thinning out the color difference according to the optimum thinning out from the thinning information.

FIG. 3 is a block diagram of an image / sound reproducing apparatus of an inter-frame difference encoding system showing an embodiment of the present invention. The image / sound reproducing apparatus shown in FIG. 3 includes a control unit 27, a hard disk 21, a frame memory 22, a frame buffer 23, a sound decoder 24, a memory 25, and a keyboard 26. 4 includes a device control unit 28, a decoding unit 29, and a reproduction information extracting unit 30. The processing of the image / sound reproducing apparatus is designated by an input from the keyboard 26, and the control unit 27 controls the entire apparatus. The compressed image / audio code recorded on the hard disk 21 is read and stored in the memory 25. The compressed image code stored in the memory 25 is decoded by the decoding means 29 and stored in the frame memory 22. The luminance signal and the chrominance signal stored in the frame memory 22 are combined to output an image signal. Data being calculated and the like are stored in the frame buffer 23 as a work memory for decoding image data. Also, the memory 25
Is decoded by the audio decoder 24 to output an audio signal. Also, by reading the thinning information inserted in the compression code and interpolating the color difference in accordance with the thinning from the thinning information, the reproduction is performed in synchronization with the audio.

FIG. 4 shows an example of an image and audio code format of the inter-frame difference encoding method. As shown in FIG. 4A, the code of the image and the sound includes a code SOI indicating the start of the compression code of one frame (or field), a header part (Head) of the compression code, and a code of one frame (or field). It is composed of an audio code (Audio), an image code (Video) of one frame (or field), and a code (EOI) indicating the end of the compression code of one frame (or field).

As shown in FIG. 4B, the header portion of the compression code includes a header portion size (length), an image horizontal size (width), an image vertical size (height), and the number of frames. (Frame), the number of effective bits of the color difference (depth), the horizontal value of the color difference (addx), and the vertical value of the color difference (ad)
dy) and a line difference number (line) for color difference. The thinning information is the depth and add of the header.
It is stored in x, addy, and line. depth is information of bit thinning, addx and addy are information of block thinning, and line is information of line thinning.

As shown in FIG. 4C, the image code includes a frame number (no) and a key frame flag (k).
ey: 1, a key frame), a color difference compression processing flag (flg: 1, a color difference compression code is present), a luminance compression code part (light), and a color difference compression code part (co
lor: flg = 1 only).
The thinning information is stored in (3) of the image code. When the compression processing flag is 0, it indicates that the color difference of the frame is thinned.

FIG. 5 shows an example of image thinning information. FIG.
As shown in (4), four pieces of thinning information (parameters) are set for each level of the number of pixels to be processed by the device, and tbit (the number of effective bits of color difference) and tadx (horizontal addition value of color difference) , Tady (added value of color difference in the vertical direction), tlin (line thinning number of color difference),
tmax (the maximum value of the average of the color difference differences). tbit is bit thinning information, tadx and tady are block thinning information, tlin is line thinning information, and tmax is information when frame thinning is performed.

The setting of each thinning information is set so that image processing can be performed within the image processing time of one frame required for synchronizing with sound. For example, if it takes 5 milliseconds to display a decoded image, it is necessary to perform decoding of one frame in about 28 milliseconds in order to perform processing at 30 frames / second. Since the image processing time is proportional to the number of pixels, there is no need to thin out when processing a smaller number of pixels than the number of pixels for which the image processing time is within 28 milliseconds. However, when processing a pixel number larger than the pixel number that requires an image processing time of 29 milliseconds or more, it is necessary to thin out.

In the example shown in FIG. 5, when the number of pixels is less than 19200, there is no need to thin out the image. When the number of pixels is 19200 to 38399, the image is thinned out to about 1/2 and 38400 to 57599.
In the case of, it is necessary to thin out to about 1/3. Therefore, when the number of pixels is less than 19200, the number of effective bits is 8 bits, the added value in the horizontal direction is 1, the added value in the vertical direction is 1, the number of thinned lines is 1, and the maximum value of the average of the color difference difference is 256. Is processed. When the number of pixels is 19200 to 38399, the number of effective bits is 8 bits, the added value in the horizontal direction is 2, the added value in the vertical direction is 2, the number of thinned out lines is 1, and the maximum value of the average of the color difference difference is Processed at 128. When the number of pixels is 38400 to 57599, the number of effective bits is 4 bits, the added value in the horizontal direction is 2, the added value in the vertical direction is 2, the number of thinned out lines is 2, and the maximum value of the average of the color difference difference is 64.

Next, each thinning process in this embodiment will be described. FIG. 6 is a diagram for explaining the bit number thinning of the color difference. FIG. 6A shows the original value of 8 bits, and FIG.
Indicates a value when the number of bits is reduced to 6 bits. In the example of FIG. 6, 43H of BO to B7 is shifted right by 2 bits to become 10H of B2 to B7, and is decimated to 6 bits. In this way, by thinning out the bits, the numerical value decreases and the ratio of the difference value concentrated near 0 increases, so that the compression rate increases and the reproduction speed increases.

FIG. 7 is a diagram for explaining the block thinning of the color difference. FIG. 7A shows the original image area.
(B) shows an image area in which blocks are thinned out. In the example of FIG. 7, the values of the pixels adjacent in the horizontal direction and the vertical direction are regarded as the same, and the 16 pixels A1 to A16 are A1,
It is thinned out to four pixels A3, A9 and A11. Since the number of times of calculating the difference is reduced by thinning out the blocks in this manner, the compression rate is increased and the reproduction speed is increased.

FIG. 8 is a diagram for explaining the line thinning of the color difference. FIG. 8A shows an original image area of 1 to 3 frames, and FIG. 8B shows an image area of 1 to 3 frames with thinned lines. In the example of FIG. 8, 16 pixels A1 to A16 in the first frame are not thinned out. The 16 pixels B1 to B16 in the second frame are thinned out on the odd lines by assuming that the values of the pixels on the odd lines are the same as the values of the pixels on the previous frame, and B5 to B8, B1
It is thinned out to eight pixels 3 to B16. Also, the 16 pixels C1 to C16 of the third frame are deemed to have the same values of the pixels of the even lines as the values of the pixels of the previous frame, and the even lines are thinned out to obtain eight pixels C1 to C4 and C9 to C12. Pixels are thinned out. Since the number of times of calculating the difference is reduced by thinning out the lines in this manner, the compression rate is increased and the reproduction speed is increased.

FIG. 9 is a diagram for explaining frame thinning of color difference. FIG. 9A shows an original image area of one to three frames, FIG. 9B shows an image area in a case where interpolation is not performed by thinning out two frames, and FIG. 9C shows thinning out two frames. 3 shows an image area when interpolation is performed.
In the example of FIG. 9, the pixels B1 to B16 of the two frames are thinned out.
The value of A16 is used. In the case of performing interpolation, a value (bi = Ai + (Ci−A
i) / 2 (i = 1 to 16)). Since the number of times of calculating the difference is reduced by thinning out the frames in this manner, the compression rate is increased and the reproduction speed is increased. Further, the image quality is improved by interpolating the frames thinned out from the previous and next frames.

The processing at the time of compression by the above configuration will be described. FIG. 10A is a flowchart for controlling compression of an image and sound. When the compression device starts as shown in the flowchart of FIG. 10A, the number of frames to be recorded and the file name are input from the keyboard 6 (step 3).
1) Extracting the thinning information from the thinning information memory 11 to create a header part and storing it in the memory 5 (step 3)
2) The input video signal is stored in the frame memory 2 (step 33), the luminance signal and the chrominance signal are extracted from the frame memory 2 (step 34), and the image is compressed (step 35). The audio signal is encoded by the audio encoder 4 and stored in the memory 5 (step 36).
The codes of the images and sounds stored in the memory 5 are recorded on the hard disk 1 (step 37). Next, it is determined whether or not the frame is the last frame (step 38). If not, the process returns to step 33, and if so, the process ends.

FIG. 10B is a flowchart for controlling reproduction of an image and a sound. When the playback device starts as shown in the flowchart of FIG. 10B, the number of frames to be played back and the file name are input from the keyboard 26 (step 41), the header portion is read from the hard disk 21 and stored in the memory 25. (Step 4
2) The image / audio code is read from the hard disk 21 and stored in the memory 25 (step 43), and the audio compression code stored in the memory 25 is decoded by the audio encoder 24 and output as an audio signal. (Step 44)
The image is reproduced (step 45), the luminance signal and the color difference signal are stored in the frame memory 22 (step 46), and the video signal is output from the frame memory 22 (step 4).
7). Next, it is determined whether or not the frame is the last frame (step 4).
8) If not, return to step 43; otherwise, end the process.

FIG. 11 is a flowchart of image compression. As shown in the flowchart of FIG. 11, it is determined whether or not the frame is a key frame (step 51). If not, the luminance is compressed (step 57), and the color difference is compressed (step 58). If so, the luminance value of the current frame is encoded by the encoding means 9 and stored in the memory 5 (step 52), and the luminance value of the current frame is stored in the frame buffer 3 (step 53). The color difference value of the current frame is encoded by the encoding means 9 and stored in the memory 5 (step 54), and the color difference value of the current frame is stored in the frame buffer 3 (step 55). Next, it is determined whether one frame has ended (step 56). If not, the process returns to step 52, and if so, the process ends. In the luminance compression, the luminance value of the previous frame is extracted from the frame buffer 3 (step 61), and the luminance value of the previous frame is stored in a variable A (step 62). Is taken out (step 63), and the luminance value of the current frame is stored in a variable B (step 64). Next, the difference value (BA) is encoded by the encoding means 9 and stored in the memory 5 (step 6).
5) The luminance value B of the current frame is stored in the frame buffer 3 (step 66). Next, it is determined whether one frame has ended (step 67). If not, the process returns to step 61, and if so, the process ends.

FIG. 12 is a flowchart of image reproduction. As shown in the flowchart of FIG. 11, it is determined whether or not the frame is a key frame (step 71). If not, the luminance is reproduced (step 77), and the color difference is reproduced (step 78). If so, the compression code of the image stored in the memory 25 is decoded by the decoding means 29 to extract a luminance value (step 72), and the luminance value is stored in the frame memory 22 (step 73). ), Memory 2
5, the decoding means 29 decodes the compression code of the image, extracts the color difference value (step 74), and stores the color difference value in the frame memory 22 (step 7).
5). Next, it is determined whether or not one frame has been completed (step 76). If not, the process returns to step 72, and if so, the process ends. In the luminance reproduction, the luminance value of the previous frame is extracted from the frame memory 81 (step 81), and the luminance value of the previous frame is stored in a variable A (step 82). The compression code is decoded by the decoding means 29 to extract a difference value (step 83), and the difference value is stored in a variable d (step 84). Next, the luminance value of the current frame (A +
d) is calculated and stored in the frame memory 22 (step 85). Next, it is determined whether or not one frame has ended (step 86). If not, the process returns to step 81, and if so, the process ends.

Next, a description will be given of the processing in the case where the compression is performed by thinning out the color difference bit number, the color difference block, and the color difference line.

FIG. 13 is a flowchart of color difference compression for thinning out the number of bits of color difference, thinning out blocks of color difference, and thinning out lines of color difference. As shown in the flow chart of FIG. 13, the start number y0 of the line, the effective bit number r, the horizontal direction addition value dx, and the vertical direction addition value dy are extracted by extracting the compression information (step 91). Next, the line start number y0 is stored in the variable y (step 92), 0 is stored in the variable x (step 93), and the color difference of the pixel at the position (x, y) of the previous frame from the frame buffer 3 is stored. (Step 94), the value of the color difference of the previous frame is stored in the variable A (step 95), and the value of the color difference of the pixel at the position (x, y) of the current frame is extracted from the frame memory 2. (Step 96), the color difference value of the current frame is stored in the variable B (Step 97), and the difference value ((BA) >> (8-r)) of the effective bit number r is calculated (Step 96). (Step 98), the difference value is encoded by the encoding means 9 and stored in the memory 5 (Step 99), and the color difference value B of the current frame is stored in the frame buffer 3 (Step 10).
0). Next, the horizontal addition value dx is added to the variable x (step 101), and it is determined whether the variable x is larger than the horizontal size of the image (step 102). If not, the process returns to step 94. If so, the vertical direction addition value dy is added to the variable y (step 103), and the variable y
Is larger than the vertical size of the image (step 104). If not, the process returns to step 93.
If so, the process ends.

FIG. 14 is a flowchart for extracting compressed information. As shown in the flowchart of FIG. 14, the corresponding index of the thinning information table is extracted from the image size to be processed (step 111), the index is stored in a variable n (step 112), and stored in the thinning information memory 11. The address of the thinning-out information table is extracted (step 113), the address is stored in a variable TBL (step 114), the effective bit number of color difference (TBL [n] [0]) is extracted and stored in a variable r ( Step 115), add the color difference in the horizontal direction (TB
L [n] [1]) is taken out and stored in a variable dx (step 116), and the vertical sum of the color difference (TBL [n]) is obtained.
[2]) is taken out and stored in a variable dy (step 1).
17), number of color difference line thinning out (TBL [n] [3])
Is taken out and stored in a variable L (step 118), the current frame number is taken out and stored in a variable F (step 119), and a start line number (the remainder when F is divided by L) is calculated. Stored in the variable y0 (step 12
0), the value of the variable L is stored in the variable dy (step 12).
1).

Next, a process for reproducing a compressed code by thinning out the number of color difference bits, thinning out color difference blocks, and thinning out color difference lines will be described.

FIG. 15 is a flow chart for reproducing a compressed code by thinning out the number of color difference bits, thinning out color difference blocks, and thinning out color difference lines. As shown in the flowchart of FIG. 15, the start number y0 of the line, the number of effective bits r, the horizontal direction added value dx, and the vertical direction added value dy are extracted by extracting the reproduction information (step 131). Next, the start number y0 of the line is stored in the variable y (step 13).
2) 0 is stored in the variable x (step 133), and the color difference value of the pixel at the position (x, y) of the previous frame is extracted from the frame memory 22 (step 134).
Is stored with the value of the color difference of the previous frame (step 13).
5) The decoding means 29 decodes the compressed code of the image stored in the memory 25 to obtain a difference value (step 136), and stores the difference value as a variable d (step 13).
7) The color difference value (A + (d << r)) of the current frame is calculated from the effective bit r and the difference value d and stored in the variable B (step 138), and the value of the variable B is stored in the frame memory 2
It is stored in the color difference of the pixel at the position of (x, y) 2 (step 139). Next, the value of y + 1 is stored in the variable j (step 140), the value of x + 1 is stored in the variable i (step 141), and the value of the variable B is stored in (i,
The color difference of the pixel at the position j) is stored (step 14).
2) Add 1 to the variable i (step 143), determine whether the variable i is greater than x + dx (step 144), and return to step 142 if not. If so, add 1 to the variable j (step 14
5) It is determined whether or not the variable j is larger than y + dy (step 146). If not, step 141 is performed.
Return to If so, add the horizontal direction added value dx to the variable x
Is added (step 147), and it is determined whether or not the variable x is larger than the horizontal size of the image (step 148).
Otherwise, return to step 134. If so, the vertical direction addition value dy is added to the variable y (step 149), and it is determined whether or not the variable y is larger than the vertical size of the image (step 150). If so, the process ends.

FIG. 16 is a flowchart of the reproduction information extraction. As shown in the flowchart of FIG. 16, the address of the header portion stored in the memory 25 is taken out (step 161), the address is stored in the variable HEAD (step 162), and the number of effective bits of the color difference (HEA)
D [5]) and store it in a variable r (step 1).
63) The horizontal addition value (HEAD [6]) of the color difference is extracted and stored in a variable dx (step 164), and the vertical addition value (HEAD [7]) of the color difference is extracted and stored in a variable dy. (Step 165) The line-thinning number of the color difference is extracted and stored in the variable L (Step 16).
6), take out the current frame number and store it in variable F (step 167), calculate the start line number (the remainder when F is divided by L) and store it in variable y0 (step 168); The value of the variable L is stored in the variable dy (step 169).

Next, a description will be given of a process in a case where a frame of a color difference is thinned out and compressed.

FIG. 17 is a flowchart of chrominance compression for thinning out chrominance frames. As shown in the flowchart of FIG. 17, the maximum value dmax of the sum of the differences, the horizontal addition value dx, and the vertical addition value dy are extracted by extracting the compression information (step 181). Next, 0 is stored in the variable m (Step 182), 0 is stored in the variable y (Step 183), and 0 is stored in the variable x (Step 18).
4), (x, y) of the previous frame from the frame buffer 3
The color difference value of the pixel at the position is extracted (step 18).
5) The value of the color difference of the previous frame is stored in the variable A (step 186), and the value of the color difference of the pixel at the (x, y) position of the current frame is extracted from the frame memory 2 (step 187). The color difference value of the current frame is stored in B (step 188), the difference value (BA) is calculated and stored in variable d (step 189), and the value of d is added to variable m (step 189). (Step 190), the horizontal direction addition value dx is added to the variable x (Step 191), and it is determined whether or not the variable x is larger than the horizontal size of the image (Step 19).
2) If not, return to step 185. If so, the vertical direction addition value dy is added to the variable y (step 193), and it is determined whether the variable y is larger than the vertical size of the image (step 194). If not, the process returns to step 184. If so, it is determined whether the value obtained by dividing the variable m by the number of pixels to be processed is greater than dmax (step 195). If so, the color difference compression processing flag is set to 1 and stored in the memory 5. (Step 196), color difference compression is performed from Step 92 of FIG. 13 (Step 197). Otherwise, the color difference compression processing flag is set to 0 and stored in the memory 5 (step 19).
8).

FIG. 18 is a flowchart for extracting compressed information. As shown in the flowchart of FIG. 18, the index of the corresponding thinning information table is extracted from the image size to be processed (step 201), the index is stored in a variable n (step 202), and stored in the thinning information memory 11. The address of the thinning-out information table is taken out (step 203), the address is stored in a variable TBL (step 204), the maximum value of the average of the color difference difference (TBL [n] [4]) is taken out, and the variable d is taken out.
max (step 205), and the compression information is extracted from step 115 of FIG. 14 (step 206).

Next, a description will be given of a process for reproducing a compressed code obtained by thinning out color difference frames without interpolation.

FIG. 19 is a flow chart for reproducing a compressed code by thinning out a color difference frame. As shown in the flowchart of FIG. 19, the color difference compression processing flag flg, the start number y0 of the line, the effective bit number r, the horizontal direction addition value dx, and the vertical direction addition value dy are taken out by taking out the reproduction information (step 211). Next, it is determined whether or not the compression processing flag is ON (step 212), and if so, the color difference reproduction is performed from step 132 in FIG. 15 (step 213). If not, the process ends without updating the color difference (step 214).

FIG. 20 is a flow chart for extracting reproduction information. As shown in the flowchart of FIG. 20, the compression processing flag is extracted from the compression code of the color difference stored in the memory 25 (step 221), and the compression processing flag is stored in a variable flg (step 222). 161 is extracted from the playback information (step 22).
3).

Next, a description will be given of a process in a case where a compressed code obtained by thinning out color difference frames is interpolated and reproduced.

FIG. 21 is a flow chart for reproducing a compression code by thinning out a color difference frame. As shown in the flowchart of FIG. 19, by extracting reproduction information, a color difference compression processing flag flg, a color difference average difference value, a line start number y0, an effective bit number r, a horizontal direction addition value dx, and a vertical direction addition value dy are extracted. Step 231). Next, it is determined whether or not the compression processing flag flg is ON (step 2).
32) If so, color difference reproduction is performed from step 132 in FIG. 15 (step 233). Otherwise, 0 is stored in the variable y (step 234), and 0 is stored in the variable x.
Is stored (step 235), the value of the color difference of the pixel at the position (x, y) of the previous frame is extracted from the frame memory 22 (step 236), and the value of the color difference of the previous frame is stored in the variable A. (Step 237), the average difference value of the color difference of the pixel at the position (x, y) is extracted from the frame buffer 23 (Step 238), the average difference value is stored in a variable d (Step 239), and the difference value d is obtained. , The color difference value (A + d) of the current frame is calculated and stored in the variable B (step 240), and the value of the variable B is stored in the frame memory 22.
Is stored in the color difference of the pixel at the position (x, y) (step 241). Next, the value of y + 1 is stored in variable j (step 242), the value of x + 1 is stored in variable i (step 243), and the value of variable B is stored in (i,
The color difference of the pixel at the position j) is stored (step 24).
4) 1 is added to the variable i (step 245), and it is determined whether or not the variable i is larger than x + dx (step 246). If not, the process returns to step 244. If so, 1 is added to the variable j (step 24).
7) It is determined whether or not the variable j is larger than y + dy (step 248).
Return to If so, add the horizontal direction added value dx to the variable x
Is added (step 249), and it is determined whether or not the variable x is larger than the horizontal size of the image (step 250).
Otherwise, return to step 236. If so, the vertical direction addition value dy is added to the variable y (step 251), and it is determined whether or not the variable y is larger than the vertical size of the image (step 252). If not, the process returns to step 235. If so, the process ends.

FIG. 22 is a flow chart for extracting reproduction information. As shown in the flowchart of FIG. 22, the compression processing flag is extracted from the compression code of the color difference stored in the memory 25 (step 261), the compression processing flag is stored in a variable flg (step 62), and the compression processing flag flg is stored. Is determined to be ON (Step 2)
63) Otherwise, proceed to step 276. If not, the compression processing flag is extracted from the compression code of the color difference of the next frame stored in the memory 25 (step 264), and it is determined whether or not the next frame has been subjected to the compression processing (step 265). Returns to step 264. If so, the frame interval of the compression process is stored in the variable F (step 266), 0 is stored in the variable y (step 267), 0 is stored in the variable x (step 268), and the memory 25 Is decoded by the decoding means 29 to obtain a difference value (step 269), and the average of the difference values (difference value /
F) is calculated, the average difference value is stored in the variable d (step 270), and the value of the variable d is stored in the frame buffer 23 as the color difference difference of the pixel at the position (x, y) (step 271). . The horizontal addition value dx is added to the variable x (step 272), and it is determined whether the variable x is larger than the horizontal size of the image (step 273). If not, the process returns to step 269. If so, the variable y
Is added to the vertical direction addition value dy (step 274).
It is determined whether the variable y is larger than the vertical size of the image (step 275), and if not, step 268.
Return to If so, the reproduction information is extracted from step 161 in FIG. 16 (step 276).

Next, a second embodiment of the present invention will be described with reference to FIGS.

FIG. 23 shows an image / image according to the second embodiment.
FIG. 2 is a block diagram of the audio compression device.

The basic configuration of the video / audio compression apparatus according to the second embodiment is the same as that of the video / audio compression apparatus according to the first embodiment shown in FIG. Indicate the same reference numerals.

As shown in FIG. 23, in addition to the components of the image / sound compression apparatus shown in FIG. 2, the image / sound compression apparatus further reads the information amount of the image signal to be processed. It has a timer 12 as a means.
The compression processing time is read by the timer-12, and the thinning method is changed as described later.

FIG. 24 shows a code format of an image and a sound of the inter-frame coding method in the image and sound compressing apparatus of the present embodiment.
Here is an example of a mat.

As shown in FIG. 24, the codes of the image and the sound are substantially the same as those shown in FIG.
Unlike the code shown in FIG. 4, each frame has a header portion (Head) of a compression code. Thereby, the thinning method can be changed for each frame.

The processing at the time of compression by the above configuration will be described. FIG. 25 is a flowchart for controlling the compression of the image and the sound. As shown in the flowchart of FIG. 25, when the compression apparatus starts, the number of frames to be recorded and the file name are input from the keyboard 6 (step 331).
The timer 12 is reset (step 332). Next, the input video signal is stored in the frame memory 2 (step 333), a luminance signal and a color difference signal are extracted from the frame memory 2 (step 334), and the image is compressed (step 335). The signal is encoded by the speech encoder 4 and stored in the memory 5 (step 33).
6) Record the codes of the images and sounds stored in the memory 5 on the hard disk 1 (step 337). Next, it is determined whether or not the frame is the last frame (step 338). If not, the process returns to step 333. If so, the process ends.

FIG. 26 is a flowchart for extracting compressed information. As shown in the flow chart of FIG. 26, the current time is read from the timer 12, stored in the memory 5 (step 411), and the index of the corresponding thinning information table of the image size to be processed is extracted (step 412). ), And store the index in the variable n (step 413). Next, it is determined whether or not the frame is the first frame (step 414).
Proceed to. If not, the time of the difference between the previous and current timer values stored in the memory 5 is calculated (step 41).
5) The difference time is stored in the variable dt (step 41).
6), the time to be processed by the variable dt (for example, 15F / S
In the case where the processing is performed by (1), it is determined whether the value is larger than about 66 ms) (step 417), and if so, 1 is added to the variable n (step 418). Otherwise, determine whether the variable dt is smaller than the time to process (step 419), if so, subtract 1 from variable n (step 420); Goes to step 421.

Next, the address of the thinning information table stored in the thinning information memory 11 is retrieved (step 421), and the address is stored in the variable TBL (step 422). Subsequently, the number of effective bits of the color difference (TBL
[N] [0]) is taken out and stored in a variable r (step 423), and the horizontal addition value of color difference (TBL [n])
[1]) is taken out and stored in a variable dx (step 4).
24), vertical sum of color difference (TBL [n] [2])
Is taken out and stored in a variable dy (step 425),
The line difference number (TBL [n] [3]) of the color difference is extracted and stored in a variable L (step 426), and the current frame number is extracted and stored in a variable F (step 42).
7) The start line number (the remainder when F is divided by L) is calculated and stored in the variable y0 (step 428), and the variable d is calculated.
The value of the variable L is stored in y (step 429). next,
The thinning information r, dx, dy, and L are added to the header and stored in the memory 5 (step 430).

FIG. 27 is a flowchart for extracting compressed information. As shown in the flowchart of FIG. 27, the compression information is extracted from step 411 of FIG. 26 (step 501), and the average maximum value of the sum of the differences (TBL [n])
[4]) is taken out and stored in a variable dmax (step 502).

In the first and second embodiments, the image is processed by the luminance and the color difference. However, even in the case of RGB of three primary colors, the processing can be similarly performed by thinning out.

[0056]

According to the present invention, the image / sound compression apparatus performs the image thinning processing in accordance with the thinning information having a plurality of parameters. Therefore, when compressing the image by the frame thinning, the image quality is not reduced. Can be compressed.
Therefore, the processing time of the image can be reduced, and the image and the sound can be simultaneously recorded in synchronization with the sound. Further, in the image / audio reproducing apparatus, when reproducing the image data recorded by the image / audio compressing apparatus, the color difference is interpolated according to the thinning information provided in the image data. Can be played without dropping. Therefore, since the image reproduction processing time is shortened, it is possible to simultaneously reproduce the image and the audio while synchronizing with the audio while preventing the image quality from deteriorating.

[Brief description of the drawings]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram for explaining a conventional example of an interframe coding method, in which (A) is a flowchart thereof and (B) is a conceptual diagram thereof.

FIG. 2 is a block diagram of an image / audio compression apparatus according to the first embodiment of the present invention.

FIG. 3 is a block diagram of the image / sound reproducing apparatus according to the first embodiment of the present invention.

4A and 4B are diagrams showing code formats of an image and a sound in the first embodiment of the present invention, wherein FIG.
(B) is a configuration diagram of a header portion, and (C) is a configuration diagram of a compression code of an image.

FIG. 5 is a configuration diagram of thinning information.

FIG. 6 is a diagram for explaining bit thinning of color difference;
(A) is a diagram showing the state of the original 8-bit number,
(B) is a diagram showing a state where bits are thinned out to 6 bits.

FIG. 7 is a view for explaining block thinning of color difference;
(A) is a diagram showing the values of the original pixels, and (B) is a diagram showing the values of the thinned pixels.

FIG. 8 is a diagram for explaining line thinning of a color difference;
(A) is a diagram showing the values of the original pixels, and (B) is a diagram showing the values of the thinned pixels.

FIG. 9 is a diagram for explaining frame thinning of a color difference;
(A) is a diagram showing the original pixel value, (B) is a diagram showing the non-interpolated state of the thinned pixel values, and (C) is a diagram showing the interpolated pixel value. FIG.

FIGS. 10A and 10B are flowcharts showing the operation of the first embodiment of the present invention, wherein FIG. 10A is a flowchart showing an image / audio compression operation, and FIG. 10B is a flowchart showing an image / audio reproduction operation.

FIGS. 11A and 11B are flowcharts showing the operation of the first embodiment of the present invention, wherein FIG. 11A is a flowchart showing an image compression operation, and FIG. 11B is a flowchart showing a luminance compression operation.

FIGS. 12A and 12B are flowcharts showing an operation of the first embodiment of the present invention, in which FIG. 12A is a flowchart showing an image reproducing operation, and FIG. 12B is a flowchart showing a luminance reproducing operation.

FIG. 13 is a flowchart illustrating an operation of color difference compression according to the first embodiment of the present invention.

FIG. 14 is a flowchart showing an operation of extracting compressed information in the first embodiment of the present invention.

FIG. 15 is a flowchart illustrating a color difference reproduction operation according to the first embodiment of the present invention.

FIG. 16 is a flowchart showing an operation of extracting reproduction information according to the first embodiment of the present invention.

FIG. 17 is a flowchart illustrating an operation of color difference compression according to the first embodiment of the present invention.

FIG. 18 is a flowchart showing an operation of extracting compressed information in the first embodiment of the present invention.

FIG. 19 is a flowchart showing a color difference reproducing operation according to the first embodiment of the present invention.

FIG. 20 is a flowchart showing an operation of extracting reproduction information according to the first embodiment of the present invention.

FIG. 21 is a flowchart illustrating a color difference reproducing operation according to the first embodiment of the present invention.

FIG. 22 is a flowchart showing an operation of extracting reproduction information according to the first embodiment of the present invention.

FIG. 23 is a block diagram of an image / audio compression apparatus according to a second embodiment of the present invention.

24A and 24B are diagrams showing code formats of an image and a sound in the second embodiment of the present invention, wherein FIG. 24A is a configuration diagram, FIG. 24B is a configuration diagram of a header portion, and FIG. 3 is a configuration diagram of a compression code of FIG.

FIG. 25 is a flowchart showing an image / audio compression operation according to the second embodiment of the present invention.

FIG. 26 is a flowchart showing an operation of extracting compressed information according to the second embodiment of the present invention.

FIG. 27 is a flowchart showing an operation of extracting compressed information according to the second embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Hard disk 2 Frame memory 3 Frame buffer 4 Audio encoder 5 Memory 6 Keyboard 7 Control part 8 Device control means 9 Encoding means 10 Compressed information extraction means 11 Thinning-out information memory 12 Timer 21 Hard disk 22 Frame memory 23 Frame buffer 24 Audio Decoder 25 Memory 26 Keyboard 27 Control unit 28 Device control means 29 Decoding means 30 Reproduction information extracting means

Claims (12)

(57) [Claims] 1. An image of a current frame is separated into luminance and chrominance, and a difference between luminance and chrominance between a current frame and a previous frame is compression-encoded, and the compressed image signal and audio signal are synchronized with a recording medium. In an image / sound compression apparatus for recording and converting data, thinning information storing means for storing thinning information of color differences predetermined for each processing speed of an image signal to be processed, and thinning information corresponding to the processing speed of the image signal Compression information extracting means for extracting the image data from the thinning information storage means, and compression encoding means for compressing and encoding the inter-frame difference by thinning out the color difference according to the thinning information extracted from the compressed information extracting means. Image / audio compression device. 2. A means for extracting the thinning-out information from a compressed code of an image and a sound provided with the thinning-out information of the color difference recorded by the image / sound compressing apparatus according to claim 1, and interpolating the color difference according to the thinning-out information. And a decoding means for decoding the compression code. 3. The image / sound compression apparatus according to claim 1, wherein the chrominance culling information is information of chrominance bit number culling predetermined in accordance with a processing speed of an image signal to be processed. Wherein the number of bits is reduced and the inter-frame difference is compression-encoded by the compression-encoding means. 4. A means for extracting the thinning information from a compression code of an image and a sound provided with the information of the bit number thinning of the color difference recorded by the image / sound compression apparatus according to claim 3, and the thinning information Decoding means for interpolating and decoding the number of bits of the color difference in accordance with
Audio playback device. 5. The image / sound compression apparatus according to claim 1, wherein the color difference thinning information is a color difference block thinning predetermined according to a processing speed of an image signal to be processed. An image / speech compression apparatus comprising information, wherein the compression rate is increased as the processing speed is lower, and the inter-frame difference is compression-coded by the compression-coding means. 6. A means for extracting said thinning-out information from a compressed code of an image and a sound provided with information of said color difference block thinning-out recorded by the image / sound compressing apparatus according to claim 5, and according to said thinning-out information. Decoding means for interpolating and decoding blocks of color difference.
Audio playback device. 7. The image / sound compression apparatus according to claim 1, wherein the color difference culling information is predetermined in accordance with a processing speed of an image signal to be processed. An image / speech compression apparatus including information on color difference line thinning-out, thinning out the line, and compression-coding the inter-frame difference by the compression coding means. 8. A means for extracting the thinning-out information from a compression code of an image and a sound provided with the color-difference line thinning-out information recorded by the image / sound compression apparatus according to claim 7, and a color difference according to the thinning-out information. Decoding means for interpolating and decoding the line. 9. The image / audio compression apparatus according to claim 1, wherein a difference value of a color difference between pixels of the current frame and the previous frame is determined, and the frames are thinned out. An image / speech compression apparatus characterized in that a difference between frames is compression-coded. 10. An image / audio compression apparatus according to claim 9, further comprising means for interpolating and decoding a frame of a color difference between the image and the image thinned out from the compression code of the audio. Image / audio playback device. 11. The image / sound reproduction apparatus according to claim 10, further comprising means for interpolating and decoding a color difference frame thinned out from the color difference frames of the preceding and succeeding images. Playback device. 12. An image of a current frame is separated into luminance and chrominance, and an inter-frame difference between the current frame and the previous frame of luminance and chrominance is compression-coded to synchronize the compressed image signal and audio signal with a recording medium. In an image / sound compression apparatus for recording and recording data, thinning information storage means for storing thinning information of color differences predetermined for each processing speed of an image signal to be processed, and thinning information corresponding to the processing speed of the image signal Compressed information extracting means for extracting the color difference from the compressed information extracting means, compressing and encoding the inter-frame difference according to the thinning information extracted from the compressed information extracting means, An image / audio compression apparatus, comprising: reading means for reading an information amount of an image signal to be obtained.