JPH0795565A - Device and method for restoring image data - Google Patents

Abstract

PURPOSE:To retrieve an index on the side of restoration even when encoding only the block area changing an image up to a preceding frame. CONSTITUTION:Discrimination information and a variable length code is separated from code data by a code separating means 12, and an image is decoded from the separated variable length code by a decoding means 18 for each block. The number of valid blocks for one frame is counted from the separated discrimination information by a valid block counting means 14, only the frame number provided with the valid blocks more than a threshold value is restored by an image restoring means 20 while using a restoration discriminating means 15 and the block image signal decoded by the decoding means 18, and the index can be retrieved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data decompression apparatus and method for decompressing an image from coded data obtained by compressing image data, and in particular, dividing each of a plurality of images into blocks composed of a plurality of pixels, The present invention relates to an image data restoration device and method for restoring an image from coded data obtained by orthogonally exchanging pixels in the image.

In order to store image data whose information amount is incomparably larger than that of numerical data, especially halftone image and color image data, or to transmit at high speed and high quality, gradation for each pixel is required. The values need to be encoded efficiently.
As a highly efficient compression method of image data, for example, an adaptive discrete cosine transform coding method (Adaptive Discrete Cosine Tra
nsform, hereinafter abbreviated as "ADCT").

ADCT divides an image into blocks of 8 × 8 pixels, converts the pixel signals of each block into coefficients of spatial frequency distribution by two-dimensional discrete cosine transform (hereinafter referred to as “DCT”), and It is quantized by a threshold value adapted to, and the obtained quantized coefficient is encoded by a Huffman table obtained statistically.

[0004]

2. Description of the Related Art By the way, a moving image coding system can obtain a higher compression rate than a still image coding system, but on the other hand, it has drawbacks such as a large circuit scale, low image quality, and high cost. As a method for solving these problems, the inventors of the present application coded only blocks that changed from the reference image up to the previous frame, and coded valid blocks or non-coded invalid blocks. There has been proposed an image data encoding / restoring method and device which are sent together with information indicating that (Japanese Patent Application No. 4-77957).

FIG. 8 shows a basic configuration of an encoding device proposed by the inventors of the present application. From the input terminal 100, image data obtained by dividing a frame image into blocks each having 64 pixels by 8 pixels vertically by 8 pixels horizontally is input. Effective block determination unit 104
Is a block that is stored in the reference frame holding unit 102 and is used as a reference for change determination, and reads a block at the same position as a reference block image to obtain a correlation. If there is a change in the input block image, it is determined to be an effective block. However, if there is no change, it is determined that the block is invalid.

The encoding unit 106 is an effective block discriminating unit 10.
Only the block which has been changed as a valid block in 4 is encoded into a variable length code. At this time, the effective block area of the reference frame holding unit 102 is rewritten and updated with the input block image. From the output terminal 108, the variable length code for one frame by the encoding unit 106 is added with the information indicating the valid block or the invalid block from the valid block determining unit 104 as the determination information and output.

The decompression device of FIG. 9 inputs the code data sent from the coding device of FIG. 8 from the input terminal 110, and first, the code separation unit 112 separates the discrimination information into the variable length code. The decoding unit 114 decodes the image in block units from the variable length code. At this time, the address generator 116 recognizes the valid block that is the restoration target from the discrimination information, generates an address signal, writes the decoded pixel signal for one block into the restored image holding unit 118 that functions as a frame memory, and validates it. Update only the block area.

That is, in a continuous moving image, when the current frame is compared with the previous frame, there is a moving image area (valid block area) that has changed and a still area (invalid block area) that has not changed at all. At this time, only the changed area is coded using the still image coding method and transmitted together with the discrimination information indicating the position of the changed area, and the decoding side decodes the coded data and only the moving image area based on the discrimination information. To update. By performing these processes, high image quality with a high compression rate is secured.

Conventionally, there is an index search as a moving image search method. This is a method to quickly retrieve the moving image that the user needs by restoring and displaying consecutive moving images while skipping any number of frames.
It is an essential element in moving image applications. In the conventional index search, when an image is individually recorded for each frame like a laser disk, an arbitrary number of frames may be skipped and displayed.

In addition, CCITT H.264, which is a videophone / videoconference telephone coding system. In a moving image compression method such as 261, in order to prevent deterioration of image quality due to accumulated errors, compression is performed using an independent intra frame that does not use information of the previous frame every few frames. Therefore, the index search can be performed by recognizing and displaying only the intra frame on the restoration side.

[0011]

However, in the method proposed by the inventors of the present application for encoding only a moving image area having a change, whether the effective block is correlated with the reference frame image or not. Different from the case of encoding each frame independently, it is possible to obtain a high compression rate because it determines whether it is an invalid block and encodes only valid blocks, but random access and skipping of arbitrary frames are not possible, so index search There was a drawback that you couldn't.

An object of the present invention is to provide an image data reconstructing apparatus and method capable of performing an index search on the reconstructing side even in the case where only a region where there is a change in the image up to the previous frame is encoded.

[0013]

FIG. 1 is a diagram for explaining the principle of the present invention, taking the device configuration as an example. First, the decompression device of the present invention divides each continuous image into blocks of a predetermined number of pixels, and sequentially determines from each piece of code data including discrimination information indicating the presence or absence of encoding and a variable length code of the encoded block. Restore the image.

According to the present invention as such an image data restoration device, a code separation means 12 for separating the discrimination information and the variable length code from the code data, and the image is decoded block by block from the separated variable length code. The decoding means 18, the effective block counting means 14 for counting the number of effective blocks for one frame from the separated discrimination information, and the effective block count result are compared with a predetermined threshold value, and if the threshold value is exceeded, image restoration is performed. Restoration determination means 15 for instructing and restoration determination means 1
The image restoration unit 20 restores the frame image using the block image signal decoded by the decoding unit 18 based on the restoration instruction of No. 5.

If the processing block is an invalid block that has not been encoded, the image restoration means 20 copies the image of the effective block in the vicinity. As the copy block, the image of the valid block spatially adjacent to the invalid block area is copied. Further, a skip frame discriminating unit 16 is provided to count the number of skip frames which are not restored, and when the count value is equal to or more than a predetermined threshold value, the image restoring unit 20 is allowed to restore the frame image, and this total is counted. When the numerical value is smaller than the threshold value, the skip frame discriminating means 16 for prohibiting the restoration of the frame image by the image restoring means 20 is provided.

[0016]

According to the image data restoration device and method of the present invention, the following actions can be obtained. In general, for visual images such as television images, the camera switches every few seconds. This is called a scene change, but since there is no correlation with the frame image until immediately before the scene change, all blocks are coded as effective blocks.

Therefore, at the time of a scene change, most blocks in a frame are effective blocks, so that the code amount is outstanding. The same applies not only to a scene change, but also to an intra frame in which the reference frame is refreshed. The intra frame can be easily detected by searching the discrimination information indicating the valid block or the invalid block included in the code data.

Therefore, by extracting only the intra frame from the coded data and restoring it, the index search of the moving image can be performed. Further, depending on the image, even if a scene change occurs, it may have an area that does not change much before the scene change as an invalid block. This means that the higher the threshold used when determining the effective block by obtaining the correlation with the reference frame, the more the invalid blocks in the scene change.

Therefore, when the scene change is discriminated and the frame image is restored, the information of the area which is the invalid block is lost and the image quality is deteriorated. Therefore, by copying the image of the effective block in the vicinity of the position of the invalid block, an image with no discomfort can be restored. In particular, the process can be simplified by copying the decoded valid block image stored in the block buffer immediately before or after the invalid block to the invalid block position.

Further, depending on the scene, there are cases where images having a small correlation with the previous frame, that is, images having a large change are continuous. In such a case, the number of extracted intra frames coincides with the number of original images, resulting in continuous restoration display instead of index search. Therefore, the number of skip frames between the previously extracted intra frame and the next extracted intra frame is counted, and if the number is less than a certain value, the intra frame is not restored and the continuous search of the intra frame is suppressed to search the index. Is possible.

[0021]

FIG. 2 is a block diagram of an embodiment showing an embodiment of a restoring device according to the present invention. The restoration device of FIG. 2 has an input terminal 1
0, the code separation unit 12, the effective block counting unit 14, the restoration determination unit 15, the skip frame determination unit 16, the variable length decoding unit 18, and the image restoration unit 20.

Details of the image restoration unit 20 are composed of a block image buffer 24, a frame memory 26, a frame memory control unit 30 and a write address counter 32, as shown in FIG. Here, the moving image code data restored by the restoration device of the present invention has the format configuration shown in FIG. A moving picture start code 34 is provided at the beginning of the moving picture code data. The video start code 34 is SOM (St
art of Motin).

Subsequent to the moving image start code 34, an arbitrary number of n frames of encoded data 36 are arranged as shown by frame numbers 1, 2, 3, ... N. A moving image end code 38 is provided at the end of the encoded data 36. The moving picture end code 38 is indicated by EOM (End of Motion). The coded data of each frame is frame number 1
As shown by taking out the code start code 40,1
The frame includes a code 42 and a code end code 44.

The code start code 40 is SOI (Start of I
mage), and the code end code 44 is EOI (End of
Image, m). FIG. 5 shows details of the code 42 for one frame shown in FIG. The code for one frame is composed of a header 46 and a code section 48. JP in the header 46
In the case of EG, image information such as image size, number of components, and coding order, a quantization threshold used for coding, and information such as a Huffman table used for variable length coding are provided.

The coding unit 48 is composed of discrimination information and a variable length code. In this embodiment, since the encoding of the color signal of the color difference space consisting of the y component, the Cb component and the Cr component is taken as an example, the y component, the Cb component and the Cr component.
Discrimination information 52, 56, 60 and variable length codes 54, 58, 62 are provided for each of the components. Of course, the discrimination information and the variable length code are for one frame.

The discriminant information contains block information 64 inside, as shown by taking out the y component discriminant information 52. This block information 64 is 6 pixels of 8 pixels vertically and 8 pixels horizontally.
It shows whether each block is a coded valid block or a coded invalid block when a frame image is divided into blocks with 4 pixels as one block.

More specifically, in a predetermined block order, bit 1 is stored in the form of a one-dimensional bit stream in which bit 1 is stored for valid blocks and bit 0 is stored for invalid blocks. ing. Therefore, by counting the number of bits 1 in the block information 64, the number of valid blocks in this frame can be known.
In addition, it is possible to determine the number of bits from the beginning of the position of the effective block.

Referring again to FIG. 2, the encoding / separating unit 12
Inputs the code data shown in FIGS. 4 and 5 from the input terminal 10, and separates the discrimination information and the variable length code for each frame. The separated discrimination information is the effective block counting unit 14
Given to. As shown in FIG. 5, since the block information 64 is included in the discrimination information, the number of bits 1 indicating the effective block in the bit stream as the block information is counted.

The count result of the valid blocks counted by the valid block counting unit 14 is given to the restoration discriminating unit 15 and compared with a predetermined threshold value. That is, when the search mode is set, the restoration determination unit 15 determines that a frame having an effective block number exceeding the threshold value is a so-called intra frame, and the variable length decoding unit 18 determines the effective block to restore the search image. Instruct to decode variable-length code.

The image restoration unit 20 is controlled based on the discrimination information separated by the code separation unit 12, and restores the frame image using the image decoded for each block by the variable length decoding unit 18. Specifically, as shown in FIG. 3, the image data for one block decoded only for the effective block is input from the input terminal 22 and stored in the block image buffer 24.

At this time, the frame memory control unit 32 is provided with the discrimination information separated by the code separation unit 12, and if it is a valid block, the write address of the block area to be restored in the frame memory 26 is set to the write address counter 3
The pixel signal for one block which is generated in 2 and is held in the block image buffer 24 is written in the frame memory 26.

On the other hand, when the frame memory control unit 30 determines an invalid block, the block image buffer 24
Since the restored image data is not stored in the block image buffer 24 to the frame memory 26,
Writing to the invalid block area is not performed. In this case, an image of a valid block in a predetermined adjacent area to the invalid buffer area is displayed in the frame memory 26.
And is temporarily held in the block image buffer 24.

After that, the write address counter 32 generates a write address of the invalid block, and the image data of the valid block area held in the block image buffer 24 is written in the area of the invalid block which is the current processing target of the frame memory 26. Copy with. By writing the decoded image data in the valid block and copying the image data of the valid block that has already been written in the invalid block area, the frame image determined as the intra frame can be restored in the frame memory 26.

On the other hand, the copy processing of the image of the valid block area with respect to the invalid block area of the frame memory 26 in the image restoration unit 20 is performed by the block image buffer 24.
You may make it copy using the image data of the effective block stored in. For example, when the valid block is switched to the invalid block, the block image buffer 2 by reading the image data from the valid block adjacent to the invalid block of the frame memory 26
4 is not rewritten, the image data of the block image buffer 24 restored in the previous valid block is held as it is, and the image data of this valid block is written in the invalid block area and copied.

By using the image data of the effective block stored in the block image buffer 24 for copying in this way, it is not necessary to read from the effective block area of the frame memory 26, and the frame memory 26 is correspondingly read.
The copying process of the invalid block area can be easily performed.
It should be noted that, except for copying the image data restored in the valid block to the invalid block immediately after the valid block, the copying process for the frame memory 26 is not performed while the invalid block is being processed, and the process from the invalid block to the valid block is performed. After writing the decoded image data of the block image buffer 24 to the valid block area after switching to the valid block, the current block image buffer 24
It is also possible to copy the image data of the effective block held in the.

This is a case of copying the image data of the valid block after the invalid block. Referring again to FIG. 2, a skip frame discriminating unit 16 is further provided in this embodiment. The skip frame discriminating unit 16 counts the number of skip frames that are not continuously restored between an intra frame in which the number of effective blocks by the restoration discriminating unit 15 is equal to or more than a threshold value and the next intra frame,
When the number of skip frames is smaller than a predetermined threshold value, the intraframe restoration processing by the image restoration unit 20 is prohibited.

On the other hand, when the number of skip frames exceeds the threshold, the image restoration unit 20 is allowed to restore the intra frame. This is to prevent a large amount of change in the image of each continuous moving image, which is continuously discriminated by the restoration discriminating unit 15 as an intra frame and is reconstructed and displayed by the image reconstructing unit 20. FIG. 6 is an explanatory diagram showing changes in the code amount per frame with respect to the number of frames of moving image data formed of continuous frame images. In a moving image, especially a visual moving image such as a movie or a television, a scene change occurs in which the camera switches every few seconds.

In this scene change, since there is no correlation with the immediately preceding frame image, all blocks are coded as effective blocks. In FIG. 6, the part indicated by the broken line indicates a scene change, and it is understood that the code amount remarkably increases, and as a result, the number of effective blocks greatly increases. Therefore, by setting a threshold value capable of catching such a scene change in the restoration discriminating unit 15 in FIG. 2, the first frame image after the scene change can be restored as an intra image and an index search can be performed. .

CCITT H.M. The code data of intra frames performed every few frames as in 261 and the code data accompanying the refreshing of the reference frame in the coding apparatus already proposed by the inventors of the present application are also frames for index search as intra frames. Images can be restored. FIG. 7 is a flowchart showing the restoration process of FIG. First, in step S1,
It is checked whether there is an image end from the moving picture end code (EOM) 38 shown in FIG.
Proceed to step 2 to check whether the search mode is set.

If the search mode has been set by an operator or the like, the process proceeds to step S3 and the subsequent steps.
In step S3, the discrimination information and the variable length code are separated from the code data, and in step S4, the number of effective blocks is counted from the block information included in the discrimination information. Then step S
In step 5, it is checked whether or not the count value of the number of valid blocks is equal to or more than a predetermined threshold value.
It is checked whether the number of frame skips currently counted is equal to or more than a threshold value.

If the current number of frame skips exceeds the threshold value, the frame image restoration processing shown in steps S7 to S10 is executed. In the frame image restoration processing, it is first checked in step S7 whether or not it is a valid block. If it is a valid block, the block image is decoded in step S8 and written in the valid block area of the frame memory in step S9.

On the other hand, if the block is not a valid block in step S7, that is, if it is an invalid block, the process proceeds to step S11 to copy the image of the valid block area adjacent to the invalid block area. Such image decoding and copying for each block is repeated until all blocks are completed in step S10.

On the other hand, if the count value of the effective blocks is smaller than the threshold value in step S5, the frame image is not restored, the number of frame skips is incremented by 1 in step S12, the process returns to step S1, and the code of the next frame is coded. Process the data. If the count value of the effective blocks is equal to or larger than the threshold value in step S5 and the process proceeds to step S6,
If the number of frame skips at this time is smaller than the threshold, the frame image is not restored even if it is determined to be an intra frame, and skip processing is performed until the number of frame skips exceeds the threshold in step S12.

For this reason, the change in each image is large, and even if the determination is made only based on the count value of the effective blocks as all intra frames and the frame images are continuously restored, the threshold value is exceeded in step S6. By prohibiting restoration of frame images until the number of frame skips is reached, restoration of consecutive frame images is prohibited, and image restoration for index search having a predetermined frame skip number interval can be realized.

When the search mode has not been set in step S2, the process proceeds to step S13, and normal restoration processing is performed. This normal restoration process is performed in steps S3 to S
The processing is the processing excluding the processing of S4, S5, S6, and S11 in 10, and continuous image restoration is performed. It should be noted that in the embodiment of FIG. 2, the skip frame determination unit 16 is provided to enable image restoration with a predetermined number of skip frames.
However, the decompression device may be configured without the skip frame determination unit 16.

In the encoding unit 48 of FIG. 5, the variable length code of the color difference space is taken as an example, but the RGB space and L
^It can be applied as it is to image data in other color spaces such as ^* , a ^* , b ^* (lightness, saturation, hue space).

[0047]

As described above, according to the present invention, a series of moving images can be reproduced by recognizing and restoring an intra frame whose correlation with the images up to all frames is completely eliminated from the discrimination information included in the coded data. Only intra-frames can be extracted and restored from the image, and the forward direction of video code data,
The index search in the reverse direction and further in the random direction can be appropriately performed.

[Brief description of drawings]

FIG. 1 is an explanatory view of the principle of the present invention.

FIG. 2 is a configuration diagram of an embodiment showing an embodiment of a restoration device according to the present invention.

FIG. 3 is a configuration diagram of an embodiment showing details of an image restoration unit in FIG.

FIG. 4 is a format configuration diagram of moving image code data restored by the present invention.

5 is a format configuration diagram showing details of code data for one frame in FIG.

FIG. 6 is an explanatory diagram showing a change in code amount due to a scene change.

FIG. 7 is a flowchart showing a restoration process of the present invention.

FIG. 8 is a block diagram of an encoding device that the present inventors have already proposed.

FIG. 9 is a block diagram of a restoration device that the present inventors have already proposed.

[Explanation of symbols]

10, 22, 28: Input terminal 12: Code separation unit (code separation unit) 14: Effective block counting unit (effective block counting unit) 15: Restoration determination unit (restoration determination unit) 16: Skip frame determination unit (skip frame determination) 18) Decoding unit (Decoding unit) 20: Image restoration unit (Image restoration unit) 24: Image block buffer 26: Frame memory 30: Frame memory control unit 32: Write address counter 34: Video start code (SOM; Start of) Motion) 36: Encoded data for n frames 38: Moving image end code (EOM; End of Motion) 40: Code start code (SOI; Start of Image) 42: Code for one frame 44: Code end code (EOI; End of Image) 46: Header 48: Encoding part 52: y component discrimination information 54: y component variable length code 64: block information

Claims (6)

[Claims] 1. A continuous image from code data that is divided into blocks each having a predetermined number of pixels for each continuous image and that includes, for each block, discrimination information indicating the presence or absence of encoding and a variable length code of the encoded block. In a device for restoring image data, the code separating means (12) for separating discrimination information and a variable length code from the code data, and a decoding means (18) for decoding an image for each block from the variable length code. A valid block counting means (14) for counting the number of valid blocks for one frame from the discrimination information, and a count result of the valid block counting means (14) and a predetermined threshold value are compared, In this case, a restoration discriminating means (15) for instructing image restoration and a block image signal decoded by the decoding means (18) based on the restoration instruction of the restoration discriminating means (15) are used. Image data restoration apparatus characterized by comprising an image restoration means (20) to restore over beam image. 2. The image data restoration device according to claim 1, wherein the image restoration means (20) restores an image of a valid block in the vicinity when the processing block is an invalid block which is not encoded. An image data restoration device characterized by copying. 3. The image data restoration device according to claim 2, wherein the image restoration means (20) copies an image of a valid block spatially adjacent to an invalid block region. Data recovery device. 4. The image data restoration device according to claim 1, further comprising: counting the number of skip frames that are not restored, and if the count value is equal to or larger than a predetermined threshold value, the image restoration means ( 20) allows restoration of the frame image, and when the count value is smaller than the threshold value, the image restoration means (2
The image data restoration device is provided with a skip frame discrimination means (16) for prohibiting restoration of the frame image by 0). 5. A continuous image from code data that is divided into blocks each having a predetermined number of pixels for each continuous image, and includes, for each block, discrimination information indicating the presence or absence of encoding and a variable length code of the encoded block. In the image data restoration method for restoring the code data, a code separation step of separating the discrimination information and the variable length code from the code data, a decoding step of decoding an image for each block from the variable length code, and 1 from the discrimination information A valid block counting step of counting the number of valid blocks for the frame, comparing a count result of the valid block counting step with a predetermined threshold value, and a restoration determination step of instructing image restoration only when the threshold value or more is exceeded, An image restoration process for restoring a frame image using the block image signal decoded in the decoding process based on a restoration instruction in the restoration determination process. Image data restoration method to be. 6. The image data restoration method according to claim 5, further comprising: counting the number of skip frames that have not been restored, and if the count value is equal to or greater than a predetermined threshold value, in the image restoration process. An image data restoration method comprising a skip frame determination step of permitting restoration of a frame image and prohibiting restoration of a frame image in the image restoration step when the count value is smaller than the threshold value.