JPH07131794A - Method and device for coding picture data - Google Patents

Abstract

PURPOSE:To secure a transmission frame number in the unit time by applying a quantization step smaller than a quantization step applied to other frame to a frame subject to scene change so as to improve the picture quality of a decoded picture at scene change. CONSTITUTION:A discrimination means 102 discriminates the presence of correlation with respect to a block corresponding to a picture of a preceding frame latched in a reference frame latch means 101. A detection means 111 detects a frame whose picture characteristic is remarkably changed from that of a preceding frame based on discrimination information from the means 102 as a scene change. Thus, a changeover means 112 outputs a value smaller than a usual value as a quantization step in response to a detection signal of a scene change of the means 111. A coding means 103 uses the quantization step of the means 112 to apply information processing of each frame.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image data encoding method and apparatus for encoding a moving image by encoding a series of still images for each block consisting of a plurality of pixels.

The data amount of a multi-valued image (hereinafter, simply referred to as an image) such as a halftone image or a color image is so huge that it cannot be treated as it is like other numerical data. For this reason, the image is encoded and the amount of data is compressed with high efficiency, and then stored in the image database or transmitted through the communication line.

As a method of compressing the amount of image data,
There are various methods such as a two-dimensional discrete cosine transform method (DCT method) and a predictive coding method for coding a difference between image data representing the color of each pixel and a predicted value obtained from surrounding image data.

The DCT method is one in which an image is subjected to cosine transform for each block of 8 × 8 pixels and the obtained transform coefficient is encoded. In particular, the adaptive two-dimensional cosine transform method (ADCT method) quantizes transform coefficients using a threshold value adapted to vision, and then encodes
It is known that a high compression rate can be obtained.

This ADCT method is a working group jointly established by the International Telegraph and Telephone Consultative Committee (CCITT) and the International Organization for Standardization (JPEG).
p), the international standard encoding method (J
PEG system). The JPEG method is a method of encoding the quantized coefficients obtained by the above-described ADCT method using a Huffman code table obtained by statistically checking the appearance frequency of the code.

[0006] The JPEG system is a coding system corresponding to still images, but since it is simple and can obtain high image quality, it is expected to be applied to applications such as video conferences and video telephones for transmitting moving images. .

[0007]

2. Description of the Related Art FIG. 4 shows a basic configuration of a JPEG still image encoding apparatus. In FIG. 4, the still image data is input to the block buffer 301 for each block, converted into DCT coefficients representing the spatial frequency distribution by the DCT conversion unit 302, and then quantized by the quantization unit 303, and obtained. The quantized coefficient is the encoding unit 304.
Is encoded and output.

[0008] When quantizing DCT coefficients, JPEG recommends applying a quantization step proportional to each component of the quantization matrix shown in FIG. 5 to each spatial frequency component. This quantization matrix is obtained from a value obtained from an experiment that examines the sensitivity of human vision to each spatial frequency component, and as shown in FIG.
It is held in the quantization matrix holding unit 305 in advance and used for calculating the quantization step.

Normally, a value called a scaling factor SF is input to the register 306 before starting the encoding process, and the arithmetic processing unit 307 applies the value from the scaling factor SF and each component of the quantization matrix. The quantization step for the spatial frequency component to be obtained is obtained and used for the quantization in the quantization unit 303 described above.

Here, the above-mentioned scaling factor SF
Is a control coefficient for controlling the quantization step width.In the field of still image coding, the user sets the value of this scaling factor SF in consideration of the characteristics of the image to be coded and the purpose of use. It is often decided and entered. Further, the arithmetic processing unit 307 described above calculates the quantization step by performing, for example, an operation of dividing the product of each component of the quantization matrix and the scaling factor SF by a constant (for example, 50).

The encoding unit 304 arranges the components of the quantized coefficient in an order called zigzag scan,
Further, a configuration is provided in which the effective coefficients having values other than “0” and the continuous lengths of invalid coefficients having the value “0” are converted into a combination, and each of these combinations is encoded using the Huffman code table 307. Has become.

Therefore, when the scaling factor is increased and the value of the quantization step corresponding to each component of the DCT coefficient is increased, the values of many components of the quantized coefficient become "0" and are generated by the encoding unit 304. A high compression rate can be obtained because the amount of code to be generated is significantly reduced.

The coded data thus obtained is restored to the original image by the still image restoration device shown in FIG. In the still image restoration device shown in FIG. 6, the decoding processing unit 311 restores the quantized coefficient from the input code, and the inverse quantization unit 312 uses the quantization step obtained by the quantization step generation unit 313. DCT by inverse quantization
Restore the coefficients. The inverse DCT transform unit 314 restores the image data of each block by performing an inverse DCT transform process on this DCT coefficient, and the restored image holding unit 315.
And the restored image holding unit 315 sequentially stores these image data in the addresses corresponding to the blocks.

As described above, the image data restoration device is configured to restore the image data of one screen by performing the procedure reverse to the encoding process. Here, the quantization step generation unit 313 described above calculates the quantization step corresponding to each component of the quantization coefficient from the quantization matrix and the scaling factor, as in the still image coding apparatus illustrated in FIG. It is composed.

Next, a conventional moving picture coding system will be described. FIG. 7 shows a basic configuration diagram of the moving picture coding apparatus. In FIG. 7, the moving picture coding apparatus is the same as the still picture coding apparatus shown in FIG. The configuration is such that a difference image generation unit 401 that obtains the difference between the image data of the previous frame and the image data of the current frame is added.

In the previous frame reproduction processing unit 410, as shown in FIG. 7, the inverse quantization unit 312 inversely quantizes the quantized coefficient of each block with the quantization step obtained by the quantization step generation unit 313, Inverse DCT the obtained DCT coefficient
The conversion unit 314 performs inverse DCT conversion to reproduce the difference image from the previous frame for each block. Further, the difference image and the image of the corresponding block of the previous frame held in the frame memory 411 are added by the addition unit 412, and the corresponding block of the frame memory 411 is updated by the addition result, so that the previous frame It is configured to reproduce an image.

As described above, in the conventional moving picture coding method, redundant information is obtained by coding the difference image between the image of each frame and the preceding frame image, and further performing processing such as motion compensation. To obtain a high compression rate.

FIG. 8 shows a basic structure of a moving picture restoring apparatus which restores a moving picture from the code obtained as described above. In FIG. 8, the conventional moving image restoration apparatus adds an addition unit 421 to the still image restoration apparatus shown in FIG.
The difference image of each block obtained by the T conversion unit 314 and the image of the previous frame held in the restored image holding unit 315 are added to restore the image of the current frame.

On the other hand, when the JPEG method is applied to a moving image, the moving image is regarded as composed of a series of still images, and each successive frame is coded as an independent still image. It is not necessary to generate an image, and the configuration can be simplified.

However, since a moving image is composed of a huge number of frames, if each frame is simply encoded as a still image as described above, the code amount becomes enormous.

As a technique for suppressing the code amount when compressing a moving image by applying the JPEG system, the applicant of the present invention has filed Japanese Patent Application No. 4-77957 "Image data encoding / decompression method and apparatus thereof". Has already applied.

According to this technique, when each frame is coded, the correlation with the previous frame is checked for each block, only the block determined to have little correlation is coded as a valid block, and block information indicating the position of the valid block is recorded. It is sent with.

FIG. 9 shows an example of the configuration of a moving picture coding apparatus to which this technique is applied. In FIG. 9, the frame memory 2
01 holds image data corresponding to the image of the previous frame as a reference frame, and the block determination unit 202
Compares the image data of each frame with the corresponding block of the reference frame for each block to determine whether there is a correlation. The block discrimination unit 202 discriminates a block determined to have a small correlation as a valid block, and outputs discrimination information indicating whether each block is a valid block,
In accordance with this discrimination information, the encoding processing unit 204 encodes only valid blocks, and the frame memory 201 updates the reference frame with new image data only for valid blocks.

The above-mentioned coding processing unit 204 is configured similarly to the still image coding apparatus shown in FIG. 4, and corresponds when the discrimination information indicating that the block is a valid block is input from the block discrimination unit 202. The block encoding process may be performed.

Further, the block information creating section 203 creates block information indicating the position of the effective block based on the determination for each block for one frame, and the code obtained by the encoding processing section 204 is the code. The data is sequentially stored in the buffer 205.

In this way, when the processing for all blocks of one frame is completed, the multiplexer 206 first sends the block information obtained by the block information creating unit 203, and then the code buffer 205 stores the block information. The sign of is output.

According to this technique, only the part that has changed from the previous frame is coded in block units, and a code corresponding to a part that does not change much like the background part is generated, so the code amount of each frame is increased. Can be significantly compressed.

On the other hand, the coded data obtained by the above-mentioned coding device is restored to the original image by the moving image restoration device shown in FIG. In FIG. 10, the input code data is divided into block information and a code of a valid block by the code division unit 501, and the code of the valid block is input to the restoration processing unit 502, and the corresponding block is processed in the same manner as in the past. Image data of is restored. This restoration processing unit 5
02 may be configured by a decoding processing unit 311, an inverse quantization unit 312, a quantization step generation unit 313, and an inverse DCT conversion unit 314, similarly to the still image restoration device shown in FIG.

Further, the address generator 503, based on the block information divided by the code divider 501,
The addresses of the effective blocks are sequentially calculated and sent to the restored image holding unit 504. Therefore, the restored image holding unit 31
5 sequentially stores the image of the effective block obtained by the decoding processing unit 502 based on the address from the address generation unit 503, so that the image of one frame can be restored.

[0030]

By the way, in a videophone system, a videoconference system, etc., in order to restore a natural moving image on the receiving side, the image data encoding device on the transmitting side has a predetermined number per unit time. It is necessary to send the code data of the frame.

In the field of moving image compression, various techniques have been proposed for keeping the coded data amount constant and making the transmission time of each frame uniform (for example, Japanese Patent Laid-Open No. 62-209).
984, JP-A-63-232691, JP-A-3-291081
(See Japanese Patent Laid-Open No. 4-331592).

In the technique of Japanese Patent Laid-Open No. 62-209984, when the image of the current frame changes significantly from the image of the previous frame, the quantization step is made larger than the standard value, , Is a technique for suppressing the fluctuation of the code amount for each frame by gradually approaching the standard value.

The technique disclosed in Japanese Patent Laid-Open No. 63-232691 is a system for switching between intraframe coding and interframe coding in accordance with the magnitude of change in the image from the previous frame. Then, by switching the quantization step, the code amount of each frame is made uniform.

Further, in the technique disclosed in Japanese Patent Laid-Open No. 3-291081, in order to make the code amount of each frame a constant value, the quantization applied to the frame from the normalization coefficient determined based on the code amount of each frame. In the method of calculating steps, at the time of a scene change where the image content changes significantly from the previous frame, the normalization coefficient is calculated based on the code amount of the corresponding frame and applied to each subsequent frame to perform the quantization processing. Is a technique that enables high-speed processing.

In the technique disclosed in Japanese Patent Laid-Open No. 4-331592, in the coding process of each block, the code amount obtained by the processes up to that point or the result of comparison between the predicted value and the standard code amount is compared. It is a technique for equalizing the code amount of each frame by changing the quantization step applied to that block in accordance with.

These techniques are based on the premise that all blocks of all frames are coded, and that the flow rate of coded data is strictly constant. In addition, the configuration is extremely complicated, such that the quantization step width can be changed within one frame.

By the way, the above-mentioned Japanese Patent Application No. 4-77957.
In the No. technique, only effective blocks are coded, so areas with little change, such as the background, are often not updated at all until a scene change, and the image at the first or scene change is not updated at the next scene change. Held up.

In general, human vision is insensitive to the deterioration of image quality in a region where the change is large, but conversely, it is known that the deterioration of image quality in a region where the change is small such as the background is sensitively sensed. ing. That is, when the user subjectively determines the image quality of a moving image, the determination is greatly influenced by the image quality of the background portion.

Therefore, the above-mentioned Japanese Patent Application No. 4-7795.
If the technique of equalizing the code amount of each frame is simply applied to the technique of No. 7, there is a possibility that the user may get an impression that the overall image quality is significantly deteriorated. Because in this case, a large quantization step is applied to the first frame or the frame of the scene change,
This is because the image quality of the background part and the like is deteriorated.

On the other hand, even if the code amount of each frame is not strictly equalized, if a code for a predetermined number of frames can be transmitted in a unit time, a natural moving image is reproduced on the receiving side. It is possible to

The present invention is an image data coding method for coding only effective blocks included in each frame of a moving image, and an image data code for securing the number of frames to be sent per unit time and improving the image quality of a restored image. An object of the present invention is to provide a method and a device.

[0042]

FIG. 1 is a diagram showing the principle of the image data encoding method of the present invention. The invention of claim 1 divides each of a series of still images into blocks composed of a plurality of pixels, and among blocks included in the image of each frame, an effective block having no correlation with the corresponding block of the previous image. In the image data encoding method of discriminating between the effective frames and selectively encoding and transmitting only the effective block, the frame is a scene change in which the characteristics of the image of the previous frame and the image of the previous frame largely change based on the information of each frame. It is characterized in that the size of the quantization step used in the encoding process of the corresponding frame is switched according to the determination result.

According to a second aspect of the present invention, in the image data encoding method according to the first aspect, it is determined whether or not each frame is a scene change based on the number of effective blocks of each frame, and the frame is a scene change. , A value smaller than the standard quantization step value is applied to.

FIG. 2 is a block diagram showing the principle of the image data coding apparatus of the present invention. According to a third aspect of the invention, each of the series of still images is divided into blocks composed of a plurality of pixels, and for each block of the image of each frame, the corresponding block of the image of the previous frame held in the reference frame holding means 101. Whether or not there is a correlation with is discriminated by the discriminating means 102, and only effective blocks having no correlation are encoded by the encoding means 1.
In the image data coding apparatus, which is coded by 03 and transmitted by the transmitting means 104 together with the code corresponding to the effective block and the block information indicating the position of the effective block, based on the discrimination information obtained by the discrimination means 102, the previous frame And a detection unit 111 that detects a frame in which the characteristics of the image have significantly changed as a scene change, and normally, a predetermined value is output as a quantization step, and a predetermined value is determined according to the detection result indicating that the scene change has been detected. A switching unit 112 that outputs a value smaller than the value as a quantization step is provided, and the encoding unit 103 uses the quantization step output by the switching unit 112 to perform encoding processing of information of each frame. It is characterized by being.

[0045]

According to the first aspect of the invention, the size of the quantization step used when encoding the frame can be controlled depending on whether or not each frame is a scene change.

According to the second aspect of the present invention, the determination process can be simplified by determining whether or not there is a scene change based on the number of effective blocks of each frame.
Also, by applying a quantization step smaller than the standard to a frame that is a scene change, the image quality of the corresponding frame can be improved.

Here, in the method of encoding only the effective block, the image at the scene change is held in the part where the change is scarce, such as the background part. Therefore, the image quality of the frame which is the scene change should be improved. Due to
The image quality of the background portion in the subsequent frames can be improved, and the image quality of the entire moving image can be effectively improved.

If the value of the quantization step applied to another frame is set to a value larger than the standard, these frames can be compressed at a high compression rate and the average value of the transmission time of one frame can be shortened. Therefore, the number of frames transmitted per unit time can be secured.

The invention of claim 3 is applicable when the detecting means 111 detects a frame which is a scene change based on the discrimination information and the switching means 112 switches the quantization step in accordance with the detection result. A smaller than normal quantization step can be applied to the frame. That is, the invention of claim 2 is applied to an image data encoding device that encodes only effective blocks, so that the number of frames transmitted per unit time can be secured and the quality of a restored image can be improved.

[0050]

Embodiments of the present invention will now be described in detail with reference to the drawings. FIG. 3 shows a block diagram of an embodiment of the image data encoding device of claim 3.

In FIG. 3, the image data coding apparatus according to claim 3 has a frame memory 211, a discrimination information holding section 212 and a control coefficient setting section 220 added to the conventional image data coding apparatus shown in FIG. Is configured.

The frame memory 211 has a capacity corresponding to one frame of image data, and the image data of the corresponding block of the previous frame is sequentially encoded according to the input of each block of the new frame. It is configured to be sent to 204.

Further, the discrimination information holding unit 212 outputs the discrimination information of the corresponding block of the previous frame in response to the input of the discrimination information of each block of the new frame, and sends it to the block information generation unit 203. Has become. For example, the discrimination information holding unit 212 may be configured using a FIFO having a bit length corresponding to all blocks in one frame.

Further, the frame memory 201 corresponds to the reference frame holding means 101, and in the same manner as in the conventional case, the image data of the block inputted according to the output of the block discrimination section 202 corresponding to the discrimination means 102 Is used to update the image of the reference frame. On the other hand, an encoding processing unit 204 corresponding to the encoding unit 103
Is configured to encode only valid blocks according to the discrimination information output from the discrimination information holding unit 212.

That is, the frame memory 211 and the discrimination information holding section 212 delay the image data and the discrimination information by one frame, and the control coefficient setting section 220 obtains them in parallel with the discrimination processing of the effective block of the next frame. The encoding processing unit 204 is configured to perform encoding processing of each frame using the obtained scaling factor SF.

The block information generator 203, the code buffer 205, and the multiplexer 206 are the sending means 1
The block information creation unit 203 corresponds to the block information creation unit 203.
However, the block information representing the position of the effective block is created based on the output of the discrimination information holding unit 212, and the code buffer 205 holds the code data by the coding processing unit 204 for one frame. . When the encoding process for one frame is completed, the multiplexer 206 first sends the block information obtained by the block information creating unit 203, and then sends the code data in the code buffer 205 to the transmission path. It is configured to do.

Further, in FIG. 3, the control coefficient setting unit 22
When 0, the counter 221 counts the number of blocks validated by the block determination unit 202 for each frame,
The comparator 222 is configured to compare the counting result Nc of the counter 221 with a predetermined threshold value Nth and control the selector 223 according to the comparison result.

Here, the counter 221 adds "1" to the count value in response to the input of the determination result indicating that the block is a valid block, and in response to the frame end signal indicating that the input of the image data for one frame is completed. The count result Nc is output as a result and the count value is reset to the initial value "0". Further, as the above-described threshold value Nth, for example, a value corresponding to 95% of the number of blocks in one frame is set, and the comparator 222 sets the corresponding frame when the count result Nc of the counter 221 exceeds the threshold value Nth. May be configured to output a determination result indicating that is a scene change.

As a result, the counter 221 and the comparator 22 are
2 can realize the function of the detection unit 111. Further, the selector 223 normally selects the scaling factor SFl, selects the scaling factor SFh in accordance with the input of the determination result indicating that the scene is changed, and inputs it to the register 306 provided in the encoding processing unit 204. It may be configured to. Thereby, the switching means 11
It is possible to realize the same function as that of 2 and switch the size of the quantization step between the scene change frame and another frame. Further, a value larger than a scaling factor that gives a standard image quality may be set as the scaling factor SFl, and a value smaller than the standard scaling factor may be set as the scaling factor SFh.

In this case, the DCT coefficient of each block included in the scene change frame is quantized by the quantization step calculated using the scaling factor SFh smaller than the standard value, and includes detailed information. A code is generated. On the other hand, for frames other than scene changes, the scaling factor SFl larger than the standard value is used.
Is applied, and the information of these frames is compressed at a high compression rate.

In this way, by compressing the information of the frames other than the scene change at a high compression rate, it is possible to suppress the average value of the transmission time of the code per frame as a whole, and the predetermined value is set within the unit time. It is possible to transmit codes for several frames.

Further, in the method of encoding only the effective block, the image of a portion having a small change such as the background portion is not updated until the next scene change. Therefore, if the restored image of high image quality is reproduced based on the code corresponding to the frame of the scene change, the image of high image quality is maintained until the next scene change even in the subsequent frames in the part where the change is scarce. On the other hand, in the part where the change is drastic, some deterioration of the image quality is not sensed. Therefore, by improving the image quality of the scene change frame as described above, the image quality of the entire moving image can be effectively improved. it can.

As described above, by transmitting the code including the detailed information only at the scene change, it is possible to effectively improve the image quality of the entire moving image while securing the number of frames to be transmitted in a unit time. It is possible to restore a high quality and natural moving image on the receiving side.

If the output of the selector 223 of the control coefficient setting unit 220 is input to the block information creating unit 203 and the value of the selected scaling factor is sent together with the block information to the receiving side, the image data of the receiving side is obtained. The restoration device can be notified of the appropriate scaling factor.

In response to this, the image data restoration device may separate the block factor from the code data together with the scaling factor, determine the quantization step based on the obtained scaling factor, and perform the inverse quantization process. .

[0066]

As described above, according to the present invention, the image quality of the restored image at the scene change is improved by applying the quantization step smaller than the quantization step applied to the other frame to the frame which is the scene change. By doing
It is possible to suppress the average value of the transmission time of the code data of one frame to secure the number of transmission frames and to effectively improve the image quality of the moving image.

[Brief description of drawings]

FIG. 1 is a diagram showing the principle of an image data encoding method of the present invention.

FIG. 2 is a block diagram showing the principle of the image data encoding device of the present invention.

FIG. 3 is a configuration diagram of an embodiment of an image data encoding device of the present invention.

FIG. 4 is a basic configuration diagram of a conventional still image encoding device.

FIG. 5 is a diagram showing a quantization matrix.

FIG. 6 is a basic configuration diagram of a conventional still image restoration device.

FIG. 7 is a basic configuration diagram of a conventional moving image encoding device.

FIG. 8 is a basic configuration diagram of a conventional moving image restoration device.

[Fig. 9] Fig. 9 is a diagram illustrating a configuration example of a moving image encoding device to which the JPEG method is applied.

FIG. 10 is a diagram showing a configuration example of a moving image restoration device to which the JPEG method is applied.

[Explanation of symbols]

 101 Reference Frame Holding Means 102 Discriminating Means 103 Encoding Means 104 Sending Means 111 Detecting Means 112 Switching Means 201, 211, 411 Frame Memory 202 Block Discriminating Unit 203 Block Information Creating Unit 204 Encoding Processing Unit 205 Code Buffer 206 Multiplexer 212 Discriminating Information Holding unit 220 Control coefficient setting unit 221 Counter 222 Comparator 223 Selector 301 Block buffer 302 DCT conversion unit 303 Quantization unit 304 Code generation unit 305 Quantization matrix holding unit 306 Register 307 Operation processing unit 308 Huffman code table 311 Decoding processing unit 312 Inverse quantization unit 313 Quantization step generation unit 314 Inverse DCT conversion unit 315 Restoration image holding unit 401 Difference image generation unit 410 Previous frame reproduction processing unit 412, 421 Part 501 code division unit 502 restoring unit 503 address generator

Claims (3)

[Claims] 1. A series of still images is divided into blocks each composed of a plurality of pixels, and an effective block having no correlation with a corresponding block of a preceding image is determined from blocks included in the image of each frame. However, in the image data encoding method for selectively encoding and transmitting only the effective block, whether the frame is a scene change in which the characteristics of the image significantly change from those of the previous frame based on the information of each frame. An image data encoding method, characterized by determining whether or not, and switching a size of a quantization step used in encoding processing of a corresponding frame according to the determination result. 2. The image data encoding method according to claim 1, wherein it is determined whether or not each frame is a scene change based on the number of effective blocks of each frame, and the frame which is a scene change is standardized. An image data encoding method characterized by applying a value smaller than the value of the quantization step. 3. A series of still images are each divided into blocks composed of a plurality of pixels, and for each block of the image of each frame, a corresponding block of the image of the previous frame held in the reference frame holding means (101). Whether or not there is a correlation with is discriminated by the discriminating means (102), and only the effective block having no correlation is encoded by the encoding means (103),
In an image data encoding device for transmitting by a sending means (104) together with a code corresponding to the valid block and block information indicating a position of the valid block, based on the discrimination information obtained by the discrimination means (102), Detection means (111) for detecting a frame in which the characteristics of the frame and the image have significantly changed as a scene change
And a switching means (112) for outputting a predetermined value as the quantization step, and outputting a value smaller than the predetermined value as the quantization step in accordance with the detection result indicating that the scene change has been detected. And the encoding means (103) includes the switching means (112).
An image data encoding device, which is configured to perform encoding processing of information of each frame by using a quantization step output by.