JPH07107475A - Method for encoding picture - Google Patents

Abstract

PURPOSE:To perform encoding without deteriorating the compression efficiency by discriminating the block activity (BA) on the prescribed block of the hierarchy data and setting the threshold value being the discrimination reference of the division processing for the low-order hierarchy data based on the degree distribution for the BA. CONSTITUTION:A generation information amount control part 40B of a picture encoder decides a threshold value TH adapted for the data to be processed by inputting input picture data. In a hierarchy encoding encoder part 40A, the optimal control value which is decided so that the input picture data are efficiently encoded is sent to the encoder. Thus, the time delay to be generated by the accurate generation information amount control and the feedforward type buffering is eliminated. When the picture data are divided into plural hierarchy data with dissimilar resolution reflexively in succession and is encoded, the BA is discriminated on the prescribed block of the hierarchy data excluding the highest order hierarchy data with the lowest resolution. Then, the threshold value is set from the distribution of the block degree corresponding to the BA and the transmission amount is increased and decreased to prevent the deterioration of the compression efficiency.

Description

Detailed Description of the Invention

[0001]

[Table of Contents] The present invention will be described in the following order. Field of Industrial Application Conventional Technology (FIGS. 17 and 18) Problem to be Solved by the Invention (FIGS. 17 and 18) Means for Solving the Problem (FIGS. 1 to 10) Action (FIGS. 1 to 10) Example (1) Principle of hierarchical coding (FIGS. 1 to 3) (2) Overall configuration of image coding apparatus (FIG. 4) (3) Hierarchical coding encoder unit (FIGS. 5 and 6) (4) Generated Information Amount Control Unit (FIGS. 7 to 16) (5) Other Embodiments Effects of the Invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding method,
For example, it is suitable to be applied to an image encoding device that divides and encodes predetermined image data into a plurality of image data having different resolutions.

[0003]

2. Description of the Related Art Conventionally, as this type of image coding apparatus,
There is one in which input image data is hierarchically encoded using a hierarchical encoding method such as pyramid encoding (Japanese Patent Application No. 5-1
No. 42836). In this hierarchical encoding device, high-resolution input image data is used as first hierarchical data, second resolution data having a lower resolution than the first hierarchical data, and further higher resolution than the second resolution data. , The third hierarchical data, which is low, are sequentially and recursively formed, and the plurality of hierarchical data are transmitted through one communication path or a transmission path including a recording / reproducing path.

In addition, at this time, in the image decoding apparatus for decoding a plurality of hierarchical data, in addition to decoding all of the plurality of hierarchical data, one of the hierarchies is selected depending on the resolution of the television monitor corresponding to each. The desired one of the data may be selected and decoded. By decoding only the desired hierarchical data from the plurality of hierarchical data thus hierarchized, desired image data can be obtained with the minimum required transmission data amount.

Here, as shown in FIG. 17, in an image coding apparatus that realizes, for example, four-layer coding as this hierarchical coding, thinning filters 2, 3, 4 for three stages and interpolation filters 5, 6 are respectively provided. , 7 and the input image data D1
With respect to each of the stages, the thinned-out filters 2, 3, and 4 form reduced image data D2, D3, and D4 having a sequentially low resolution, and the interpolation filters 5, 6, and 7 reduce the reduced image data D2, D3, and D4 before the reduction. Return to resolution.

Outputs D2 to D4 of the thinning filters 2 to 4
And the outputs D5 to D7 of the respective interpolation filters 5 to 7 are input to the difference circuits 8, 9 and 10, respectively, whereby difference data D8, D9 and D10 are generated. As a result, in the image encoding device 1, the amount of hierarchical data is reduced and the signal power is reduced. Here, this difference data D
The areas 8 to D10 and the reduced image data D4 are 1, 1/4, 1/16, and 1/64 in size with respect to the input image data D1, respectively.

The difference data D8 to D10 obtained from the respective difference circuits 8 to 10 and the reduced image data D4 obtained from the thinning filter 4 are encoded by the encoders 11, 12, 13, respectively.
Compression processing by 14 and as a result, each encoder 11,
First, second, and third with different resolutions from 12, 13, and 14
And fourth layer data D11, D12, D13 and D1
4 are transmitted to the transmission line in a predetermined order.

The first to fourth hierarchical data D11 to D14 thus transmitted are decoded by the image decoding apparatus shown in FIG. That is, the first to fourth hierarchical data D11 to D14 are respectively decrypted by the decoders 21, 22, 2 and 2.
3, 24, which results in the decoder 24
Outputs the fourth hierarchical data D24.

The output of the decoder 23 is the fourth hierarchical data D obtained from the interpolation filter 26 in the adder circuit 29.
It is added with the interpolation data of 24, whereby the third hierarchical data D23 is restored. Similarly, the output of the decoder 22 is added to the interpolation data of the third hierarchical data D23 obtained from the interpolation filter 27 in the adder circuit 30, whereby the second hierarchical data D22 is restored. Further, the output of the decoder 21 is output by the adder circuit 31 to the interpolation filter 2
8 is added to the interpolated data of the second hierarchical data D22 to restore the first hierarchical data D21.

[0010]

However, in the image coding apparatus which realizes such a hierarchical coding method, since the input image data is divided into a plurality of hierarchical data to be coded, the data amount of only the hierarchical component is inevitable. However, there is a problem that the compression rate is reduced as compared with the high-efficiency coding method that does not use hierarchical coding. Further, when trying to improve the compression efficiency, there is a problem that the image quality is deteriorated by the quantizer applied between the respective hierarchical data.

The present invention has been made in consideration of the above points, and provides an image coding method and an image coding apparatus capable of improving compression efficiency and reducing image quality degradation when hierarchically coding image data. It is a proposal.

[0012]

In order to solve such a problem, according to the present invention, image data is sequentially recursively divided into a plurality of hierarchical data D31 to D35 having a plurality of different resolutions, and is encoded. When significant activity is detected in lower layer data, the division process returns to the upper layer where the division process was interrupted, the division determination flag is reset, and the division determination process is performed again for the lower layer. When the generated information amount is controlled to the target value by selecting the division processing of the block Xj (i) spatially corresponding to, the block activity ACT defined between layers is detected over all the layers, and the detection result is detected. The optimum control value is detected using the block frequency distribution table.

Further, in the present invention, when the amount of generated information is reduced by selecting the division process of the block Xj (i), when the generated information amount is controlled to a target value, the block activity ACT defined between layers is used. Is detected across all layers,
With respect to the block frequency distribution table generated based on the detection result, the optimum control value is detected in each layer based on the history of block activity in the upper layer.

Further, in the present invention, the data D31 input to the encoder 40A for hierarchical coding is controlled in a standby state until the optimum control value is determined, and the input data D31 is hierarchically coded when the optimum control value is determined. To do so.

Further, in the present invention, the control threshold value TH is set independently for each layer in the optimum control value detection process using the frequency distribution table constituted by block activity ACTs of a plurality of layers.

Further, according to the present invention, when the control threshold value TH is set independently for each layer in the optimum control value detection process using the frequency distribution table composed of block activities of a plurality of layers, each layer is set. A combination of the control threshold values of 1 is prepared in advance, and the optimum control value is detected from the combination.

Further, according to the present invention, in the optimum control value detection process using the frequency distribution table constituted by the block activity ACTs of a plurality of layers, after the data is registered in the frequency distribution table, the integration type is calculated by the value of the block activity ACT. Create a frequency distribution table to shorten the optimum control value detection time.

Further, in the present invention, when a block is registered in the frequency distribution table, a limit LMT is set to the block activity value ACT in the frequency distribution table, and a block having a block activity equal to or larger than the limit value is registered in a predetermined coordinate. And reduce the amount of memory required.

Further, in the present invention, the image data is recursively divided into a plurality of hierarchical data having a plurality of different resolutions and encoded, and when significant activity is detected in the lower hierarchical data having a high resolution, the image data is divided. In the image coding method in which the division determination flag is reset by returning to the upper layer where the processing is interrupted and then the division determination process is performed toward the lower layer again, it occurs due to the selection of the block division process of spatially corresponding blocks between adjacent layers. When controlling the amount of information to the target value, the block activity defined between each layer is detected over all layers, and the optimal control value is detected using the block frequency distribution table based on the detection result, and the detection result is obtained. At this time, the image data waiting in the predetermined memory M1 is encoded.

[0020]

When the image data D31 is sequentially and recursively divided into a plurality of hierarchical data D31 to D35 having a plurality of different resolutions and encoded, a predetermined block of hierarchical data except the highest hierarchical data having the lowest resolution. For the block activity P, the threshold TH, which is the criterion for the division processing for the lower lower layer data, is set based on the distribution of the block frequencies corresponding to the block activity P. Hierarchical coding can be realized.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail below with reference to the drawings.

(1) Principle of Hierarchical Coding FIG. 1 shows, as a whole, the principle of hierarchical coding according to the present invention in which a still image such as a high-definition television signal is hierarchically coded and compressed. In this hierarchical encoding, the upper layer data is created by a simple arithmetic average of the lower layer data, thereby realizing a hierarchical structure without an increase in the amount of information.
For decoding from the upper layer to the lower layer, the amount of information in the flat portion is reduced by adaptively controlling the division based on the activity of each block. Further, in the encoding of the differential signal performed for the lower layer, the efficiency is improved by switching the quantization characteristic for each block without additional code based on the activity of the upper layer.

That is, in the hierarchical structure of this hierarchical encoding,
First, the high-definition television signal to be input is defined as the lower layer, and the four pixels X1 to X4 in the small block of 2 lines × 2 pixels in this lower layer are expressed by the following equations.

[Equation 1] The arithmetic mean represented by is taken, and its value m is taken as the value of the upper hierarchy. In this lower hierarchy,

[Equation 2] As shown in, the difference value from the upper layer is prepared for three pixels, so that the hierarchical structure is configured with the same information amount as the original four pixel data.

On the other hand, when decoding the lower layer, 3 pixels X1
~ X3 is the following formula

[Equation 3] As shown by, each difference value Δ is added to the average value m of the upper hierarchy.
Xi is added to obtain the decoded value E [Xi], and the remaining one pixel is

[Equation 4] As shown by, the decoded value E [X4] is determined by subtracting the three decoded values of the lower layer from the average value m of the upper layer. Here, E [•] means a decoded value.

Here, in this hierarchical encoding, the resolution is quadrupled from upper layer to lower layer, but in the flat part, this division is prohibited to reduce redundancy. A flag for instructing the presence / absence of this division is prepared for each bit and block. The necessity of division in the lower layer is determined as local activity, for example, the maximum value of the difference data.

Here, as an example of hierarchical coding, HD of ITE is used.
FIG. 2 shows the adaptive division result when the standard image (Y signal) is used and five-layer coding is performed. The ratio of the number of pixels in each layer to the original number of pixels when the threshold value for the maximum difference data is changed is shown, and it can be seen that redundancy is reduced based on the spatial correlation. The reduction efficiency changes depending on the image, but if the threshold value for the maximum difference data is changed from 1 to 6, the average reduction rate becomes 28 to 69 [%].

In practice, the resolution of the upper layer is quadrupled to form the lower layer, and at that time, the difference data from the upper layer data is encoded in the lower layer, whereby the signal level width can be effectively reduced. FIG. 3 shows the case of five layers by the layer coding described above with reference to FIG. 2, but here, the layers are named from the lower layers to the first to fifth layers.

Compared to the 8-bit PCM data of the original image,
A reduction in signal level width can be seen. First with a large number of pixels
Since the ~ 4 layers are differential signals, a significant reduction can be achieved and efficiency is improved by subsequent quantization. As can be seen from the table of FIG. 3, the reduction efficiency has little dependence on the pattern, and is effective for all pictures.

By forming the upper layer with the average value of the lower layer, the error propagation is stopped in the block, and the lower layer is converted into the difference from the average value of the upper layer, so that the efficiency is also provided. You can In actual upper layer coding, there is a correlation in the activities between layers at the same spatial position, and by determining the quantization characteristic of the lower layer from the quantization result of the upper layer, adaptive quantization that does not require additional code Can be realized.

In practice, an image is hierarchically encoded based on the above-mentioned five-stage hierarchical structure to be expressed in multi-resolution, and adaptive division and adaptive quantization using the hierarchical structure are performed to obtain various HD standard images (8 Y / PB / PR of bit is about 1 /
It can be compressed to 8. The additional code for each block prepared for adaptive division is run-length coded in each layer to improve compression efficiency. In this way, an image with sufficient image quality can be obtained in each layer, and a good image without visual deterioration can be obtained in the final lowest layer.

(2) Overall Structure of Image Coding Device In FIG. 4, reference numeral 40 denotes an image coding device according to the present invention, which is a hierarchical coding encoder unit 40A for hierarchically coding and outputting input image data D1 and a hierarchical coding. Encoder part 4
The generated information amount control unit 40B controls the generated information amount at 0A so as to reach the target value.

The hierarchical coding encoder section 40A is composed of a memory (not shown) for data delay and an encoder. Of these, the memory is the generated information amount control unit 40B.
The input stage is provided so that the data can be delayed so that the encoding process is not executed until the optimum control value is determined in.

On the other hand, the generated information amount control unit 40B is adapted to input the input image data and determine the threshold value TH which is suitable for the data to be processed, and the input image data is efficiently encoded in the hierarchical encoding encoder unit 40A. The optimum control value determined to be converted is transmitted to the encoder. This is a so-called feedforward type buffering structure. With this configuration, it is possible to accurately control the amount of generated information and to eliminate the time delay caused by the feedback control type feedback.

Here, the block activity defined based on the difference value between layers is used to select the division process in the lower layer. That is, upper layer data is composed of 2 × 2 4 pixels in the lower layer, and a block is defined.

That is, the upper layer data is X0 (i + 1)
And the lower layer data is Xj (i), the inter-layer difference code value ΔXj (i) = X0 (i + 1) −Xj (i). However, j = 0 to 3. If the block activity determination function is G (·), the block activity ACT = G (ΔXj (i)).

Further, the hierarchy determination flag FLG (0: stop division, 1: continue division) is set. Here, a method of reviewing the division determination flag will be described. Regarding each layer determination flag determination process (generation information amount control process), first, the activity ACT of all layers is generated for each block, and subsequently, the determination flag FLG is generated by the threshold value corresponding to the activity ACT of all layers. . Further, for each block, a hierarchy in which the determination flag FLG becomes 1 for the first time is searched from the lower hierarchy.

The hierarchy determination flags FL of all upper hierarchy blocks above the hierarchy in which the determination flag FLG becomes 1 for the first time.
Let G be 1. The determination flag FLG is updated according to this rule. Further, the above-mentioned determination threshold value is changed based on the generated information amount control, and the optimum threshold value that falls within the target value is selected.

Next, regarding the block division selection process (actual encoding process), when the judgment flag FLG = 0, the division of the lower hierarchy is stopped, and the judgment flag FLG = 1.
In the case of, the lower layer is divided. The above-mentioned processing is composed of two steps in which the layer determination flag FLG is determined for each block of each layer in advance by controlling the amount of generated information, and the actual block division processing is executed based on that.

(3) Hierarchical coding encoder section The hierarchical coding encoder section 40A has the structure shown in FIG. 5, and in this example, processing is performed by dividing it into five layers.

First, the input image data D31 is input to the first difference circuit 41 and the first averaging circuit 42. The first averaging circuit 42 generates the second layer data D32 by averaging four pixels of the input image data D31 (that is, the first layer data (the lowest layer data)). In the case of this embodiment, the first averaging circuit 42, as shown in FIGS. 6D and 6E, has four pixels X1 (1) to X1 of the input image data D31.
The pixel X1 (2) of the second hierarchical data D2 is generated from 4 (1).
Further, the pixels X2 (2) to X4 (2) adjacent to the pixel X1 (2) of the second layer data D32 are similarly generated by obtaining the four pixel average of the first layer data D31.

The second layer data D32 is the second difference circuit 4
3 and the second averaging circuit 44, and the second averaging circuit 44 generates the third hierarchical data D33 by averaging four pixels of the second hierarchical data D32. For example, FIG. 6 (C)
And pixels X1 (2) of the second hierarchical data D32 shown in (D)
The pixel X1 (3) of the third layer data D33 is generated from X4 (2) and the pixels X2 (3) to X4 (3) adjacent to the pixel X1 (3) are generated.
Is similarly generated by averaging four pixels of the second layer data D32.

The third layer data D33 is the third difference circuit 4
5 and the third averaging circuit 46, and the third averaging circuit 46 uses the four-pixel average of the third hierarchical data D33 as shown in FIGS. 6B and 6C, as in the case described above. ,
The fourth layer data D34 including the pixels X1 (4) to X4 (4) is generated.

The fourth layer data D44 is the fourth difference circuit 4
7 and the fourth averaging circuit 48, and the fourth averaging circuit 48 generates the fifth layer data D35 which is the highest layer by averaging four pixels of the fourth layer data D34. That is, as shown in FIGS. 6A and 6B, by averaging the four pixels X1 (4) to X4 (4) of the fourth layer data D34, the pixel X1 (5) of the fifth layer data D35 becomes Is generated.

Here, the first to fifth hierarchical data D31 to D3
As for the block size of 5, when the block size of the first layer data D31 which is the lowest layer is 1 line × 1 pixel, the second layer data D32 is 1/2 line × 1/2 pixel, and the third layer data D33 is 1/2 line × 1/4 pixel, the fourth layer data D34 is 1/8 line × 1/8 pixel, and the fifth layer data D35 which is the highest layer data is 1
/ 16 line × 1/16 pixel.

The hierarchical coding encoder section 40A repeats the recursive processing in order from the highest hierarchical data (that is, the fifth hierarchical data D35) among the first to fifth hierarchical data D31 to D35 and adjoins two. Differences between one hierarchical data are obtained in the difference circuits 41, 43, 45, 47, and only the difference data is compression-encoded by the encoders 51-55. As a result, the hierarchical encoding encoder unit 40A compresses the amount of information transmitted on the transmission path.

In order to keep such a compression condition optimum, the layered coding encoder section 41 decodes the transmission data D51 to D55 obtained for each layer by the decoders 56 to 59.

The decoder 5 corresponding to the highest layer among these
Reference numeral 9 denotes the decoded data D48 corresponding to the fifth layer data D35 compression-coded by the encoder 55, and the transmission data D48.
It is decoded from 55 and is applied to the fourth level difference circuit 47.

On the other hand, the other decoders 51 to 54 switch the decoding operation based on the flags indicating the presence / absence of the division / non-division processing. That is, when the division processing is performed, the upper hierarchical data (that is, the fourth, third, and second hierarchical data) is decoded by the decoding processing from the difference data transmitted as the transmission data D52 to D54. Third
The difference circuit 45 of the hierarchy, the difference circuit 43 of the second hierarchy, and the first hierarchy data 41 are given respectively.
As a result, the difference data D41, D42, D between the adjacent layers is output from the difference circuits 41, 43, 45, 47.
43 and D44 are obtained.

The encoders 51 to 55 corresponding to the respective layers are the difference circuits 41, 43, 45 and 47 and the averaging circuit 4.
Difference data D41, D42, D4 obtained by
3, D44 or fifth layer data D35 is input, and threshold value determination and division selection processing for the activity obtained for each block are executed. At this time, encoder 5
1 to 55, when the processing target is a division block, the difference data obtained between layers is compression-coded as it is, and at the same time, a division determination flag for each block is added and transmitted.

On the other hand, when the processing target is a non-divided block, the encoders 51 to 55 exclude this block from the encoding target as being composited from the upper layer data on the receiving side. Incidentally, also in this case, the division determination flag for each block is attached and transmitted. The first to fifth hierarchical compression coded data output from the five sets of encoders 51 to 55 are sent to a predetermined transmission path.

Next, specific signal processing by the hierarchical coding encoder section 40A will be described. First, consider the case where the processing for the difference value between layers is selected by the block activity based on the difference value between layers. Each block is composed of 2 lines × 2 pixels.

Here, the data value of each pixel is X, and the hierarchy of the data value X is represented by a suffix. That is, when the upper layer data is X (0) i + 1, the adjacent lower layer data is Xj (i) (j = 0 to 3). The differential code value between layers is ΔXj (i) (j = 0 to 3), and the hierarchical coding encoder unit 40A compresses and codes this differential code value.

In the compression coding process by the encoders 51 to 55 in each layer, the block activity P obtained for each block is compared with the threshold data D57, and the process is selected according to the comparison result. That is, when the block activity P is equal to or higher than the threshold value TH, the lower layer is sequentially divided, whereas when the block activity P is less than the threshold value TH, the lower layer is divided.

As a result, with respect to the area having a low block activity P, only the upper layer data need be sent.
The amount of transmitted information can be reduced. Further, the image data decoding device which receives these data across the transmission path restores the lower layer data as the upper layer data in the region with low block activity by using the upper layer data among the transmission data sequentially transmitted. On the other hand, in a region with high block activity, the data is restored by adding the inter-layer difference decoded value and the upper layer data.

A 1-bit judgment flag is introduced for the judgment result of this division or non-division. It is possible to instruct the determination result for each block by this flag. This judgment flag is 1 for each block in each layer.
Bit by bit is required, but it is effective when the image quality is taken into consideration.

(4) Generated Information Amount Control Unit Next, FIG. 7 shows an example of a block diagram showing the configuration of the generated information amount control unit. First, the same averaging circuit 42 performs 1/4 averaging processing on the same input image data D31 as the encoder section of FIG. 5, and second layer data D32 is generated. In the averaging circuit 44, the second layer data D32 is 1 /
The four averaging process is executed to generate the third tier data D33.

Similarly, the fourth tier data D34 is obtained by the quarter averaging process of the averaging circuit 46 for the third tier data D33.
Is generated. Finally, the averaging circuit 48 performs the 1/4 averaging process to generate the fifth layer data D35.

The frequency of the data in the fifth layer data D35 is registered in the frequency distribution table 73. This measures the frequency of data corresponding to the compression processing executed by the encoder unit described above. For example, for the fifth layer data D35,
When compression processing such as ADRC is performed, the dynamic range (DR) of the ADRC block is registered as data.

Next, difference data D64 is generated from the fourth hierarchy data D34 and the fifth hierarchy data D35. With respect to the difference data D64, the block activity detection described above is performed by the block activity detection circuit 68.
The block activity D68 detected there is registered in the frequency distribution table 72.

Block division determination in the upper layer is determined with reference to the block activity determination results of all lower layers. Therefore, the frequency distribution table 72 is defined by the four variables of the first hierarchy block activity D65, the second hierarchy block activity D66, the third hierarchy block activity D67, and the fourth hierarchy block activity D68.

Third layer data D32 and fourth layer data D
The difference data D63 is generated from 34. Difference data D
With respect to 63, the block activity is detected by the activity detection circuit 67. The detected activity D67 is registered in the frequency distribution table 71.

Also in this case, the block division is determined with reference to the determination results of the block activities of all the lower layers below the third layer. Therefore, the frequency distribution table 71 of the third layer is based on the block activity D65 of the first layer and the second block activity D65.
It is defined by the three variables of the hierarchical block activity D66 and the third hierarchical block activity D67.

Second layer data D32 and third layer data D
The difference data D62 is generated from 32, and the block activity D66 is output in the activity detection circuit 66. Block activity detected D66
Is registered in the frequency distribution table 70.

In this case, the determination is made with reference to the block activity determination result of the first layer. The frequency distribution table 70 of the second layer is the block activity D65 of the first layer.
And the second layer block activity D66.

Finally, the difference data D61 is generated from the first layer data D31 and the second layer data D32, and the activity detecting circuit 65 outputs the block activity D65. The detected block activity D65 is registered in the frequency distribution table 69.

With respect to the first layer, it is not necessary to monitor the block activity of the upper layer because the threshold determination of the block activity is independently performed and the result is executed. That is, the frequency distribution table 1 of the first layer is
This is a one-dimensional frequency distribution table composed of hierarchical block activity D65.

The frequency distribution tables 69 to 73 thus generated.
The generated information amount control is executed by using. Each frequency distribution table and the control unit at the subsequent stage are connected by bidirectional signal paths D69 to D73. In the control unit, first, the threshold value for each frequency distribution table is transmitted to each frequency distribution table.

In each frequency distribution table, the amount of generated information corresponding to the threshold value is detected. The generated information amount in each frequency distribution table is controlled by the control unit 74 through the signal paths D69 to D73.
Be transmitted to. The control unit 74 integrates the generated information amounts in the received frequency distribution tables and calculates the total generated information amount to be controlled.

The total generated information amount is compared with the target value, and the threshold value is changed so as to satisfy the target value according to the comparison result. Again, the updated threshold value is set by the controller 74 to the signal path D
69 to D73 to be transmitted to each frequency distribution table. The corresponding amount of generated information is transmitted to the control unit again.

By repeating the above processing, the control result D57 that finally achieves the target value is determined. The determined generated information amount control value D57 is transmitted to the hierarchical coding encoder unit, as shown in the block diagram of FIG.

During the processing period of the information amount control unit, the data to be controlled is kept on standby by the memory M1 included in the encoder unit. In the above-mentioned information amount control, the threshold value suitable for the target data is determined, so that efficient encoding can be realized.

Here, FIGS. 8A to 8E show block activity frequency distribution tables obtained for the highest hierarchical data to the lowest hierarchical data, respectively. Here, regarding the frequency distribution table for the fifth layer shown in FIG. 8A, since the target data is not difference data, a frequency distribution table based on the dynamic range is generated. For example PCM
When encoding is applied, the dynamic range for the encoded block will be registered.

Next, an example of the frequency distribution table for controlling the amount of generated information is shown in FIGS. First,
A definitional expression for calculating the total amount of generated information is derived. In order to calculate the amount of generated information, it is necessary to measure the number of effective blocks that are equal to or greater than the division determination threshold in each layer, but in the generated information amount control of layer coding by the determination flag review method, in the upper layer for each block. The division determination of the target block must be performed after considering the division determination results of all lower layers.

At this time, the block activity determination thresholds used for the division determination in each layer are the first layer division determination threshold value TH1, the second layer division determination threshold value TH2,
The third layer division determination threshold is TH3, and the fourth layer division determination threshold is TH4.

Blocks lower than the above threshold are divided into lower layers in all layers. In the upper layer, if there is no block activity equal to or higher than the threshold in all the lower layers, and the block activity of the layer is less than the threshold, block division is stopped. First, in FIG. 9A, a frequency distribution table of the fifth highest layer, which is the highest layer, is shown.

Regarding the frequency distribution table of the fifth layer, since the target data is not difference data, the information amount control corresponding to the encoding process is performed. When applying fixed length coding such as linear quantization, it is not necessary to create a frequency distribution table. The following generated information amount calculation formula can be used for the generated information amount control.

That is, if the sum of the number of pixels in the block to be divided in the first layer is M1, the number of quantization bits of the first layer data is Q1, and the number of determination flag bits of the first layer is N1, the amount of generated information in the first layer is I1 is the following formula

[Equation 5] Can be given by

The number of bits in the first term in the equation (5) is multiplied by 4 because each block is divided into 2 lines × 2 pixels in this example. In addition, what is multiplied by 3/4 in the first term is that the upper layer value is generated from the average value of the lower layer values,
This is because it reflects the property that the fourth non-transmitted pixel value of the lower layer can be restored by an arithmetic expression using the upper layer value and the transmitted lower layer value of 3 pixels.

Incidentally, in the second term, the fact that N1 is added to the number of blocks in the first layer indicates that one bit is added to each block as a division determination flag and transmitted.

Similarly, for the second, third, and fourth layers, the sum of the numbers of pixels in the block to be divided in each layer is M.
2, M3, M4, and the number of quantization bits in each layer is Q
2, Q3, Q4, the number of judgment flag bits in each layer is N
2, N3, N4, the generated information amount I in each layer
k (k = 2, 3, 4) is

[Equation 6] Can be given by

When the generated information amounts I1 to I4 for the first to fourth layers and the generated information amount I5 for the fifth layer are used, the total generated information amount generated by the encoding process of the layer encoding encoder unit 40A. I is the following formula

[Equation 7] As described above, it can be obtained as the sum of the generated information amount for each resolution.

Here, the bit number of the determination flag is added to the generated information amount of each layer, and the information amount of this flag is equal to the number of blocks for which the division processing is executed in the upper layer. The spatial position of each block can be specified in each layer by the history of the determination flag from the upper layer.

Here, each frequency distribution table will be described. As described above, the frequency distribution table of the highest hierarchy data depends on the compression method, and therefore cannot be uniquely determined. But,
The amount of generated information can be controlled by using a means such as a frequency distribution table.

Next, regarding the first layer data, the generated information amount for the threshold value TH1 can be easily calculated by using the first layer frequency distribution table of FIG. 9 in which the block frequencies for the block activity ACT1 are registered. be able to. Since blocks with a threshold value TH1 or more are to be divided, by obtaining the sum of the block numbers with a threshold value or more,
The amount of generated information in the first layer is calculated.

Next, an example of the frequency distribution table of the second layer is shown in FIG.
Shown in. In the determination flag review method, the number of blocks having a threshold TH2 or more in the second layer is measured with respect to the block that has received the block division execution stop determination in the first layer.

Therefore, the first hierarchy block activity ACT1 and the second hierarchy block activity ACT2.
The frequency distribution table defined by the two variables of is introduced. That is, the threshold TH of the second layer is equal to or higher than the threshold TH1 of the first layer
Find a block frequency of 2 or more.

In this operation, in the frequency distribution table of FIG. 10, by calculating the block frequencies of the threshold TH1 or more on the ACT1 axis and the threshold TH2 or more on the ACT2 axis, the generated information amount of the second layer satisfying the above condition is calculated. Calculation is realized. FIG. 10 shows a case where ACT1 is discretely measured, and the block activity ACT2 of the second layer is distributed for each value of ACT1.

Next, FIG. 11 shows an example of the frequency distribution table of the third and fourth layers. According to the same idea as the frequency distribution table of the second layer, the frequency distribution table defined by multivariables is generated.

In the third hierarchy, the block activity AC of each of the first hierarchy, the second hierarchy, and the third hierarchy.
The blocks defined by T1, ACT2, and ACT3 are registered in the frequency distribution table. This state is shown in FIG. In the third hierarchical layer, the threshold value TH1 or more on the ACT1 axis,
The amount of generated information in the third layer is calculated by calculating the block frequencies of the threshold TH2 or more on the axis and the threshold TH3 or more on the ACT3 axis.

In the fourth layer, the blocks defined by the block activities ACT1, ACT2, ACT3, and ACT4 of the first layer, the second layer, the third layer, and the fourth layer are registered in the frequency distribution table. This state is shown in FIG. In the case of the fourth layer, the threshold value is TH1 or more on the ACT1 axis, the threshold value TH2 or more on the ACT2 axis, and AC.
Threshold TH3 or more on the T3 axis, threshold TH4 on the ACT4 axis
The amount of generated information in the fourth layer is calculated by calculating the above block frequencies.

By using the above five types of frequency distribution tables, it is possible to calculate the amount of generated information with respect to the threshold value and perform control that matches the target amount of information. Here, there is a method of independently changing the threshold value of each layer used for controlling the generated information amount for each layer.

For example, this is a method in which the target information amount is set in advance for each layer and the threshold value is changed independently for each layer to perform control to match the target information amount. Another method is to prepare a combination of thresholds for each layer in advance and apply the threshold combination according to the control order to simplify the control.

Next, the frequency distribution table will be described. In the generated information amount control method using the frequency distribution table in each layer described above, in each layer, the block frequency of each layer is calculated while considering the division determination results of all lower layers. The optimum control value was detected at. In order to speed up the block frequency calculation above this threshold, it is proposed to reconfigure the frequency distribution table in which the block frequencies are registered into an integrated type frequency distribution table.

An example of this integrated type frequency distribution table is shown in FIG. As a result of registering the block activity, it is assumed that the frequency distribution table example of FIG. 12 is obtained. For simplicity of explanation,
Block activity is an example of one variable.

The cumulative type frequency distribution table (FIG. 13) starts from the block frequency corresponding to the maximum value of the block activity in the frequency distribution table of FIG. 12, and the block frequency corresponding to the smaller block activity value is integrated. ,
It has a structure in which each integration result is registered again in the frequency distribution table.

This process is expressed by a mathematical expression as follows.

[Equation 8] Is represented by Here, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act represents the block activity variable in the cumulative frequency distribution table, and ACT represents the block activity variable in the frequency distribution table. Represents the maximum value of the variable in the frequency distribution table.

The expression (8) means that the block frequency of the block activity value address is read out, and the result of addition to the integrated value up to the upper block activity value is written to the block activity value address. The result is shown in FIG. In the integrated type frequency distribution table, the block frequency sum in the shaded portion of FIG. 12 corresponds to the threshold TH coordinate data I.

According to this integrated type frequency distribution table, it is not necessary to calculate the block frequency sum in the shaded area in the above figure every time the threshold value TH is changed. That is, the total block frequency sum can be calculated by outputting the integrated block frequency corresponding to the threshold value of the integrated type frequency distribution table.

Since FIG. 13 is also an example of one variable, FIG.
It is applied to the frequency distribution table of the first layer of. The second layer frequency distribution table of FIG. 10 shows the case of two variables. By expanding the expression (8), an integrated type frequency distribution table is generated.

[Equation 9] Is represented by However, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act1 represents the variable corresponding to the first layer in the cumulative frequency distribution table, and act2 represents the variable in the cumulative frequency distribution table. A variable corresponding to two layers is represented, ACT1 represents a first layer variable in the frequency distribution table, ACT2 represents a second layer variable in the frequency distribution table, and n represents a variable maximum value in the frequency distribution table.

In the integrated type frequency distribution table generated according to the equation (9), the integrated block frequencies corresponding to the addresses of the first layer determination threshold value TH1 and the second layer determination threshold value TH2 are equal to or higher than the first layer threshold value TH1. And the second layer determination threshold TH
The sum of block frequencies of 2 or more is shown.

In this way, it becomes possible to calculate the amount of generated information in the second layer. With respect to the frequency distribution tables of the third and fourth hierarchies in FIG. 11, the block frequency sum calculation time can be shortened by using the integrated type frequency distribution table.
In these cases, since the number of block activity variables increases, the number of times of integration increases.

First, the arithmetic expression for the third layer is

[Equation 10] It is represented by However, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act1 represents the variable corresponding to the first layer in the cumulative frequency distribution table, and act2 represents the variable in the cumulative frequency distribution table. Represents a variable corresponding to the second hierarchy, act3 represents a variable corresponding to the third hierarchy in the cumulative frequency distribution table, ACT1 represents a first hierarchy variable in the frequency distribution table, ACT2 represents a second hierarchy variable in the frequency distribution table, and ACT3 represents It represents the third hierarchical variable in the frequency distribution table, and n represents the variable maximum value in the frequency distribution table.

In the integrated type frequency distribution table generated according to the equation (10), the integrated block frequencies corresponding to the addresses of the first layer determination threshold value TH1, the second layer determination threshold value TH2, and the third layer determination threshold value TH3. Is the first layer threshold TH1
Above and above the second layer threshold TH2 Above and above the third layer threshold TH
A block frequency sum of 3 or more is shown.

In this way, it is possible to calculate the amount of generated information in the third layer. Further, the processing regarding the frequency distribution table of the fourth layer in FIG. 11 will be described. In this case, since there are four types of block activity variables, the number of integration calculations is the largest.

The arithmetic expression for the fourth layer is

[Equation 11] It is represented by However, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act1 represents the variable corresponding to the first layer in the cumulative frequency distribution table, and act2 represents the variable in the cumulative frequency distribution table. Represents a two-tier corresponding variable, act3 represents a third-tier corresponding variable in the cumulative frequency distribution table, act4 represents a fourth-tier corresponding variable in the cumulative frequency distribution table, ACT1 represents a first-tier variable in the frequency distribution table, ACT2 represents the second hierarchical variable in the frequency distribution table, ACT3 represents the third hierarchical variable in the frequency distribution table, ACT4 represents the fourth hierarchical variable in the frequency distribution table, and n represents the variable maximum value in the frequency distribution table. .

In the integrated type frequency distribution table generated by the equation (11), the first layer judgment threshold value TH1, the second layer judgment threshold value TH2, the third layer judgment threshold value TH3, and the fourth layer judgment threshold value TH4. The total block frequency corresponding to the address is the sum of block frequencies of the first layer threshold TH1 or more, the second layer threshold TH2 or more, the third layer threshold TH3 or more, and the fourth layer threshold TH4 or more.

In this way, the generated information amount in the fourth layer is also calculated. As a result of this processing, calculation of the amount of generated information based on the number of blocks to be divided in each layer by Expression (6) is realized. With the introduction of the integrated type frequency distribution table as described above, it becomes possible to significantly reduce the generated information amount control time.

Further, with respect to this integrated type frequency distribution table, a proposal is made to further reduce the generated information amount control time. The integrated type frequency distribution table used in this proposal is used to calculate the generated information amount with respect to the division determination threshold.

In the actual threshold processing, a large judgment threshold cannot be practically used from the viewpoint of image quality deterioration. Therefore, we propose to create a frequency distribution table that clips the block activity values. Figure 14
15 and FIG. As shown in FIG. 14, when the block activity value is clipped by LMT, all block frequencies equal to or higher than LMT are registered in LMT in the frequency distribution table.

As a result, the block frequency in the LMT becomes large. The block frequency sum to be calculated is the slope. An integrated type frequency distribution table for this frequency distribution table is shown in FIG. The integration calculation shown in the equations (8) to (11) is performed not in the maximum block activity value n but in the interval from the block activity value LMT to 0.

The block frequency sum to be calculated is the integrated block frequency I of the coordinates of the threshold value TH. As shown in this example, the same result as in FIG. 13 is obtained. By introducing a clip into the block activity value of the frequency distribution table, it is possible to shorten the time for creating the integrated type frequency distribution table and the size of the frequency distribution table memory space.

As a frame to which this method is applied, there are two types of cases, that is, the clip value LMT is changed for each layer, and the clip value LMT is fixed for all layers.
The former is used when there is a clear difference in the difference value distribution between layers in each layer, and the latter is used when the difference value distribution between layers is not significantly different.

Incidentally, FIG. 16 shows a flow chart of the hierarchical coding process. In step SP2, "4" is registered in the hierarchical counter I which stores the hierarchical number and the hierarchical frame is determined.

Further, in step SP3, hierarchical data is generated by calculation of the generated information amount, and in step SP4, each block activity is detected. For this activity, the generated information amount control is performed by creating and registering the multidimensional frequency distribution table described above with reference to FIG. 10 in step SP5, and the optimum control value is determined.

Further, in step SP6, the encoder side executes the hierarchical encoding based on this control value. That is, first, encoding and decoding are performed on the uppermost five-layer data. This result becomes the initial value of the processing in the lower layer, and the difference value between layers with the lower layer is generated in step SP7. Further step SP
In 8, the division selection and encoding in the lower layer are executed based on the generated information amount control value determined in the upper stage.

After each layer processing, the layer counter I is decremented in step SP9. And step S
In P10, the end determination is performed on the contents of the hierarchy counter I. If not completed, the lower layer processing is continued. When the processing of all the layers is completed, the loop is exited and the processing is completed in step SP11. By controlling the amount of generated information as described above, it becomes possible to perform hierarchical coding with high compression efficiency with little deterioration in image quality.

(5) Other Embodiments In the above embodiment, the block activity P is determined by the maximum difference value between the decoded data obtained for the upper hierarchical data and the lower hierarchical data for each block. Although the case has been described, the present invention is not limited to this, and the judgment may be made based on the average error and the sum of absolute values in the block, the standard deviation and the sum of n, and the data frequency equal to or higher than the threshold value.

Further, in the above-mentioned embodiment, the case where the image data is PCM-encoded in the encoder has been described, but the present invention is not limited to this, and another encoding system, for example, an orthogonal encoding system is applied. Is also good.

Further, in the above-described embodiment, a plurality of combinations of the threshold values of the frequency distribution table obtained for each layer are stored in the ROM, and the threshold value at which the generated information amount is closest to the target value is set. The case where the combination is obtained has been described, but the present invention is not limited to this, and each layer may be set independently.

Further, in the above-described embodiment, the case where the average value of the lowest layer data is calculated for every 2 lines × 2 pixels to obtain the image data of the upper layer has been described, but the present invention is not limited to this. The average value may be obtained by using another combination.

[0121]

As described above, according to the present invention, when the image data is sequentially recursively divided into a plurality of hierarchical data having a plurality of different resolutions and encoded, a predetermined block of the hierarchical data is blocked. By determining the activity and setting the threshold value, which is the criterion for division processing for lower layer data, from the block frequency distribution corresponding to the block activity, a hierarchical encoding method for image data that does not reduce compression efficiency can be easily performed. Can be realized.

[Brief description of drawings]

FIG. 1 is a schematic diagram for explaining the principle of an image encoding method according to the present invention.

FIG. 2 is a table showing processing results of picked-up images adaptively divided by the image coding method according to the present invention.

FIG. 3 is a table showing signal levels for respective layers obtained by the image coding method according to the present invention.

FIG. 4 is a block diagram showing an embodiment of an image encoding device according to the present invention.

FIG. 5 is a block diagram showing a hierarchical encoding encoder unit.

FIG. 6 is a schematic diagram for explaining a hierarchical structure.

FIG. 7 is a block diagram showing a generated information amount control unit.

FIG. 8 is a characteristic curve diagram showing a frequency distribution table of each layer.

FIG. 9 is a characteristic curve diagram showing an example of a frequency distribution table.

FIG. 10 is a characteristic curve diagram showing an example of a frequency distribution table.

FIG. 11 is a characteristic curve diagram showing an example of a frequency distribution table.

FIG. 12 is a characteristic curve diagram showing a frequency distribution table.

FIG. 13 is a characteristic curve diagram showing an integrated type frequency distribution table.

FIG. 14 is a characteristic curve diagram showing a frequency distribution table using clip values.

FIG. 15 is a characteristic curve diagram showing an integrated type frequency distribution table using clip values.

FIG. 16 is a flow chart showing a hierarchical encoding process.

FIG. 17 is a block diagram showing the configuration of a conventional pyramid coding encoder.

FIG. 18 is a block diagram showing a conventional hierarchical decoding device.

[Explanation of symbols]

40 ... Hierarchical coding device, 40A ... Hierarchical coding encoder unit, 40B ... Generated information amount control unit, 41, 43, 4
5, 47, 61, 62, 63, 64 ... Difference circuit, 4
2, 44, 46, 46 ... Averaging circuit, 51, 52, 5
3, 54, 55 ... Encoder, 56, 57, 58, 59 ...
Decoder, 65, 66, 67, 68 ... Activity detection circuit, 69, 70, 71, 72, 73 ... Frequency distribution table, 74 ... Control section.

[Procedure amendment]

[Submission date] November 24, 1994

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] Full text

[Correction method] Change

[Correction content]

[Document name] Statement

Title of image coding method

[Claims]

Detailed Description of the Invention

[0001]

[Table of Contents] The present invention will be described in the following order. Industrial Application Conventional Technology (FIGS. 19 and 20) Problem to be Solved by the Invention (FIGS. 19 and 20) Means for Solving the Problem (FIGS. 1 to 12) Action (FIGS. 1 to 12) Example (1) Principle of hierarchical coding (FIGS. 1 to 3) (2) Overall configuration of image coding apparatus (FIG. 4) (3) Hierarchical coding encoder unit (FIGS. 5 to 8) (4) Generated Information Amount Control Unit (FIGS. 9 to 18) (5) Other Embodiments Effects of the Invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image coding method,
For example, it is suitable to be applied to an image encoding device that divides and encodes predetermined image data into a plurality of image data having different resolutions.

[0003]

2. Description of the Related Art Conventionally, as this type of image coding apparatus,
There is one in which input image data is hierarchically encoded using a hierarchical encoding method such as pyramid encoding (Japanese Patent Application No.
No. 142836). In this hierarchical encoding device, high-resolution input image data is used as first hierarchical data, second resolution data having a lower resolution than the first hierarchical data, and further higher resolution than the second resolution data. .. are sequentially formed recursively, and the plurality of hierarchical data are transmitted through a transmission line such as a communication line or a recording / reproducing route.

Further, at this time, in the image decoding apparatus for decoding a plurality of hierarchical data, all of the plurality of hierarchical data may be decoded, and any one of the hierarchical data may be decoded depending on the resolution of the television monitor corresponding to each. One of the desired ones may be selected and decoded. By decoding only the desired hierarchical data from the plurality of hierarchical data thus hierarchized, desired image data can be obtained with the minimum required transmission data amount.

Here, as shown in FIG. 19, in an image coding apparatus which realizes, for example, four-layer coding as this hierarchical coding, thinning filters 2, 3 and 4 and interpolation filters 5 and 6 for three stages are respectively provided. , 7 and the input image data D1
With respect to each of the stages, the thinned-out filters 2, 3, and 4 form reduced image data D2, D3, and D4 having a sequentially low resolution, and the interpolation filters 5, 6, and 7 reduce the reduced image data D2, D3, and D4 before the reduction. Return to resolution.

Outputs D2 to D4 of the thinning filters 2 to 4
And the outputs D5 to D7 of the respective interpolation filters 5 to 7 are input to the difference circuits 8, 9 and 10, respectively, whereby difference data D8, D9 and D10 are generated. As a result, in the image encoding device 1, the amount of hierarchical data is reduced and the signal power is reduced. Here, this difference data D
8 to D10 and the reduced image data D4 have an area of 1, 1/4, 1/16, and 1/4, respectively, with respect to the input image data D1.
The size is 1/64.

The difference data D8 to D10 obtained from the respective difference circuits 8 to 10 and the reduced image data D4 obtained from the thinning filter 4 are encoded by the encoders 11, 12, 13, respectively.
Compression processing by 14 and as a result, each encoder 11,
First, second, and third with different resolutions from 12, 13, and 14
And fourth layer data D11, D12, D13 and D1
4 are transmitted to the transmission line in a predetermined order.

The first to fourth hierarchical data D11 to D14 thus transmitted are decoded by the image decoding apparatus shown in FIG. That is, the first to fourth hierarchical data D11 to D14 are respectively decrypted by the decoders 21, 22, 2 and 2.
3, 24, which results in the decoder 24
Outputs the fourth hierarchical data D24.

The output of the decoder 23 is the fourth hierarchical data D obtained from the interpolation filter 26 in the adder circuit 29.
It is added with the interpolation data of 24, whereby the third hierarchical data D23 is restored. Similarly, the output of the decoder 22 is added to the interpolation data of the third hierarchical data D23 obtained from the interpolation filter 27 in the adder circuit 30, whereby the second hierarchical data D22 is restored. Further, the output of the decoder 21 is output by the adder circuit 31 to the interpolation filter 2
8 is added to the interpolated data of the second hierarchical data D22 to restore the first hierarchical data D21.

[0010]

However, in the image coding apparatus which realizes such a hierarchical coding method, since the input image data is divided into a plurality of hierarchical data to be coded, the data amount of only the hierarchical component is inevitable. However, there is a problem that the compression rate is reduced as compared with the high-efficiency coding method that does not use hierarchical coding. Further, when trying to improve the compression efficiency, there is a problem that the image quality is deteriorated by the quantizer applied between the respective hierarchical data.

The present invention has been made in consideration of the above points, and provides an image coding method and an image coding apparatus capable of improving compression efficiency and reducing image quality degradation when hierarchically coding image data. It is a proposal.

[0012]

In order to solve such a problem, according to the present invention, image data is sequentially recursively divided into a plurality of hierarchical data D31 to D35 having a plurality of different resolutions, and is encoded. When significant activity is detected in lower layer data, the division process returns to the upper layer where the division process was interrupted, the division determination flag is reset, and the division determination process is performed again for the lower layer. When the generated information amount is controlled to the target value by selecting the division processing of the block Xj (i) spatially corresponding to, the block activity ACT defined between the layers is detected over all the layers, and the detection result is detected. The optimum control value is detected using the block frequency distribution table.

In the present invention, the block Xj
When the generated information amount is reduced by selecting the division processing of (i) and the generated information amount is controlled to a target value, the block activity ACT defined between layers is detected over all layers, and the detection result is detected. With respect to the block frequency distribution table generated based on the above, the optimum control value is detected in each layer based on the history of block activity in the upper layer.

Further, in the present invention, the data D31 input to the encoder 40A for hierarchical coding is controlled in a standby state until the optimum control value is determined, and the input data D31 is hierarchically coded when the optimum control value is determined. To do so.

Further, in the present invention, the control threshold value TH is set independently for each layer in the optimum control value detection process using the frequency distribution table constituted by block activity ACTs of a plurality of layers.

Further, according to the present invention, when the control threshold value TH is set independently for each layer in the optimum control value detection process using the frequency distribution table composed of block activities of a plurality of layers, each layer is set. A combination of the control threshold values of 1 is prepared in advance, and the optimum control value is detected from the combination.

Further, according to the present invention, in the optimum control value detection process using the frequency distribution table constituted by the block activity ACTs of a plurality of layers, after the data is registered in the frequency distribution table, the integration type is calculated by the value of the block activity ACT. Create a frequency distribution table to shorten the optimum control value detection time.

Further, in the present invention, when a block is registered in the frequency distribution table, a limit LMT is set to the block activity value ACT in the frequency distribution table, and a block having a block activity equal to or larger than the limit value is registered in a predetermined coordinate. And reduce the amount of memory required.

Further, in the present invention, the image data is recursively divided into a plurality of hierarchical data having a plurality of different resolutions and encoded, and when significant activity is detected in the lower hierarchical data having a high resolution, the image data is divided. In the image coding method in which the division determination flag is reset by returning to the upper layer where the processing is interrupted and then the division determination process is performed toward the lower layer again, it occurs due to the selection of the block division process of spatially corresponding blocks between adjacent layers. When controlling the amount of information to the target value, the block activity defined between each layer is detected over all layers, and the optimal control value is detected using the block frequency distribution table based on the detection result, and the detection result is obtained. At this time, the image data waiting in the predetermined memory M1 is encoded.

[0020]

When the image data D31 is sequentially and recursively divided into a plurality of hierarchical data D31 to D35 having a plurality of different resolutions and encoded, a predetermined block of hierarchical data except the highest hierarchical data having the lowest resolution. For the block activity P, the threshold TH, which is the criterion for the division processing for the lower lower layer data, is set based on the distribution of the block frequencies corresponding to the block activity P. Hierarchical coding can be realized.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail below with reference to the drawings.

(1) Principle of Hierarchical Coding FIG. 1 shows, as a whole, the principle of hierarchical coding according to the present invention in which a still image such as a high-definition television signal is hierarchically coded and compressed. In this layer coding, upper layer data is created by a simple arithmetic average of lower layer data, and lower layer data to be transmitted is reduced to realize a layered structure without an increase in information amount. For decoding from the upper layer to the lower layer, the amount of information in the flat portion is reduced by adaptively controlling the division based on the activity of each block. Further, in the encoding of the differential signal performed for the lower layer, the efficiency is improved by switching the quantization characteristic for each block without additional code based on the activity of the upper layer.

That is, in the hierarchical structure of this hierarchical encoding,
First, the high-definition television signal to be input is defined as the lower layer, and the four pixels X1 to X4 in the small block of 2 lines × 2 pixels in this lower layer are expressed by the following formula

[Equation 1] The arithmetic mean represented by is taken, and its value m is taken as the value of the upper hierarchy. In this lower hierarchy,

[Equation 2] As shown in, the difference value from the upper layer is prepared for three pixels, so that the hierarchical structure is configured with the same information amount as the original four pixel data.

On the other hand, when decoding the lower layer, 3 pixels X1
~ X3 is the following formula

[Equation 3] As shown by, each difference value Δ is added to the average value m of the upper hierarchy.
Xi is added to obtain the decoded value E [Xi], and the remaining one pixel is

[Equation 4] As shown by, the decoded value E [X4] is determined by subtracting the three decoded values of the lower layer from the average value m of the upper layer. Here, E [•] means a decoded value.

Here, in this hierarchical encoding, the resolution is quadrupled from upper layer to lower layer, but in the flat part, this division is prohibited to reduce redundancy. A flag for instructing the presence / absence of this division is prepared for each bit and block. The necessity of division in the lower layer is determined as local activity, for example, the maximum value of the difference data.

Here, as an example of hierarchical coding, HD of ITE is used.
FIG. 2 shows the adaptive division result when the standard image (Y signal) is used and five-layer coding is performed. The ratio of the number of pixels in each layer to the original number of pixels when the threshold value for the maximum difference data is changed is shown, and it can be seen that redundancy is reduced based on the spatial correlation. Although the reduction efficiency changes depending on the image, if the threshold value for the maximum difference data is changed to 1 to 6, the average reduction rate becomes 28 to 69 [%].

In practice, the resolution of the upper layer is quadrupled to form the lower layer, and at that time, the difference data from the upper layer data is encoded in the lower layer, whereby the signal level width can be effectively reduced. FIG. 3 shows the case of five layers by the layer coding described above with reference to FIG. 2, but here, the layers are named from the lower layers to the first to fifth layers.

Compared to the 8-bit PCM data of the original image,
A reduction in signal level width can be seen. First with a large number of pixels
Since the ~ 4 layers are differential signals, a significant reduction can be achieved and efficiency is improved by subsequent quantization. As can be seen from the table of FIG. 3, the reduction efficiency has little dependence on the pattern, and is effective for all pictures.

By forming the upper layer with the average value of the lower layer, the error propagation is stopped in the block, and the lower layer is converted into the difference from the average value of the upper layer, so that the efficiency is also provided. You can Actually, in the upper layer coding, there is a correlation in the activities between layers at the same spatial position, and by determining the quantization characteristic of the lower layer from the quantization result of the upper layer, the receiver side can be used for dequantization. It is possible to realize an adaptive quantizer that does not need to transmit quantization information (however, except for initial values).

In practice, an image is hierarchically encoded based on the above-mentioned five-stage hierarchical structure to be expressed in multi-resolution, and adaptive division and adaptive quantization using the hierarchical structure are performed to obtain various HD standard images (8 Bit / Y / PB / PR) is about 1 /
It can be compressed to 8. The additional code for each block prepared for adaptive division is run-length coded in each layer to improve compression efficiency. In this way, an image with sufficient image quality can be obtained in each layer, and a good image without visual deterioration can be obtained in the final lowest layer.

(2) Overall Structure of Image Coding Device In FIG. 4, reference numeral 40 denotes an image coding device according to the present invention, which is a hierarchical coding encoder unit 40A for hierarchically coding and outputting input image data D1 and a hierarchical coding. Encoder part 4
The generated information amount control unit 40B controls the generated information amount at 0A so as to reach the target value.

The hierarchical coding encoder section 40A is composed of a memory (not shown) for data delay and an encoder. Of these, the memory is the generated information amount control unit 40B.
The input stage is provided so that the data can be delayed so that the encoding process is not executed until the optimum control value is determined in.

On the other hand, the generated information amount control unit 40B is adapted to input the input image data and determine the threshold value TH which is suitable for the data to be processed, and the input image data is efficiently encoded in the hierarchical encoding encoder unit 40A. The optimum control value determined to be converted is transmitted to the encoder. This is a so-called feedforward type buffering structure. With this configuration, it is possible to accurately control the amount of generated information and to eliminate the time delay caused by the feedback control type feedback.

Here, the block activity defined based on the difference value between layers is used to select the division process in the lower layer. That is, upper layer data is composed of 2 × 2 4 pixels in the lower layer, and a block is defined.

Here, the activity is the maximum value, the average value, the sum of absolute values, and the standard deviation of the inter-tier difference data in a predetermined block when the lower hierarchy data area corresponding to the upper hierarchy data is defined as "block". Alternatively, it is a correlation value represented by the sum of nth power. That is, when the activity is low, this block can be called a flat block.

That is, the upper layer data is X0 (i + 1).
And the lower layer data is Xj (i), the inter-layer difference code value ΔXj (i) = X0 (i + 1) −Xj (i). However, j = 0 to 3. If the block activity determination function is G (·), the block activity ACT = G (ΔXj (i)).

Further, the hierarchy determination flag FLG (0: stop division, 1: continue division) is set. Here, a method of reviewing the division determination flag will be described. Regarding each layer determination flag determination process (generation information amount control process), first, the activity ACT of all layers is generated for each block, and subsequently, the determination flag FLG is generated by the threshold value corresponding to the activity ACT of all layers. . Further, for each block, a hierarchy in which the determination flag FLG becomes 1 for the first time is searched from the lower hierarchy.

The hierarchy determination flags FL of all upper hierarchy blocks above the hierarchy in which the determination flag FLG becomes 1 for the first time.
Let G be 1. The determination flag FLG is updated according to this rule. Further, the above-mentioned determination threshold value is changed based on the generated information amount control, and the optimum threshold value that falls within the target value is selected.

Next, the block division selection process (actual encoding process) will be explained. When the judgment flag FLG = 0, the division of the lower layer is stopped, whereas when the judgment flag FLG = 1, the division of the lower layer is judged. Perform a split. The above-mentioned processing is composed of two steps in which the layer determination flag FLG is determined for each block of each layer in advance by controlling the amount of generated information, and the actual block division processing is executed based on that.

(3) Hierarchical coding encoder section The hierarchical coding encoder section 40A has the structure shown in FIG. 5, and in this example, processing is performed by dividing into five layers.

First, the input image data D31 is input to the first difference circuit 41 and the first averaging circuit 42. The first averaging circuit 42 generates the second layer data D32 by averaging four pixels of the input image data D31 (that is, the first layer data (the lowest layer data)). In the case of this embodiment, the first averaging circuit 42, as shown in FIGS. 6D and 6E, has four pixels X1 (1) of the input image data D31.
From X4 (1) to pixel X1 (2) of the second hierarchical data D2
To generate. Also, the pixel X1 of the second layer data D32
Pixels X2 (2) to X4 (2) adjacent to (2) are similarly generated by obtaining the 4-pixel average of the first layer data D31.

The second layer data D32 is the second difference circuit 4
3 and the second averaging circuit 44, and the second averaging circuit 44 generates the third hierarchical data D33 by averaging four pixels of the second hierarchical data D32. For example, FIG. 6 (C)
And pixel D1 of the second hierarchical data D32 shown in (D)
(2) to X4 (2) to the pixel X of the third hierarchical data D33
1 (3) is generated, and pixels X2 (3) to X4 (3) adjacent to the pixel X1 (3) are similarly generated by the 4-pixel average of the second hierarchical data D32.

The third layer data D33 is the third difference circuit 4
5 and the third averaging circuit 46, and the third averaging circuit 46 uses the four-pixel average of the third hierarchical data D33 as shown in FIGS. 6B and 6C, as in the case described above. ,
Fourth layer data D3 including pixels X1 (4) to X4 (4)
4 is generated.

The fourth layer data D44 is the fourth difference circuit 4
7 and the fourth averaging circuit 48, and the fourth averaging circuit 48 generates the fifth layer data D35 which is the highest layer by averaging four pixels of the fourth layer data D34. That is, as shown in FIGS. 6A and 6B, by averaging the four pixels X1 (4) to X4 (4) of the fourth layer data D34, the pixel X1 (5) of the fifth layer data D35 becomes Is generated.

Here, the first to fifth hierarchical data D31 to D3
As for the block size of 5, when the block size of the first layer data D31 which is the lowest layer is 1 line × 1 pixel, the second layer data D32 is 1/2 line × 1/2 pixel, and the third layer data D33 is 1/2 line × 1/4 pixel, the fourth layer data D34 is 1/8 line × 1/8 pixel, and the fifth layer data D35 which is the highest layer data is 1
/ 16 line × 1/16 pixel.

The hierarchical coding encoder unit 40A repeats the recursive processing in order from the highest hierarchical data (that is, the fifth hierarchical data D35) among the first to fifth hierarchical data D31 to D35 and adjoins two. Differences between one hierarchical data are obtained in the difference circuits 41, 43, 45, 47, and only the difference data is compression-encoded by the encoders 51-55. As a result, the hierarchical encoding encoder unit 40A compresses the amount of information transmitted on the transmission path.

Further, the hierarchical encoding encoder 40A is
As described above with respect to the equation (2), the encoders 51 to 54 reduce the transmission data amount by reducing one pixel of the lower layer 4 pixels corresponding to the upper layer 1 pixel.

In order to keep such a compression condition optimum, the layered coding encoder section 41 decodes the transmission data D51 to D55 obtained for each layer by the decoders 56 to 59.

Of these, the decoder 5 corresponding to the highest layer
Reference numeral 9 denotes the decoded data D48 corresponding to the fifth layer data D35 compression-coded by the encoder 55, and the transmission data D48.
It is decoded from 55 and is applied to the fourth level difference circuit 47.

On the other hand, the other decoders 51 to 54 switch the decoding operation based on the flags indicating the presence / absence of the division / non-division processing. That is, when the division processing is performed, the upper hierarchical data (that is, the fourth, third, and second hierarchical data) is decoded by the decoding processing from the difference data transmitted as the transmission data D52 to D54. Third
The difference circuit 45 of the hierarchy, the difference circuit 43 of the second hierarchy, and the first hierarchy data 41 are given respectively.
As a result, the difference data D41, D42, D between the adjacent layers is output from the difference circuits 41, 43, 45, 47.
43 and D44 are obtained.

In practice, the decoders 58, 57, 56 are shown in FIG.
It is configured as shown in. Here, the decoder 58 will be described for simplification. The decoder 58 is the decoding circuit 5
8A receives the fourth layer compression encoded data D54 and decodes it. As a result, the decoding circuit 58A outputs, for example, X1 (4) -X1 (5) and X2 (4) -X1 shown in FIG.
Output values of (5) and X3 (4) -X1 (5) are obtained.
This output value is reconstructed by the reconstructed data D in the addition circuit 58B.
By adding 48, X1 (4), X2 (4),
The output value of X3 (4) is obtained. Difference value generation circuit 58C
Is X1 (4), X2 (4), X3 (4) and X1 (5)
Is used to perform the operation based on the equation (4) to generate the non-transmission pixel X4 (4). Accordingly, the following synthesis circuit 58
From D, the fourth layer data X1 (4), X2 before the difference
(4), X3 (4), and X4 (4) are generated and provided to the difference circuit 45.

The encoders 51 to 55 corresponding to the respective layers are the difference circuits 41, 43, 45 and 47 and the averaging circuit 4.
Difference data D41, D42, D4 obtained by
3, D44 or fifth layer data D35 is input, and threshold value determination and division selection processing for the activity obtained for each block are executed. At this time, encoder 5
1 to 55, when the processing target is a division block, the difference data obtained between layers is compression-coded as it is, and at the same time, a division determination flag for each block is added and transmitted.

On the other hand, if the processing target is a non-divided block, the encoders 51 to 55 exclude this block from the encoding target as being generated from the upper layer data on the receiving side. Incidentally, also in this case, the division determination flag for each block is attached and transmitted. The first to fifth hierarchical compression coded data output from the five sets of encoders 51 to 55 are sent to a predetermined transmission path.

The encoders 54, 53 and 52 in this case respectively use the threshold value judgment result information J1, J2 and J3 used to represent the division or non-division of the blocks, respectively, to the encoders 53, 52 and 52 51 to 52, the encoders 51, 52 and 53 respectively send threshold value judgment result information J
4, J3, and J2 are adjacent to the upper-layer encoders 52 and 5
It is designed to be sent to 3, 54.

That is, in the hierarchical coding encoder section 40A, regarding the hierarchical data, even if the block whose division processing is interrupted due to the threshold judgment of the upper hierarchy is detected in the subsequent lower hierarchy, significant division is performed. The process returns to the layer in which the processing is interrupted, the determination flag is reset, and the threshold value determination is performed again toward the lower layer. This is because in the hierarchical data structure, the number of lower layer data corresponding to the upper layer block increases, and the significance of re-determination is recognized.

In practice, the encoders 52 and 53 are constructed as shown in FIG. The encoder 53 uses the difference data D4
3 is input to the encoding circuit 53A and the activity detection circuit 53C of the division control unit 53B. The activity detection circuit 53C detects the activity of the differential data D43 for each predetermined block, and supplies the detection result obtained thereby to the subsequent threshold value determination circuit 53D. Threshold judgment circuit 5
The 3D compares the activity detection result for each block with the threshold value data D57, and sends the determination result obtained as the threshold value determination result information J2 to the encoding circuit 53A and the encoder 52 of the adjacent lower layer. . Encoding circuit 5
3A compresses and encodes a block having high activity based on the threshold judgment result information J2, and transmits the block.
On the other hand, blocks with low activity are not transmitted.

Here, the activity detection circuit 53C and the threshold value determination circuit 53D are the encoder 5 of the adjacent upper layer.
4. When the threshold value judgment result information J1 output from No. 4 is received and the threshold value judgment result information J1 indicates that the block is divided, the activity detection and the threshold value judgment result are executed. To do. On the other hand, if the threshold judgment result information J1 indicates that the block is not divided, the activity detection and the threshold judgment are not performed for the corresponding block, and the threshold judgment circuit 53D is used. To output threshold value judgment result information J2 indicating non-division of the block.

Similarly, when the activity detection circuit 52C and the threshold value judgment circuit 52D receive the threshold value judgment result information J2 indicating the division of the block from the adjacent upper level encoder 53, While the activity detection and the threshold value determination for the corresponding block are executed, when the threshold value determination result information J2 indicating the non-division of the block is received, the activity detection and the threshold value determination are not performed. , Threshold judgment circuit 52D
From threshold value judgment result information J3 indicating non-division of block
Is output. In this way, the hierarchical encoding encoder unit 40A
In, once we get the non-division decision result of the block,
The block corresponding to it is not divided into blocks (that is, not encoded) in the subsequent lower layers.

In addition to this, the hierarchical coding encoder section 40
In A, for example, even if the division control unit 53B of the encoder 53 obtains the threshold value determination result information J2 indicating the non-division of the block, for example, by the encoder 51,
When the threshold value judgment result information J4 representing the division of the block is obtained, the encoder 52 receives the threshold value judgment result information J4 in the division control unit 52B to perform the threshold judgment of the block activity. , Decide whether to split the block or not.

Next, specific signal processing by the hierarchical coding encoder section 40A will be described. First, consider the case where the processing for the difference value between layers is selected by the block activity based on the difference value between layers. Each block is composed of 2 lines × 2 pixels.

Here, the data value of each pixel is X, and the hierarchy of the data value X is represented by a suffix. That is, when the upper layer data is X (0) i + 1, the adjacent lower layer data is Xj (i) (j = 0 to 3). The difference code value between layers is ΔXj (i) (j = 0 to 3),
The hierarchical encoding encoder unit 40A compresses and encodes this differential code value.

In the compression encoding processing by the encoders 51 to 55 in each layer, the block activity P obtained for each block is compared with the threshold data D57, and the processing is selected according to the comparison result. That is, when the block activity P is equal to or higher than the threshold value TH, the lower layer is sequentially divided, whereas when the block activity P is less than the threshold value TH, the lower layer is divided.

As a result, with respect to the region having a low block activity P, only the upper layer data need be sent.
The amount of transmitted information can be reduced. Further, the image data decoding device which receives these data across the transmission path restores the lower layer data as the upper layer data in the region with low block activity by using the upper layer data among the transmission data sequentially transmitted. On the other hand, in a region with high block activity, the data is restored by adding the inter-layer difference decoded value and the upper layer data.

A one-bit judgment flag is introduced for the judgment result of this division or non-division. It is possible to instruct the determination result for each block by this flag. This judgment flag is 1 for each block in each layer.
Bit by bit is required, but it is effective when the image quality is taken into consideration.

(4) Generated Information Amount Control Unit Next, FIG. 9 shows an example of a block diagram of the generated information amount control unit. First, the same averaging circuit 42 performs 1/4 averaging processing on the same input image data D31 as the encoder section of FIG. 5, and second layer data D32 is generated. In the averaging circuit 44, the second layer data D32 is 1 /
The four averaging process is executed to generate the third tier data D33.

Similarly, the fourth tier data D34 is obtained by the quarter averaging process of the averaging circuit 46 for the third tier data D33.
Is generated. Finally, the averaging circuit 48 performs the 1/4 averaging process to generate the fifth layer data D35.

The frequency of the data in the fifth layer data D35 is registered in the frequency distribution table 73. This measures the frequency of data corresponding to the compression processing executed by the encoder unit described above. For example, for the fifth layer data D35, P
When the compression processing by CM coding is performed, the dynamic range given for each block is registered as data, and ADRC (adaptive dynamic range coding (USP-4703352)) is applied as the compression processing method. The block DR is registered.

Next, difference data D64 is generated from the fourth hierarchy data D34 and the fifth hierarchy data D35. With respect to the difference data D64, the block activity detection described above is performed by the block activity detection circuit 68.
The block activity D68 detected there is registered in the frequency distribution table 72.

The block division determination in the upper layer is determined with reference to the block activity determination results of all the lower layers. Therefore, the frequency distribution table 72 is defined by the four variables of the first hierarchy block activity D65, the second hierarchy block activity D66, the third hierarchy block activity D67, and the fourth hierarchy block activity D68.

Third layer data D32 and fourth layer data D
The difference data D63 is generated from 34. Difference data D
With respect to 63, the block activity is detected by the activity detection circuit 67. The detected activity D67 is registered in the frequency distribution table 71.

Also in this case, the block division is determined with reference to the determination results of the block activities of all the lower layers below the third layer. Therefore, the frequency distribution table 71 of the third layer is based on the block activity D65 of the first layer and the second block activity D65.
It is defined by the three variables of the hierarchical block activity D66 and the third hierarchical block activity D67.

Second layer data D32 and third layer data D
The difference data D62 is generated from 32, and the block activity D66 is output in the activity detection circuit 66. Block activity detected D66
Is registered in the frequency distribution table 70.

In this case, the determination is made with reference to the block activity determination result of the first layer. The frequency distribution table 70 of the second layer is the block activity D65 of the first layer.
And the second layer block activity D66.

Finally, the difference data D61 is generated from the first layer data D31 and the second layer data D32, and the activity detecting circuit 65 outputs the block activity D65. The detected block activity D65 is registered in the frequency distribution table 69.

With respect to the first layer, it is not necessary to monitor the block activity of the upper layer because the threshold determination of the block activity is independently performed and the result is executed. That is, the frequency distribution table 1 of the first layer is
This is a one-dimensional frequency distribution table composed of hierarchical block activity D65.

Here, in the process of generating the frequency distribution table, in order to accurately grasp the transmission data amount of the encoder unit, among the lower layer 4 pixels corresponding to the upper layer 1 pixel, 3 pixels which are actually transmission targets by the encoder are used. It is designed to use.

The frequency distribution tables 69 to 73 thus generated.
The generated information amount control is executed by using. Each frequency distribution table and the control unit at the subsequent stage are connected by bidirectional signal paths D69 to D73. In the control unit, first, the threshold value for each frequency distribution table is transmitted to each frequency distribution table.

In each frequency distribution table, the amount of generated information corresponding to the threshold value is detected. The generated information amount in each frequency distribution table is controlled by the control unit 74 through the signal paths D69 to D73.
Be transmitted to. The control unit 74 integrates the generated information amounts in the received frequency distribution tables and calculates the total generated information amount to be controlled.

The total generated information amount is compared with the target value, and the threshold value is changed so as to satisfy the target value according to the comparison result. Again, the updated threshold value is set by the controller 74 to the signal path D
69 to D73 to be transmitted to each frequency distribution table. The corresponding amount of generated information is transmitted to the control unit again.

By repeating the above processing, the control result D57 that finally achieves the target value is determined. The determined generated information amount control value D57 is transmitted to the hierarchical coding encoder unit, as shown in the block diagram of FIG.

During the processing period of the information amount control unit, the data to be controlled is kept on standby by the memory M1 included in the encoder unit. In the above-mentioned information amount control, the threshold value suitable for the target data is determined, so that efficient encoding can be realized.

10A to 10E show block activity frequency distribution tables obtained for the highest hierarchy data to the lowest hierarchy data, respectively. Here, regarding the frequency distribution table for the fifth layer shown in FIG. 10A, since the target data is not difference data, a frequency distribution table based on the dynamic range is generated. For example, when compression processing by PCM encoding is performed on the fifth layer data D35, the dynamic range given for each block is registered as data, and when ADRC is applied as the compression processing method, the DR of the ADRC block is be registered.

Next, an example of the frequency distribution table for controlling the amount of generated information is shown in FIGS. 11 to 17 for the case of 5 layers. First, a definitional expression for calculating the total generated information amount is derived. In order to calculate the amount of generated information, it is necessary to measure the number of effective blocks that are equal to or greater than the division determination threshold in each layer, but in the generated information amount control of layer coding by the determination flag review method, in the upper layer for each block. The division determination of the target block must be performed after considering the division determination results of all lower layers.

At this time, the block activity determination thresholds used for the division determination in each layer are the first layer division determination threshold value TH1, the second layer division determination threshold value TH2,
The third layer division determination threshold is TH3, and the fourth layer division determination threshold is TH4.

Blocks lower than the above threshold are divided into lower layers in all layers. In the upper layer, if there is no block activity equal to or higher than the threshold in all the lower layers, and the block activity of the layer is less than the threshold, block division is stopped. First, in FIG. 11A, a frequency distribution table of the fifth highest layer, which is the highest layer, is shown.

Regarding the frequency distribution table of the fifth layer, since the target data is not difference data, the information amount control corresponding to the encoding process is performed. When applying fixed length coding such as linear quantization, it is not necessary to create a frequency distribution table. The following generated information amount calculation formula can be used for the generated information amount control.

That is, assuming that the sum of the numbers of pixels in the block to be divided in the first layer is M1, the number of quantization bits of the first layer data is Q1, and the number of determination flag bits of the first layer is N1, the amount of information generated in the first layer is I1 is the following formula

[Equation 5] Can be given by

The number of bits in the first term in the equation (5) is multiplied by 4 because each block is divided into 2 lines × 2 pixels in this example. In addition, what is multiplied by 3/4 in the first term is that the upper layer value is generated from the average value of the lower layer values,
This is because it reflects the property that the fourth non-transmitted pixel value of the lower layer can be restored by an arithmetic expression using the upper layer value and the transmitted lower layer value of 3 pixels.

Incidentally, in the second term, the fact that N1 is added to the number of blocks in the first layer indicates that 1 bit is added to each block as a division determination flag and transmitted.

Similarly, for the second, third and fourth layers, the sum of the numbers of pixels in the block to be divided in each layer is M.
2, M3, M4, and the number of quantization bits in each layer is Q
2, Q3, Q4, the number of judgment flag bits in each layer is N
2, N3, N4, the generated information amount I in each layer
k (k = 2, 3, 4) is

[Equation 6] Can be given by

When the generated information amounts I1 to I4 for the first to fourth layers and the generated information amount I5 for the fifth layer are used, the total generated information amount generated by the encoding process of the layer encoding encoder section 40A. I is the following formula

[Equation 7] As described above, it can be obtained as the sum of the generated information amount for each resolution.

Here, the bit number of the determination flag is added to the generated information amount of each layer, and the information amount of this flag is equal to the number of blocks for which the division processing is executed in the upper layer. The spatial position of each block can be specified in each layer by the history of the determination flag from the upper layer.

Here, each frequency distribution table will be described. As described above, the frequency distribution table of the highest hierarchy data depends on the compression method, and therefore cannot be uniquely determined. But,
The amount of generated information can be controlled by using a means such as a frequency distribution table.

Next, regarding the first layer data, the generated information amount for the threshold value TH1 is easily calculated by using the first layer frequency distribution table of FIG. 11 in which the block frequencies for the block activity ACT1 are registered. be able to. Since blocks with a threshold value TH1 or more are to be divided, by obtaining the sum of the block numbers with a threshold value or more,
The amount of generated information in the first layer is calculated.

Next, an example of the frequency distribution table of the second layer is shown in FIG.
Shown in. In the determination flag review method, the number of blocks having a threshold TH2 or more in the second layer is measured with respect to the block that has received the block division execution stop determination in the first layer.

Therefore, the first layer block activity ACT1 and the second layer block activity ACT2.
The frequency distribution table defined by the two variables of is introduced. That is, the threshold TH of the second layer is equal to or higher than the threshold TH1 of the first layer
Find a block frequency of 2 or more.

In this operation, in the frequency distribution table of FIG. 12, by calculating the block frequencies of the threshold value TH1 or more on the ACT1 axis and the threshold value TH2 or more on the ACT2 axis, the generated information amount of the second layer satisfying the above condition can be calculated. Calculation is realized. FIG. 12 shows a case where ACT1 is discretely measured, and the block activity ACT2 of the second layer is distributed for each value of ACT1.

Next, an example of the frequency distribution table of the third and fourth layers is shown in FIG. According to the same idea as the frequency distribution table of the second layer, the frequency distribution table defined by multivariables is generated.

In the third hierarchy, the block activity AC of each of the first hierarchy, the second hierarchy, and the third hierarchy.
The blocks defined by T1, ACT2, and ACT3 are registered in the frequency distribution table. This state is shown in FIG. In the third hierarchical layer, the threshold value TH1 or more on the ACT1 axis,
The amount of generated information in the third layer is calculated by calculating the block frequencies of the threshold TH2 or more on the axis and the threshold TH3 or more on the ACT3 axis.

In the fourth layer, the blocks defined by the block activities ACT1, ACT2, ACT3, and ACT4 of the first layer, the second layer, the third layer, and the fourth layer are registered in the frequency distribution table. This state is shown in FIG. In the case of the fourth layer, the threshold value is TH1 or more on the ACT1 axis, the threshold value TH2 or more on the ACT2 axis, and AC.
Threshold TH3 or more on the T3 axis, threshold TH4 on the ACT4 axis
The amount of generated information in the fourth layer is calculated by calculating the above block frequencies.

By using the above five types of frequency distribution tables, it is possible to calculate the amount of generated information with respect to the threshold value and perform control that matches the target amount of information. Here, there is a method of independently changing the threshold value of each layer used for controlling the generated information amount for each layer.

For example, this is a method in which the target information amount is set in advance for each layer and the threshold value is changed independently for each layer to perform control to match the target information amount. Another method is to prepare a combination of thresholds for each layer in advance and apply the threshold combination according to the control order to simplify the control.

Next, the frequency distribution table will be described. In the generated information amount control method using the frequency distribution table in each layer described above, in each layer, the block frequency of each layer is calculated while considering the division determination results of all lower layers. The optimum control value was detected at. In order to speed up the block frequency calculation above this threshold, it is proposed to reconfigure the frequency distribution table in which the block frequencies are registered into an integrated type frequency distribution table.

An example of this integrated type frequency distribution table is shown in FIG. As a result of registering the block activity, it is assumed that the frequency distribution table example of FIG. 14 is obtained. For simplicity of explanation,
Block activity is an example of one variable.

The cumulative type frequency distribution table (FIG. 15) is started from the block frequency corresponding to the maximum value of the block activity in the frequency distribution table of FIG. 14, and the block frequency corresponding to the smaller block activity value is integrated. ,
It has a structure in which each integration result is registered again in the frequency distribution table.

This process can be expressed by a mathematical expression as follows.

[Equation 8] Is represented by Here, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act represents the block activity variable in the cumulative frequency distribution table, and ACT represents the block activity variable in the frequency distribution table. Represents the maximum value of the variable in the frequency distribution table.

The expression (8) means that the block frequency of the block activity value address is read out, and the result obtained by adding it to the integrated value up to the upper block activity value is written into the block activity value address. The result is shown in FIG. In the integrated type frequency distribution table, the block frequency sum in the shaded portion of FIG. 14 corresponds to the threshold TH coordinate data 1.

With this integrated type frequency distribution table, it is not necessary to calculate the block frequency sum in the shaded area in the above figure every time the threshold value TH is changed. That is, the total block frequency sum can be calculated by outputting the integrated block frequency corresponding to the threshold value of the integrated type frequency distribution table.

Since FIG. 15 is also an example of one variable, FIG.
It is applied to the first layer frequency distribution table of 1. The second layer frequency distribution table of FIG. 12 shows a case of two variables, and an integrated type frequency distribution table is generated by expanding the expression (8), and

[Equation 9] Is represented by However, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act1 represents the variable corresponding to the first layer in the cumulative frequency distribution table, and act2 represents the variable in the cumulative frequency distribution table. A variable corresponding to two layers is represented, ACT1 represents a first layer variable in the frequency distribution table, ACT2 represents a second layer variable in the frequency distribution table, and n represents a variable maximum value in the frequency distribution table.

In the integrated type frequency distribution table generated according to the equation (9), the integrated block frequencies corresponding to the addresses of the first layer judgment threshold value TH1 and the second layer judgment threshold value TH2 are equal to or higher than the first layer threshold value TH1. And the second layer determination threshold TH
The sum of block frequencies of 2 or more is shown.

In this way, it is possible to calculate the amount of generated information in the second layer. With respect to the frequency distribution tables of the third and fourth hierarchies of FIG. 13, the block frequency sum calculation time can be shortened by using the integrated type frequency distribution table.
In these cases, since the number of block activity variables increases, the number of times of integration increases.

First, the arithmetic expression for the third layer is

[Equation 10] It is represented by However, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act1 represents the variable corresponding to the first layer in the cumulative frequency distribution table, and act2 represents the variable in the cumulative frequency distribution table. Represents a variable corresponding to the second hierarchy, act3 represents a variable corresponding to the third hierarchy in the cumulative frequency distribution table, ACT1 represents a first hierarchy variable in the frequency distribution table, ACT2 represents a second hierarchy variable in the frequency distribution table, and ACT3 represents It represents the third hierarchical variable in the frequency distribution table, and n represents the variable maximum value in the frequency distribution table.

In the integrated type frequency distribution table generated according to the equation (10), the integrated block frequencies corresponding to the addresses of the first layer determination threshold value TH1, the second layer determination threshold value TH2, and the third layer determination threshold value TH3. Is the first layer threshold TH1
Above and above the second layer threshold TH2 Above and above the third layer threshold TH
A block frequency sum of 3 or more is shown.

In this way, it is possible to calculate the amount of generated information in the third layer. Furthermore, the processing regarding the frequency distribution table of the fourth layer in FIG. In this case, since there are four types of block activity variables, the number of integration calculations is the largest.

The arithmetic expression for the fourth layer is

[Equation 11] It is represented by However, SUM (•) represents the cumulative block frequency, N (•) represents the block frequency in the frequency distribution table, act1 represents the variable corresponding to the first layer in the cumulative frequency distribution table, and act2 represents the variable in the cumulative frequency distribution table. Represents a two-tier corresponding variable, act3 represents a third-tier corresponding variable in the cumulative frequency distribution table, act4 represents a fourth-tier corresponding variable in the cumulative frequency distribution table, ACT1 represents a first-tier variable in the frequency distribution table, ACT2 represents the second hierarchical variable in the frequency distribution table, ACT3 represents the third hierarchical variable in the frequency distribution table, ACT4 represents the fourth hierarchical variable in the frequency distribution table, and n represents the variable maximum value in the frequency distribution table. .

In the integrated type frequency distribution table generated by the equation (11), the first layer judgment threshold value TH1, the second layer judgment threshold value TH2, the third layer judgment threshold value TH3, and the fourth layer judgment threshold value TH4. The total block frequency corresponding to the address is the sum of block frequencies of the first layer threshold TH1 or more, the second layer threshold TH2 or more, the third layer threshold TH3 or more, and the fourth layer threshold TH4 or more.

Thus, the amount of generated information in the fourth layer is also calculated. As a result of this processing, calculation of the amount of generated information based on the number of blocks to be divided in each layer by Expression (6) is realized. With the introduction of the integrated type frequency distribution table as described above, it becomes possible to significantly reduce the generated information amount control time.

Further, with respect to this integrated type frequency distribution table, a proposal is made to further reduce the generated information amount control time. The integrated type frequency distribution table used in this proposal is used to calculate the generated information amount with respect to the division determination threshold.

In the actual threshold processing, a large judgment threshold cannot be practically used from the viewpoint of image quality deterioration. Therefore, we propose to create a frequency distribution table that clips the block activity values. Figure 16
17 and FIG. As shown in FIG. 16, when the block activity value is clipped by LMT, all block frequencies equal to or higher than LMT are registered in LMT in the frequency distribution table.

As a result, the block frequency in LMT becomes large. The block frequency sum to be calculated is the slope. An integrated type frequency distribution table for this frequency distribution table is shown in FIG. The integration calculation shown in the equations (8) to (11) is performed not in the maximum block activity value n but in the interval from the block activity value LMT to 0.

The block frequency sum to be calculated is the integrated block frequency 1 of the coordinates of the threshold value TH. As shown in this example, the same result as in FIG. 15 is obtained. By introducing a clip into the block activity value of the frequency distribution table, it is possible to shorten the time for creating the integrated type frequency distribution table and the size of the frequency distribution table memory space.

There are two types of frames to which this method can be applied: one in which the clip value LMT is changed for each layer, and the other in which the clip value LMT is fixed for each layer.
The former is used when there is a clear difference in the difference value distribution between layers in each layer, and the latter is used when the difference value distribution between layers is not significantly different.

Incidentally, FIG. 18 shows a flow chart of the hierarchical coding process. In step SP2, "4" is registered in the hierarchical counter I which stores the hierarchical number, and the hierarchical frame is determined.

Further, in step SP3, hierarchical data is generated by calculating the generated information amount, and in step SP4, each block activity is detected. With respect to this activity, in step SP5, the generated information amount control is performed by creating and registering the multidimensional frequency distribution table described above with reference to FIG. 12, and the optimum control value is determined.

Further, in step SP6, the encoder side executes the hierarchical coding based on this control value. That is, first, encoding and decoding are performed on the 5th highest level data. This result becomes the initial value of the processing in the lower layer, and the difference value between layers with the lower layer is generated in step SP7. Further step SP
In 8, the division selection and encoding in the lower layer are executed based on the generated information amount control value determined in the upper stage.

After each layer processing, the layer counter I is decremented in step SP9. And step S
In P10, the end determination is performed on the contents of the hierarchy counter I. If not completed, the lower layer processing is continued. When the processing of all the layers is completed, the loop is exited and the processing is completed in step SP11. By controlling the amount of generated information as described above, it becomes possible to perform hierarchical coding with high compression efficiency with little deterioration in image quality.

(5) Other Embodiments In the above embodiment, the block activity P is determined by the maximum difference value between the decoded data obtained for the upper layer data and the lower layer data for each block. Although the case has been described, the present invention is not limited to this, and the judgment may be made based on the average error and the sum of absolute values in the block, the standard deviation and the sum of n, and the data frequency equal to or higher than the threshold value.

Further, in the above embodiment, the case where the image data is PCM-encoded by the encoder has been described, but the present invention is not limited to this, and another encoding system, for example, an orthogonal encoding system is applied. Is also good.

Further, in the above-described embodiment, a plurality of combinations of the threshold values of the frequency distribution table obtained for each layer are stored in the ROM, and the threshold value at which the generated information amount is closest to the target value is set. The case where the combination is obtained has been described, but the present invention is not limited to this, and each layer may be set independently.

Further, in the above-described embodiment, the case where the average value of the lowest layer data is calculated for every 2 lines × 2 pixels to obtain the image data of the upper layer has been described, but the present invention is not limited to this. The average value may be obtained by using another combination.

[0131]

As described above, according to the present invention, when the image data is sequentially recursively divided into a plurality of hierarchical data having a plurality of different resolutions and encoded, a predetermined block of the hierarchical data is blocked. By determining the activity and setting the threshold value, which is the criterion for division processing for lower layer data, from the block frequency distribution corresponding to the block activity, a hierarchical encoding method for image data that does not reduce compression efficiency can be easily performed. Can be realized.

[Brief description of drawings]

FIG. 1 is a schematic diagram for explaining the principle of an image encoding method according to the present invention.

FIG. 2 is a chart showing a processing result of a captured image adaptively divided by the image encoding method according to the present invention.

FIG. 3 is a table showing signal levels for respective layers obtained by the image coding method according to the present invention.

FIG. 4 is a block diagram showing an embodiment of an image encoding device according to the present invention.

FIG. 5 is a block diagram showing a hierarchical encoding encoder unit.

FIG. 6 is a schematic diagram for explaining a hierarchical structure.

FIG. 7 is a block diagram showing the configuration of a decoder.

FIG. 8 is a block diagram showing a configuration of an encoder.

FIG. 9 is a block diagram showing a generated information amount control unit.

FIG. 10 is a characteristic curve diagram showing a frequency distribution table of each layer.

FIG. 11 is a characteristic curve diagram showing an example of a frequency distribution table.

FIG. 12 is a characteristic curve diagram showing an example of a frequency distribution table.

FIG. 13 is a characteristic curve diagram showing an example of a frequency distribution table.

FIG. 14 is a characteristic curve diagram showing a frequency distribution table.

FIG. 15 is a characteristic curve diagram showing an integrated type frequency distribution table.

FIG. 16 is a characteristic curve diagram showing a frequency distribution table using clip values.

FIG. 17 is a characteristic curve diagram showing an integrated type frequency distribution table using clip values.

FIG. 18 is a flow chart showing a hierarchical encoding process.

FIG. 19 is a block diagram showing a configuration of a conventional pyramid coding encoder.

FIG. 20 is a block diagram showing a conventional hierarchical decoding device.

[Description of Codes] 40 ... Hierarchical coding device, 40A ... Hierarchical coding encoder unit, 40B ... Generated information amount control unit, 41, 43, 4
5, 47, 61, 62, 63, 64 ... Difference circuit, 4
2, 44, 46, 46 ... Averaging circuit, 51, 52, 5
3, 54, 55 ... Encoder, 56, 57, 58, 59 ...
Decoder, 65, 66, 67, 68 ... Activity detection circuit, 69, 70, 71, 72, 73 ... Frequency distribution table, 74 ... Control section.

[Procedure Amendment 2]

[Document name to be corrected] Drawing

[Correction target item name] All drawings

[Correction method] Change

[Correction content]

[Figure 1]

[Fig. 12]

[Fig. 2]

[Figure 3]

[Figure 4]

[Figure 5]

FIG. 14

[Figure 6]

[Figure 7]

FIG. 15

FIG. 16

FIG. 17

[Figure 8]

[Figure 9]

[Fig. 13]

[Figure 10]

FIG. 11

FIG. 18

FIG. 19

FIG. 20

Claims (8)

[Claims] 1. Image data is recursively divided into a plurality of hierarchical data having a plurality of different resolutions and encoded, and the division processing is interrupted when significant activity is detected in lower hierarchical data having a high resolution. In the image coding method that resets the division determination flag to the upper layer and resets the division determination process to the lower layer again, the amount of generated information is targeted by selecting the division processing of blocks that spatially correspond to adjacent layers. When controlling to a value, the block activity defined between each layer is detected over all layers, and the optimal control value is detected using the block frequency distribution table based on the detection result. Encoding method. 2. When reducing the amount of generated information by selecting the block division processing, when controlling the generated information amount to a target value, block activity defined between layers is detected over all layers, The block frequency distribution table generated based on the detection result, wherein the optimum control value is detected based on the history of block activity in an upper layer in each layer. Method. 3. The data input to the hierarchical coding encoder is controlled to a standby state until the optimum control value is determined, and the input data is hierarchically coded when the optimum control value is determined. The image coding method according to claim 1, wherein: 4. A control threshold value is independently set for each layer in the optimum control value detection process using a frequency distribution table composed of block activities of a plurality of layers. The image coding method according to claim 1. 5. A control threshold for each layer when setting a control threshold value independently for each layer in the optimum control value detection process using a frequency distribution table composed of block activities of a plurality of layers. The image coding method according to claim 1, wherein a combination of values is prepared in advance, and the optimum control value is detected from the combination. 6. An optimum control value detection process using a frequency distribution table composed of block activities of a plurality of layers, wherein after registering data in the frequency distribution table, an integrated type frequency distribution table is created according to the block activity value. The image encoding method according to claim 1, wherein the image encoding method is created to shorten the optimum control value detection time. 7. When registering a block in the frequency distribution table,
2. The block activity value of the frequency distribution table is set to a limit, and blocks having block activity equal to or greater than the limit value are registered at predetermined coordinates to reduce the required memory amount. Image coding method. 8. Image data is sequentially and recursively divided into a plurality of hierarchical data having different resolutions and encoded, and the dividing process is interrupted when significant activity is detected in lower hierarchical data having a high resolution. In the image coding method that resets the division determination flag to the upper layer and resets the division determination process to the lower layer again, the amount of generated information is targeted by selecting the division processing of blocks that spatially correspond to adjacent layers. When controlling to a value, the block activity defined between layers is detected over all layers, the optimal control value is detected using the block frequency distribution table based on the detection result, and the above detection result is obtained. The image coding method is characterized in that the above-mentioned image data held in a predetermined memory is coded.