JP2000201354A - Moving image encoder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent degradation of image quality even under environment in which an error is easy to be generated during transmission of data. SOLUTION: A significance degree 26 of a block is outputted based on an evaluation index preset for each block to be a unit of encoding by a block significance degree judging part 2. A refresh map signal 27 to indicate a priority degree of a refresh processing in each block is generated based on the significance degree 26 of the block by a map generating part 3. A refresh signal 22 is outputted by referring to the priority degree of refresh to be indicated by the refresh map signal 27 and the allowable number of blocks to be an object of the refresh processing by an adaptive refresh signal generating part 4. Encoding in a frame is performed for the block specified by the refresh signal 22 and the encoding in the frame or prediction encoding between frames is performed for the block by properly selecting one of them when the refresh signal 22 is not received by a moving image encoding part 1.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture coding apparatus for coding a continuous input picture signal, and more particularly to a moving picture coding apparatus for suppressing picture quality deterioration due to loss of coded data due to an error during transmission. Device.

[0002]

2. Description of the Related Art Conventionally, as a moving picture coding apparatus of this kind, a publication {Title of publication: Final Commit}
tee Draft 14496-2 (MPEG-4
Visual), Author: ISO / IEC JTC1 /
SC29 / W11, date of issue: March 1998, explanation page / line / drawing: Appendix × H (Adapt
(Intra Refresh) is known. As shown in FIG. 8, a conventional moving picture coding apparatus includes a moving picture coding unit 101 that appropriately selects intra-frame coding and inter-frame predictive coding for each block as a coding unit. Block to be subjected to intra-frame encoding. The conventional video coding device includes a video coding unit 101, a refresh map generation unit 102, and a refresh signal generation unit 103. The refresh map generator 102 is connected to the video encoder 101. The refresh signal generator 103 is connected to the video encoder 101 and the refresh map generator 102.

[0003] The moving image encoding unit 101 includes a motion amount estimating unit 1
04, a frame memory 105, a predicted image generation unit 106, a subtractor 107, a mode control unit 108, and a block selection unit 1.
09, two-dimensional discrete cosine transform unit 110, and quantization unit 11
1, a variable length coding unit 112, an inverse quantization unit 113, a two-dimensional inverse discrete cosine transform unit 114, and an adder 115.

[0004] The motion amount estimating unit 104 receives the input image signal 11
When 6 is input, a motion between frames is detected from the input image signal 116 and the reference image signal 117 stored in the frame memory 105, and a motion vector 118 is output. In addition, the motion amount estimating unit 104 outputs the input image signal 116
The error power signal 119 between each block in the block and the block located at the same position on the reference image signal 117 is calculated and output to the refresh map generation unit 102. The refresh map generation unit 102 determines that a block having the error power signal 119 larger than a certain threshold has a higher priority of the forced refresh, and outputs a refresh map signal 120 indicating the forced refresh priority of each block. The refresh signal generation unit 103 includes an allowable block number signal 121 indicating an allowable number of blocks capable of performing the forced refresh process in one frame, and a refresh map signal 120.
And outputs a refresh signal 122 that specifies the block to be forcibly refreshed to the mode control unit 108 that controls the encoding mode of each block. The allowable block number signal 121 is provided as an external signal according to the transmission environment of the encoded data. Increasing the number of refresh blocks improves the error concealment processing capability but degrades the coding efficiency. Allowable block number signal 121
Is a signal indicating an amount uniquely determined by a trade-off between the error concealment processing capability and the coding efficiency according to the transmission rate.

On the other hand, a predicted image generation unit 106 generates a predicted image signal 123 obtained by performing a motion compensation process on the reference image signal 117 from the frame memory 105 based on the motion vector 118 from the motion amount estimating unit 104. Subtractor 10
Give 7 The subtracter 107 subtracts the value of the predicted image signal 123 of the predicted image generator 106 from the value of the input image signal 116 to generate an error signal 124, and
Give 9 Further, the block selection unit 109 also receives the input image signal 116. The mode control unit 108 receives the input image signal 116, the error signal 124, and the refresh signal 1
Receive 22.

When receiving the refresh signal 122 from the refresh signal generator 103, the mode controller 108 sends a selection control signal 125 for selecting intra-frame coding to a block specified by the refresh signal 122. This is given to the block selection unit 109. At this time, the block selection unit 109 selects a block (data) of the input image signal 101 and supplies it to the two-dimensional discrete cosine transform unit 110. The mode control unit 108 receives the refresh signal 122 from the refresh signal generation unit 103 and, when the value of the error signal 124 is equal to or less than a predetermined threshold value, selects a control signal 125 for selecting inter-frame predictive coding. Is given to the block selection unit 109. At this time, the block selection unit 109 outputs the error signal 1
24 blocks (data) are selected and given to the two-dimensional discrete cosine transform unit 110. Also, the mode control unit 108
Is the error signal 124 when the refresh signal 122 from the refresh signal generator 103 is not received.
When the value of is greater than or equal to a predetermined threshold value, a selection control signal 125 for selecting intra-frame encoding
Give 9 At this time, the block selection unit 109 selects a block (data) of the input image signal 101 and supplies it to the two-dimensional discrete cosine transform unit 110.

[0007] The two-dimensional discrete cosine transform unit 110 performs two-dimensional discrete cosine transform on the data selected by the block selecting unit 109, and supplies the data to the quantizing unit 111. The quantization unit 111 quantizes the data and performs variable length coding
Give to 2. The variable length coding unit 112 generates output data 126 by performing variable length coding on the data. Also, the inverse quantization unit 113 receives the data from the quantization unit 111, performs inverse quantization, and performs a two-dimensional discrete cosine inverse transformation unit 114.
Give to. This two-dimensional discrete cosine inverse transform unit 114
The data from the inverse quantization unit 113 is inversely quantized and the adder 1
Give to 15. The adder 115 converts the value of the data from the two-dimensional discrete cosine inverse transform unit 114 into the predicted image generation unit 1
The reference image signal 127 is generated by combining the predicted image signal 123 with the value of the predicted image signal 123 and stored in the frame memory 105.

FIG. 9 is a diagram for explaining an error concealment method on the decoding side assumed in the conventional encoding method. Blocks for which data has been lost due to errors during transmission are complemented by copying blocks at the same position in the previous frame. In the conventional encoding method, the image quality deterioration when data loss occurs can be suppressed by selectively forcibly refreshing blocks having a large image quality deterioration in the complementing process. As described above, in the conventional moving picture coding apparatus, those whose image degradation at the time of the error concealment processing is equal to or more than a threshold are subjected to the forced refresh, and these target blocks are periodically refreshed with equal probability.

[0009]

However, in the conventional moving picture coding apparatus, the frequency of the forced refresh for determining the performance of the error concealment processing and the setting of the threshold for determining the refresh candidate block are determined by the environment of the transmission path and the image. , There is a problem that a region particularly affecting the image quality is not effectively refreshed, and only an image with a low subjective evaluation is obtained even if the error concealment process is performed.

Further, in the conventional moving picture coding apparatus, only simple error concealment on the decoding side is assumed, so that an improvement in image quality can be obtained when more advanced error concealment processing is used on the decoding side. There is a problem that may not be possible.

It is an object of the present invention to provide a moving picture coding apparatus which prevents picture quality deterioration even in an environment where errors tend to occur during data transmission.

[0012]

According to a first aspect of the present invention, there is provided a moving image encoding apparatus for encoding a continuous input image signal, comprising: Block significance determining means for determining the block significance based on a predetermined evaluation index for each block, and a map for generating a refresh map signal indicating the priority of the refresh processing in each block based on the block significance A refresh signal for selecting a block to be processed by referring to a refresh priority indicated by the refresh map signal and a permissible number of blocks to be refreshed in a frame to be coded and specifying the block to be processed; An adaptive refresh signal generating means for generating a block, and a block designated by the refresh signal. Video encoding means for performing intra-frame encoding with respect to, and appropriately selecting intra-frame encoding or inter-frame prediction encoding for blocks not specified by the refresh signal. It is characterized by.

According to a second aspect of the present invention, in the first aspect of the present invention, the block significance determining means calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. The image processing apparatus further includes a feature value calculating unit and a significance generation unit configured to compare the block feature value with one or more threshold values to generate a block significance value for each block.

According to a third aspect of the present invention, in the first aspect of the present invention, the block significance determining means calculates a block feature quantity which is a quantity representing a variance of a signal in the block; A significance generation unit configured to compare the block feature quantity with one or more thresholds to generate a block significance for each block.

According to a fourth aspect of the present invention, in the first aspect of the present invention, the block significance determining means calculates a block feature amount which is an amount representing a power of a signal obtained by passing a signal in the block through a band-pass filter. And a significance generating means for comparing the block features with one or more thresholds to generate a block significance for each block.

According to a fifth aspect of the present invention, there is provided a moving picture encoding apparatus for encoding a continuous input image signal, wherein the block information and an evaluation index determined in advance for each block serving as an encoding unit of the input image signal. Block significance determining means for determining the block significance based on the block map, map generation means for generating a refresh map signal representing the priority of the refresh processing in each block based on the block significance, An adaptive refresh signal generating means for selecting a processing target block by referring to the refresh priority and the allowable number of blocks to be subjected to the refresh processing in the encoding target frame and generating a refresh signal specifying the processing target block; Error power and motion between frames generated when coding a block The block information representing the refresh signal is generated and given to the block significance determining means, and the intra-frame coding is performed on the block specified by the refresh signal, and the frame is performed on the block not specified by the refresh signal. Moving image encoding means for appropriately selecting the inner encoding or the inter-frame prediction encoding.

According to a sixth aspect of the present invention, in the fifth aspect of the present invention, the block significance determining means calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. A feature value calculation unit, a first significance generation unit that compares the block feature value with one or a plurality of threshold values to generate a first block significance of each block, and predicts the block information from the block information in units of blocks. Visual deterioration amount calculation means for calculating a visual deterioration amount representing a degree of visual image quality deterioration when a loss of an error signal occurs,
The visual degradation amount is compared with one or a plurality of thresholds to determine a second
Second significance generating means for generating a block significance of
The image processing apparatus further includes a block significance integration unit that combines the first block significance and the second block significance and provides the combined result to the map generation unit.

According to a seventh aspect of the present invention, in the fifth aspect of the present invention, the block significance determining means calculates a block feature quantity which is a quantity representing a signal distribution feature quantity and a visual characteristic for each block. A feature value calculating unit, a first significance generation unit configured to compare the block feature value with one or more threshold values to generate a first block significance of each block, and a block on the input image signal. A visual deterioration amount calculating means for calculating a visual deterioration amount representing an error power with respect to a block located at the same position on the reference frame; and comparing the visual deterioration amount with one or more thresholds to determine a second block significance. A second significance generating means for generating; and a block significance integrating means for synthesizing the first block significance and the second block significance and providing the combined significance to the map generating means. And it features.

According to an eighth aspect of the present invention, in the fifth aspect of the present invention, the block significance determining means calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. A feature value calculating unit, a first significance generation unit configured to compare the block feature value with one or more threshold values to generate a first block significance of each block, and a block on the input image signal. A visual deterioration amount calculating means for calculating a visual deterioration amount representing an error power with respect to a block located at the same position on the predicted image obtained by the inter-frame prediction process; Second to generate a block significance of 2
, And a block significance integration means for combining the first block significance and the second block significance and giving the combined significance to the map generation means.

According to a ninth aspect of the present invention, in the fifth aspect of the present invention, the block significance determining means calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. A feature value calculating unit, a first significance generation unit configured to compare the block feature value with one or more threshold values to generate a first block significance of each block, and a block on the input image signal. Visual deterioration amount calculating means for calculating a visual deterioration amount representing an amount obtained by weighting the error power of a block located at the same position on the reference frame in accordance with the motion amount of the block, and the visual deterioration amount and one or more threshold values A second significance generating means for generating a second block significance by comparing the first block significance and the second block significance with each other to give to the map producing means. And having a that blocks significance integrating means.

According to a tenth aspect of the present invention, in the fifth aspect of the present invention, the block significance determining means calculates a block feature quantity which is a quantity representing a signal distribution feature quantity and a visual characteristic for each block. A feature amount calculating unit;
First significance generating means for comparing the block feature value with one or a plurality of thresholds to generate first block significance of each block, and a block on the input image signal and inter-frame prediction processing. Visual deterioration amount calculating means for calculating a visual deterioration amount representing an amount obtained by weighting error power with respect to a block located at the same position on the predicted image according to the amount of motion of the block; A second significance generating means for producing a second block significance by making a determination by comparing with a threshold value of the first and second maps, and combining the first block significance and the second block significance to generate the map And means for integrating block significance given to the means.

According to an eleventh aspect of the present invention, in the fifth aspect of the present invention, the block significance determining means refers to information on a luminance change and a luminance level of a signal in the block to determine a human visual characteristic for the information. A block feature value calculating unit for generating sensitivity information based on the sensitivity information as a block feature value, and generating a first block significance of each block by comparing the block feature value with one or more threshold values A first significance generating means for calculating, a visual deterioration amount calculating means for calculating a visual deterioration amount representing a degree of visual image quality deterioration when a prediction error signal is lost in block units from the block information; A second block generating a second block significance by comparing the visual deterioration amount with one or more thresholds;
, And a block significance integration means for combining the first block significance and the second block significance and giving the combined significance to the map generation means.

According to a twelfth aspect of the present invention, in the moving picture coding apparatus for coding a continuous input image signal, a block based on a predetermined evaluation index for each block as a coding unit of the input image signal is used. Block significance determination means for determining significance, map generation means for generating a refresh map signal representing the priority of refresh processing in each block based on the block significance, and the refresh map signal from the map generation means Refresh history judging means for temporarily modifying the refresh map signal and referring to the history of the refresh map signal and the refresh signal to modify the value of the forced refresh priority indicated by the refresh map signal to generate a modified refresh map signal; Refresh priority indicated by map signal and encoding pair An adaptive refresh signal generating means for selecting a block to be processed by referring to the allowable number of blocks to be refreshed in the frame of the above and generating the refresh signal for specifying the block to be processed, and the refresh signal specifies A moving image encoding unit that performs intra-frame encoding on a block, and appropriately selects intra-frame encoding or inter-frame prediction encoding on a block not specified by the refresh signal. It is characterized by the following.

According to a thirteenth aspect of the present invention, in the twelfth aspect of the present invention, the refresh history determining means determines the refresh map signal from the map generating means and the refresh signal from the adaptive refresh signal generating means. A map history memory for updating and storing the history of the refresh map with reference to the start of the encoding process; a refresh signal history memory for storing the history of the refresh signal; a map history stored in the map history memory; A map correction unit that corrects the forced refresh priority indicated by the refresh map signal from the map generation unit with reference to the refresh history stored in the signal history memory.

[0025]

Next, embodiments of the present invention will be described in detail with reference to the drawings. As shown in FIG. 1, a video encoding device according to a first embodiment of the present invention includes a video encoding unit 1, a block significance determination unit 2, and a map generation unit 3.
And an adaptive refresh signal generator 4. The block significance determination unit 2 receives the input image signal 5. The map generation unit 3 is connected to the block significance determination unit 2. The adaptive refresh signal generator 4 is connected to the video encoder 1 and the map generator 3.

The video encoding unit 1 includes a motion estimation unit 6, a frame memory 7, a predicted image generation unit 8, a subtractor 9, a mode control unit 10, a block selection unit 11, a two-dimensional discrete cosine transform unit 12, a quantum It has a transforming unit 13, a variable-length coding unit 14, an inverse quantization unit 15, an inverse two-dimensional discrete cosine transform unit 16, and an adder 17. The motion estimation unit 6 receives the input image signal 5. The frame memory 7 is connected to the motion amount estimating unit 6. The predicted image generator 8 is connected to the motion amount estimator 6 and the frame memory 7. The subtractor 9
An input image signal 5 is received. The subtractor 9 is connected to the predicted image generator 8. Mode controller 10 receives input image signal 5. The mode controller 10 is connected to the adaptive refresh signal generator 4 and the subtractor 9. The block selector 11 receives the input image signal 5. The block selector 11 is connected to the subtractor 9 and the mode controller 10. The two-dimensional discrete cosine transform unit 12 is connected to the block selecting unit 11. The quantization unit 13 is connected to the two-dimensional discrete cosine transform unit 12. The variable length coding unit 14 is connected to the quantization unit 13. The inverse quantization unit 15 is connected to the quantization unit 13. The two-dimensional inverse discrete cosine transform unit 16 is connected to the inverse quantization unit 15. The adder 17 is connected to the predicted image generation unit 8 and the inverse two-dimensional discrete cosine transform unit 16.

When the input image signal 5 is input, the motion amount estimating unit 6 detects a motion between frames from the input image signal 5 and the reference image signal 18 stored in the frame memory 7, and detects a motion vector 19. Is output. Predicted image generator 8
Generates a predicted image signal 20 obtained by performing a motion compensation process on the reference image signal 18 from the frame memory 7 based on the motion vector 19 from the motion amount estimating unit 6, and supplies the predicted image signal 20 to the subtractor 9. The subtractor 9 subtracts the value of the predicted image signal 20 of the predicted image generation unit 8 from the value of the input image signal 5 to generate an error signal 21.
Is generated and given to the block selection unit 11. Further, the block selection unit 11 also receives the input image signal 5. Mode controller 10 receives input image signal 5 and error signal 21. Further, mode control unit 10 receives refresh signal 22 from adaptive refresh signal generation unit 4.

When receiving the refresh signal 22 from the adaptive refresh signal generator 4, the mode controller 10 selects a control signal 23 for selecting the intra-frame coding for the block specified by the refresh signal 22. Is given to the block selection unit 11. At this time, the block selecting unit 11 selects a block (data) of the input image signal 5 and supplies the selected block (data) to the two-dimensional discrete cosine transform unit 12. The mode control unit 10 selects a control signal for selecting inter-frame predictive coding when the value of the error signal 21 is equal to or less than a predetermined threshold value when the refresh signal 22 from the adaptive refresh signal generation unit 4 is not received. 23 is given to the block selection unit 11. At this time, the block selection unit 11 selects a block (data) of the error signal 21 and supplies it to the two-dimensional discrete cosine transform unit 12. The mode control unit 10 performs selection control for selecting intra-frame encoding when the value of the error signal 21 is equal to or larger than a predetermined threshold value when the refresh signal 22 from the adaptive refresh signal generation unit 4 is not received. The signal 23 is provided to the block selection unit 10. At this time, the block selection unit 10
Selects the block (data) of the input image signal 5 and
This is given to the dimensional discrete cosine transform unit 12.

The two-dimensional discrete cosine transform unit 12 performs two-dimensional discrete cosine transform on the data selected by the block selecting unit 10 and supplies the data to the quantizing unit 13. This quantization unit 1
3 quantizes the data and supplies it to the variable length coding unit 14. The variable length coding unit 14 generates output data 24 by performing variable length coding on the data. In addition, the inverse quantization unit 15
Receives the data from the quantization unit 13, performs inverse quantization on the data, and supplies the inversely quantized two-dimensional discrete cosine transform unit 16. The two-dimensional discrete cosine inverse transform unit 16 inversely quantizes the data from the inverse quantization unit 15 and supplies the data to the adder 17. The adder 17 combines the value of the data from the inverse two-dimensional discrete cosine transform unit 16 with the value of the predicted image signal 20 from the predicted image generation unit 8 to generate a reference image signal 25, and stores the reference image signal 25 in the frame memory 7. Store.

The block significance judgment section 2 calculates a block significance 26 representing the significance of each block with reference to the input image signal 5. The map generation unit 3 regards the block significance 26 from the block significance determination unit 2 as the priority for performing the forced refresh process, and generates a refresh map signal 27 representing the refresh process priority for each block of the input image signal 5. The signal is generated and supplied to the adaptive refresh signal generator 4. The adaptive refresh signal generator 4 specifies a block to be forcibly refreshed based on a permissible block number signal 28 indicating the permissible number of blocks capable of performing the forcible refresh processing and a refresh map signal 27 in one frame. 22 is given to the mode control unit 10 of the video encoding unit 1. The allowable block number signal 28 is provided as an external signal according to the data transmission environment. Increasing the number of refresh blocks improves the error concealment processing capability but degrades the coding efficiency. The allowable block number signal 28 is a signal indicating an amount uniquely determined by a trade-off between the error concealment processing capability and the coding efficiency according to the transmission rate.

FIG. 2 shows the block significance judgment section 2. The block significance determination unit 2 includes a block feature value calculation unit 29 and a significance generation unit 30. The block feature value calculator 29 receives the input image signal 5. The significance generator 30 is connected to the block feature calculator 29 and the map generator 3. The block feature value calculator 29 calculates a block feature value 31 representing the importance in improving the image quality for each block with respect to the input image signal 5. The degree of importance in image improvement refers to a degree including visually important information such as a contour portion or a degree that image quality deterioration is subjectively noticeable. In the first embodiment, the variance of the luminance component and the chrominance component representing the degree of fluctuation of the signal in the block and the weighted sum thereof or the edge component obtained by a high-pass filter or another edge extraction filter are used as the block feature amount. Is used. A portion or a contour portion having a large variation in the block is visually important information and greatly affects the subjective image quality. Other block features include the distance between the maximum value and the minimum value of the signal in the block, the difference between the power of the luminance signal in the block and the power of the luminance signal in the adjacent block, the power of the chrominance signal in the block and the power in the adjacent block. Difference with the power of the color difference signal,
A value obtained by weighting and adding the coefficient signal after the frequency conversion based on the visual model, saturation (absolute value of the color difference signal), and the like can be considered.

The significance generation unit 30 includes a block feature quantity 31
And a predetermined threshold value, and the block significance 26
Occurs. The significance generation unit 30 has one or a plurality of thresholds, and determines the block significance 26 by comparing the block feature value 31 with the threshold. According to the refresh priority indicated by the refresh map signal 27, the adaptive refresh signal generator 4 shortens the refresh cycle for high-priority blocks and lengthens the refresh cycle for low-priority blocks. And implements an adaptive refresh process according to.

Next, a second embodiment of the present invention will be described in detail with reference to the drawings. In the second embodiment of the present invention, the same components as those in the first embodiment of the present invention are denoted by the same reference numerals. In the second embodiment of the present invention, in addition to the input image signal 5, the block significance determination unit 2 uses the moving image encoding unit 1 and the block information generation unit 32 encodes each block in the input image signal 5. The block significance level 26 is generated with reference to the generated block information 33.

FIG. 4 shows a block significance determining section 2 according to the second embodiment of the present invention. The block significance determination 2 includes a block feature amount calculation unit 29 and a significance generation unit 30
And a visual deterioration amount calculation unit 34, a significance generation unit 35, and a block significance integration unit 36. Visual deterioration amount calculation unit 3
4 is connected to the block information generation unit 32 of the video encoding unit 1. The visual deterioration amount calculation unit 34 receives the block information 33 from the block information generation unit 32. The significance generation unit 35 is connected to the visual deterioration amount calculation unit 34. The block significance integration unit 36 is connected to the significance generation unit 30, the significance generation unit 35, and the map generation unit 3. The significance generation unit 30 compares the block feature quantity 31 with one or more predetermined thresholds, and calculates the block significance 37
Occurs. The visual deterioration amount calculation unit 34 calculates the block information 3
3, a visual degradation amount 38 representing the degree of visual image degradation when a prediction error signal is lost in block units.
Is calculated. The significance generation unit 35 compares the visual degradation amount 38 with a predetermined threshold to generate a block significance 39. The significance generation unit 35 has one or a plurality of thresholds, compares the visual deterioration amount 38 with the thresholds, and determines and sets the block significance 26.

As the block information 33, in addition to the first error power with respect to the block located at the same position in the reference frame used in the above-described conventional apparatus, the block information 33 is located at the same position on the motion-compensated predicted image. There is a second error power with respect to a certain block and a motion amount of the block. Visual degradation 38
Is used, in addition to the first or second error power or the motion amount, the product of the motion amount and the error power. The error power represents the energy of a signal lost when an error occurs. In addition, even if the blocks have the same error power, the motion between frames may be more likely to propagate image quality degradation to a wider area. Therefore, it is conceivable to weight the error power with the motion amount.

The block significance integration unit 36 combines the block significance 37 from the significance generation unit 30 and the block significance 39 from the significance generation unit 35 to generate a block significance 2
6 is given to the map generator 3. As the integrated value of the significance in the block significance integration unit 36, the significance is 2
If it is a value, a logical sum or logical product is used,
When the significance is multi-valued, an addition value or a multiplication value is used.

In the conventional apparatus, the error power with respect to the block located at the same position on the reference frame is used as the evaluation value of the image quality deterioration in the error concealment processing. I was giving it. On the other hand, according to the second embodiment of the present invention, it is possible to preferentially refresh those blocks that have an effect on the image quality improvement at the time of error concealment among the refresh candidate blocks. The block affecting the image quality improvement is a block including visually important information such as the contour information described above.

On the other hand, as a human visual characteristic, even if a block has the same level of error power, the sensitivity to image quality degradation increases if the average luminance of the block is medium.
Furthermore, if the luminance change of the block is large,
Sensitivity is known to further increase. The block feature value calculator 29 in FIG. 4 calculates sensitivity information based on human visual characteristics for image quality degradation from the average and variance of signals in the block. The block significance 37 output from the significance generation unit 30 using the sensitivity information as the block feature amount 31 is converted to a significance generation unit 35 based on the visual deterioration amount 38.
By synthesizing with the block significance level 39 output from the above, it is possible to preferentially refresh a visually noticeable block among the blocks with remarkable image quality deterioration. Thereby, the second embodiment of the present invention can further improve the image quality at the time of error concealment processing as compared with the conventional device.

Also, ISO / IEC JTC1 / SC2
9 / WG11 Publication @ Final Comm
itte Draft 14496-2 (MPEG
-4Visual)}, as a method of transmitting data in a transmission environment where error tolerance is required, a method of independently transmitting a motion amount and transform coefficient information used for an inter-frame predictive coding method is described. By using this transmission method, when data loss occurs in the transform coefficient information due to an error, a predicted image is generated by a motion compensation process from a reference frame and a motion amount as shown in FIG. 5, and a corresponding block is copied. Advanced error concealment processing becomes possible. However, in the execution of the forced refresh of the conventional apparatus, since the error power of the block at the same position on the reference frame is used as the evaluation value of the forced refresh, it is possible to effectively select a block having a remarkable image quality deterioration. Can not. In this case, the forced refresh process using the above-described second error power, that is, the power of the prediction error signal as the evaluation value is effective. The second embodiment of the present invention can realize effective coding control corresponding to the second embodiment.

Next, a third embodiment of the present invention will be described in detail with reference to the drawings. In the third embodiment of the present invention, the same components as those in the first embodiment of the present invention are denoted by the same reference numerals. As shown in FIG.
The moving picture coding apparatus according to the embodiment is the same as the moving picture coding apparatus shown in FIG.
Is added. That is, the moving picture coding apparatus according to the third embodiment of the present invention includes a moving picture coding section 1, a block significance determination section 2, a map generation section 3, a refresh history determination section 40, and an adaptive refresh signal generation section 4
And The block significance determination unit 2 receives the input image signal 5. The map generation unit 3 is connected to the block significance determination unit 2. Refresh history determination unit 40
Are connected to the map generator 3. The adaptive refresh signal generator 4 is connected to the video encoder 1 and the refresh history determiner 40. The refresh history judging section 40 updates the history information regarding the refresh stored therein by referring to the refresh signal 22 from the adaptive refresh signal generating section 4 and the refresh map signal 27 from the map generating section 3, and updates this. The adaptive refresh signal generator 4 corrects the refresh map signal 27 to generate a corrected refresh map signal 41 and
Give to.

FIG. 7 shows the refresh history judging section 40. The refresh history determination unit 40 has a map history memory 42, a refresh signal history memory 43, and a map correction unit 44. The map history memory 42
It is connected to the map generator 3. Map history memory 4
2 is a refresh map signal 2 from the map generator 3
Receive 7. The refresh signal history memory 43 is connected to the adaptive refresh signal generator 4. Refresh signal history memory 43 receives refresh signal 22 from adaptive refresh signal generator 4. The map correction unit 44 is connected to the map history memory 42, the refresh signal history memory 43, and the adaptive refresh signal generation unit 4.

The map history memory 42 stores information relating to the accumulation of the forced refresh priority indicated by the refresh map signal 27 from the start of encoding and the current time when the block has a high refresh priority at a certain time. A map history 45 that represents information on the elapsed time up to is stored. In the conventional moving picture coding apparatus or the moving picture coding apparatus of the present invention, an allowable block number signal indicating the allowable block number is determined according to the data transmission environment, and blocks to be subjected to refresh processing are determined according to the number. decide. Therefore, there are blocks that are not actually refreshed even if the forced refresh priority is high.
When data loss occurs in a block having a high refresh priority at a certain time and a low priority thereafter, that is, when data loss occurs in an area where there is no motion between frames after a certain time, the refresh processing is executed at the time of movement. Otherwise, blocks with large image quality degradation will continue to be used as predicted images. Map history memory 42
When the refresh map signal 27 is input, the refresh history is cumulatively added to generate the map history 45, and at the same time, the map history 45 is equal to or greater than a certain threshold value. When the refresh priority is zero, the cumulative refresh priority shown in the map history 45 is increased by a predetermined value.
The map history memory 42 stores the refresh signal 22
Is referred to at the time of occurrence, and the accumulated refresh priority shown in the map history 45 for the refreshed block is reset to zero. Map correction unit 44
Refers to the map history 45 and raises the corresponding refresh priority in the refresh map signal 27 for a block whose accumulated refresh priority is larger than a certain threshold.

On the other hand, the refresh signal history memory 43
Stores a refresh signal history 46 which is information indicating the position of the block actually refreshed in the last several frames. Refresh signal history memory 4
3 indicates that the refresh signal history 46 is stored in the refresh signal 2
2 is output and the refresh signal history 46 is output at the stage when the encoding of the next frame is performed.
The map correction unit 44 refers to the refresh signal history 46 and lowers the corresponding refresh priority in the refresh map signal 27 for the blocks that have been refreshed during the past several frames.

For example, in the conventional image coding method described in Japanese Patent Application Laid-Open No. 4-176291, when coding a block in an input image, either intra-frame coding or inter-frame predictive coding is selected. There is an encoding control method in which the interframe predictive encoding is continuously monitored for a certain period of time or more, and refreshing is forcibly performed in the next frame. On the other hand, in the third embodiment of the present invention, by referring to the history of the refresh map, of the blocks in which the inter-frame predictive coding is continuously selected, the block having the influence on the image quality degradation is prioritized. By performing refresh, it is possible to realize more effective coding control than the above-described conventional method. The second embodiment of the present invention
This embodiment may be combined with the third embodiment.

In the embodiment of the present invention, the forced refresh priority of the significant block selected by the block significance determination unit 2 is increased, and the area having an influence on the subjective image quality is preferentially refreshed, so that during the transmission, In this case, it is possible to suppress the deterioration of the image quality when data is lost. In the embodiment of the present invention, adaptive error concealment processing can be performed by the block significance determination unit 2 using a conventional method using information obtained at the time of motion detection. Furthermore, in the third embodiment of the present invention, the refresh history determination unit 40 refers to the history of the refresh map, and it is possible to prevent an area where image degradation is remarkable from continuing to be displayed without being refreshed.

[0046]

According to the present invention, while the number of blocks to be forcibly refreshed is limited, important information in an image such as contour information or a region which is visually noticeable or remains without being refreshed. By giving the forced refresh priority in consideration of blocks that affect image quality deterioration such as blocks that may be present, appropriate error concealment processing can be realized, so it is possible to improve an image in which missing data is restored when an error occurs it can.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a moving picture encoding device according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a block significance determination unit in the moving picture coding device of FIG. 1;

FIG. 3 is a block diagram illustrating a moving image encoding device according to a second embodiment of the present invention.

FIG. 4 is a block diagram illustrating a block significance determination unit in the moving picture encoding device of FIG. 3;

5 is a diagram for explaining advanced error concealment processing using a motion amount in the moving picture coding apparatus of FIG. 3;

FIG. 6 is a block diagram illustrating a moving image encoding device according to a third embodiment of the present invention.

FIG. 7 is a block diagram showing a refresh history determination unit in the moving picture encoding device of FIG. 6;

FIG. 8 is a block diagram showing a conventional moving picture encoding device.

FIG. 9 is a diagram for explaining an error concealment process of a conventional video encoding device.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 moving image encoding unit 2 block significance determination unit 3 map generation unit 4 adaptive refresh signal generation 5 input image signal 6 motion amount estimation unit 7 frame memory 8 predicted image generation unit 9 subtractor 10 mode control unit 11 block selection unit 12 2D discrete cosine transform unit 13 Quantization unit 14 Variable length coding unit 15 Inverse quantization unit 16 2D discrete cosine inverse transform unit 17 Adder 18 Reference image signal 19 Motion vector 20 Predicted image signal 21 Error signal 22 Refresh signal 23 Selection control signal 24 Output data 25 Reference image signal 26 Block significance 27 Refresh map signal 28 Allowable number of blocks signal 29 Block feature amount calculation unit 30 Significance generation unit 31 Block feature amount 32 Block information generation unit 33 Block information 34 Visual degradation amount Calculation unit 35 Significance generation unit 36 Block significance integration unit 37 Block significance 38 Visual degradation 39 Block significance 40 Refresh history determination unit 41 Modified refresh map signal 42 Map history memory 43 Refresh signal history memory 44 Map modification unit 45 Map history 46 Refresh signal history

Claims (13)

[Claims] 1. A video coding apparatus for coding a continuous input image signal, wherein a block for determining a block significance based on a predetermined evaluation index for each block serving as a coding unit of the input image signal. Significance determination means; map generation means for generating a refresh map signal indicating the priority of refresh processing in each block based on the block significance; refresh priority indicated by the refresh map signal and a frame to be encoded. An adaptive refresh signal generating means for selecting a block to be processed by referring to the allowable number of blocks to be refreshed and generating a refresh signal specifying the block to be processed; and for a block specified by the refresh signal. Performs intra-frame coding and performs the refresh signal There video encoding apparatus characterized by having a moving image encoding means for performing selected intraframe coding or interframe predictive coding a suitably for not specified block. 2. The moving image encoding apparatus according to claim 1, wherein the block significance determining unit calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. A moving picture coding apparatus comprising: a quantity calculation unit; and a significance generation unit configured to compare the block feature value with one or more thresholds to generate a block significance of each block. 3. The video encoding apparatus according to claim 1, wherein the block significance determination unit calculates a block feature amount that is a quantity representing a variance of an intra-block signal; A moving image encoding apparatus comprising: a significance generation unit configured to compare a block feature amount with one or more thresholds to generate a block significance of each block. 4. The moving picture coding apparatus according to claim 1, wherein the block significance determining means calculates a block feature quantity which is an amount representing a power of a signal obtained by passing a signal in the block through a band-pass filter. A moving picture coding apparatus comprising: a block feature value calculation unit; and a significance generation unit configured to compare the block feature value with one or more thresholds to generate a block significance value of each block. 5. A moving picture coding apparatus for coding a continuous input image signal, wherein a block significance is determined based on block information and an evaluation index predetermined for each block as a coding unit of the input image signal. , A map generating means for generating a refresh map signal representing the priority of refresh processing in each block based on the block significance, and a refresh priority and coding indicated by the refresh map signal. An adaptive refresh signal generating means for selecting a block to be processed and generating a refresh signal specifying the block to be processed by referring to the allowable number of blocks to be refreshed in the frame to be processed; and The block representing the error power and the amount of motion between the frames to be generated. Generates information and provides it to the block significance determination means. Performs intra-frame coding for a block specified by the refresh signal, and performs intra-frame coding or A moving image encoding device that appropriately selects inter-frame prediction encoding. 6. The moving image encoding apparatus according to claim 5, wherein the block significance determining unit calculates a block characteristic amount that is a characteristic amount of a signal distribution and a visual characteristic of each block. Quantity calculating means, first significance generating means for comparing the block feature quantity with one or more thresholds to generate first block significance for each block, and a prediction error in block units from the block information. A visual deterioration amount calculating means for calculating a visual deterioration amount representing a degree of visual image quality deterioration when a signal loss occurs, and comparing the visual deterioration amount with one or a plurality of thresholds to obtain a second
Second significance generation means for generating the significance of the block, and block significance integration means for synthesizing the first significance and the second significance and giving the significance to the map generation means. A video encoding device characterized by the above-mentioned. 7. The moving image encoding apparatus according to claim 5, wherein the block significance determination unit calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. Quantity calculating means, first significance generating means for comparing the block feature quantity with one or a plurality of thresholds to generate first block significance of each block, and referencing blocks on the input image signal A visual deterioration amount calculating means for calculating a visual deterioration amount representing an error power with respect to a block located at the same position on the frame; and comparing the visual deterioration amount with one or a plurality of thresholds,
Second significance generation means for generating the significance of the block, and block significance integration means for synthesizing the first significance and the second significance and giving the significance to the map generation means. A video encoding device characterized by the above-mentioned. 8. The video encoding apparatus according to claim 5, wherein the block significance determination unit calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. Quantity calculating means, first significance generating means for comparing the block feature quantity with one or more thresholds to generate first block significance of each block, and blocks and frames on the input image signal A visual deterioration amount calculating means for calculating a visual deterioration amount representing an error power with respect to a block located at the same position on the predicted image obtained by the inter-prediction process; Second significance generation means for generating the block significance of: and the block significance given to the map generation means by combining the first block significance and the second block significance A moving picture coding apparatus comprising: integrating means. 9. The moving image encoding apparatus according to claim 5, wherein the block significance determining unit calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. Quantity calculating means, first significance generating means for comparing the block feature quantity with one or a plurality of thresholds to generate first block significance of each block, and referencing blocks on the input image signal Visual deterioration amount calculating means for calculating a visual deterioration amount representing an amount obtained by weighting the error power with respect to the block at the same position on the frame in accordance with the motion amount of the block; and the visual deterioration amount and one or more thresholds; Compare the second
Second significance generation means for generating the significance of the block, and block significance integration means for synthesizing the first significance and the second significance and giving the significance to the map generation means. A video encoding device characterized by the above-mentioned. 10. The moving image encoding apparatus according to claim 5, wherein the block significance determining unit calculates a block characteristic amount which is a characteristic amount of a signal distribution and a visual characteristic of each block. Quantity calculating means, first significance generating means for comparing the block feature quantity with one or more thresholds to generate first block significance of each block, and blocks and frames on the input image signal Visual deterioration amount calculation means for calculating a visual deterioration amount representing an amount obtained by weighting an error power with respect to a block at the same position on a predicted image obtained by the inter prediction process according to the amount of motion of the block; Second significance generating means for producing a second block significance by comparing the amount with one or more thresholds, and determining whether the first block significance and the second block are present. A moving image encoding apparatus comprising: a block significance integration means for synthesizing the significance and giving the combined significance to the map generation means. 11. The moving picture coding apparatus according to claim 5, wherein the block significance determination unit refers to information on a luminance change and a luminance level of a signal in the block and based on a human visual characteristic of the information. A block feature value calculation means for generating sensitivity information and calculating the sensitivity information as a block feature value; and generating a first block significance of each block by comparing the block feature value with one or more threshold values. A first significance generation unit; a visual deterioration amount calculation unit that calculates a visual deterioration amount representing a degree of visual image quality deterioration when a loss of a prediction error signal occurs in block units from the block information; By comparing the amount of degradation with one or more thresholds, a second
Second significance generation means for generating the significance of the block, and block significance integration means for synthesizing the first significance and the second significance and giving the significance to the map generation means. A video encoding device characterized by the above-mentioned. 12. A moving image encoding apparatus for encoding a continuous input image signal, wherein a block for determining a block significance based on a predetermined evaluation index for each block serving as a unit for encoding the input image signal. Significance determination means; map generation means for generating a refresh map signal representing the priority of refresh processing in each block based on the block significance; temporarily holding the refresh map signal from the map generation means and refreshing the refresh map signal. Refresh history determining means for correcting the value of the forced refresh priority indicated by the refresh map signal with reference to the history of the map signal and the refresh signal to generate a corrected refresh map signal; and refresh priority indicated by the corrected refresh map signal. Within the frame to be encoded An adaptive refresh signal generating means for selecting the processing target block by referring to the allowable number of blocks to be subjected to the refresh processing and generating the refresh signal specifying the processing target block; and for the block specified by the refresh signal. And moving picture encoding means for performing intra-frame encoding, and appropriately selecting intra-frame encoding or inter-frame predictive encoding for blocks not specified by the refresh signal. Video encoding device. 13. The moving picture coding apparatus according to claim 12, wherein the refresh history determination unit refers to the refresh map signal from the map generation unit and the refresh signal from the adaptive refresh signal generation unit. A map history memory for updating and storing the history of the refresh map from the start of the encoding process, a refresh signal history memory for storing the history of the refresh signal, a map history stored in the map history memory, and the refresh signal. A moving image encoding apparatus comprising: a map correction unit that corrects a forced refresh priority indicated by the refresh map signal from the map generation unit with reference to a refresh history stored in a history memory.