JP4259974B2 - Image data converter - Google Patents

Description

The present invention relates to an image data conversion apparatus, and is particularly suitable for use in converting an MPEG-encoded data stream into a low bit rate data stream.

  In recent years, MPEG (Moving Picture Experts Group) has been used in various apparatuses as a technique for compressing and encoding image information. By using such compression encoding technology, it is possible to express original video and audio with less information without degrading the quality of the image and audio.

  FIG. 9 shows an example of the basic configuration of an image reproducing apparatus that reproduces MPEG-encoded image information.

  In the figure, a VLC decoder 101 decodes a VLC (Variable Length Code) processed MPEG data stream and derives a quantization coefficient for each block (8 pixels × 8 frames). The inverse quantization processing unit 102 inversely quantizes the decoded quantization coefficient to derive a DCT coefficient for each block. The IDCT processing unit 103 performs inverse DCT (Discrete Cosine Group) processing on the DCT coefficients for each block, and derives image data of the block. The frame memory 104 uses image data (I picture, P picture) for one frame obtained by adding the reference image data from the motion compensation processing unit 105 to the image data output from the IDCT processing unit 103 as a reference image. Hold. The motion compensation processing unit 105 generates image data obtained by shifting the reference image data held in the frame memory 104 by a motion vector.

  Here, the quantization coefficients of one block subjected to the VLC decoding process are arranged as shown in FIG. 10 according to the scan order (zigzag scan, alternate scan) applied to the picture. In the figure, the quantized coefficient at the position (0, 0) in the upper left is a quantized DCT coefficient indicating the DC component of the block, and the other quantized coefficients have horizontal and vertical frequencies, respectively. A DCT coefficient of an AC component having a predetermined value is quantized.

  For one block of image data, the quantization coefficients shown in FIG. 10 are inversely quantized (each quantization coefficient is multiplied by a quantization coefficient value) to obtain DCT coefficients for each frequency component, and the obtained DCT coefficients are subjected to inverse DCT processing. Is obtained by Here, the DCT coefficient indicates the size of the transform base (basic pattern) corresponding to each frequency component. Therefore, the image data obtained by the inverse DCT process represents an image obtained by synthesizing the conversion bases of the respective frequency components with weighting corresponding to the DCT coefficients.

  By the way, when an MPEG stream is transmitted to a reproduction processing system, a process for converting the original MPEG stream into a stream having a lower bit rate may be executed. For example, as shown in FIGS. 11A and 11B, only three or six quantization coefficients are left from the low frequency side, and higher-order quantization coefficients are all set to “0”. . When the quantized coefficients thus converted are scanned, for example, in the zigzag scan order, all quantized coefficients after the predetermined order become “0”. By leaving the quantization coefficient until the start of the continuous “0” and adding an EOB (End of Block) after that, the data amount of the block can be considerably reduced. Thereby, the original MPEG stream can be converted into a stream having a lower bit rate.

  FIG. 12 shows a data amount reduction state when the process of leaving only the six quantization coefficients on the lower order side is executed. As described above, when an MPEG stream with reduced DCT coefficients is played back, an image is generated only with the lower-order DCT coefficients, so that a rough image as if viewed through a frosted glass is played back and displayed. .

Such stream conversion processing is disclosed in, for example, Patent Document 1 below.
JP 2000-32457 A

  By the way, in MPEG2 and MPEG4, two DCT processes of a frame DCT mode and a field DCT mode are prepared in order to efficiently encode an interlaced image.

  FIG. 13 shows a processing method in the frame DCT mode and the field DCT mode.

  As shown in FIG. 5A, in the frame DCT mode, an area consisting of 16 pixels × 16 lines is divided into four as it is to cut out a block, and this block is subjected to DCT processing. This DCT processing is also applied in MPEG1.

  On the other hand, in the field DCT mode, as shown in FIG. 13 (b), an area composed of 16 pixels × 16 lines is divided into left and right parts, and then odd lines and even lines of each divided area are separated and taken out. And the block is subjected to DCT processing.

  When decoding MPEG data to which any of the above DCT modes is applied, in the field DCT mode, inverse DCT processing is performed for each of the top field (odd frame group) and the bottom field (even frame group). Therefore, even if the inverse DCT conversion process is performed on the stream data that retains only the low-order DCT coefficients as described above, the top-field and bottom-field images do not adversely affect each other. However, in the frame DCT mode, the inverse DCT process is performed in a state where the top field and the bottom field are mixed, so that in the case of an image having a significant difference between the top field and the bottom field, The bottom field images will adversely affect each other.

  In other words, since the low-order DCT coefficient has information that averages the block, when the inverse DCT process is performed using only the low-order DCT coefficient on the block to which the frame DCT mode is applied, each pixel An image in which is mixed is generated. For this reason, for example, as shown in FIG. 14B, image data in which an image of the top field oozes out to the bottom field is generated. Further, when a frame to be decoded next refers to this frame, There may be a phenomenon in which a part of an object suddenly appears discontinuously in the background. This phenomenon is perceived as very conspicuous image degradation by human eyes.

Therefore, the present invention provides an image playback apparatus that can solve such problems and can display a playback image satisfactorily even during high-speed playback.

The present invention provides an image data conversion apparatus for converting a compression-encoded data stream into a low bit rate data stream, wherein a horizontal frequency component is low among frequency components of a block region of a frame DCT mode constituting one picture , And a reference means for determining whether or not an added value of absolute values of quantization coefficients or DCT coefficients corresponding to a specific frequency component having a high vertical frequency component exceeds a predetermined threshold; and the reference means exceeds the threshold. A block detection unit that detects the block region as a block region that causes image degradation when determined, and a DCT coefficient in the block region for the block region that the block detection unit detects that the image does not cause image degradation. Only the DCT coefficients from the lower order side to the predetermined order are left, and the block detecting means For the block area detected to cause image degradation, the data amount of the block so as to leave the DCT coefficients from the lower order to the higher order than the predetermined order among the DCT coefficients in the block area. And a data amount adjusting means for adjusting.

  The features of the present invention will become more apparent from the following description of embodiments.

However, the following embodiment is merely one embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the following embodiment. Absent.

  According to the present invention, among block areas constituting one picture, a block area that causes image degradation is left up to a DCT coefficient that is higher than the block area that is detected to cause no image degradation. In addition, since the data amount of the block is adjusted, in the reproduction system, it is possible to suppress the reproduction of the image in which the image of the top field oozes out to the bottom field. It is possible to suppress the image quality deterioration phenomenon that a part of an object appears suddenly and discontinuously.

  Whether the block area causes image degradation can be detected by monitoring the DCT mode of the block area and the value of the specific frequency component of the block area as described in claims 2 to 4. Here, the determination of the DCT mode can be performed with reference to the DCT type information inserted in the MPEG stream, and the value of the specific frequency component is determined by the quantization coefficient or DCT corresponding to the frequency component. Since it can be performed with reference to the coefficients, no complicated processing is required for these determinations and determinations. Therefore, according to the second to fourth aspects of the present invention, it is possible to smoothly detect a block that causes image degradation by simple processing.

  Further, as described in claim 5, if the quantization value in the requantization means can be adjusted, for example, requantization is performed with a quantization value larger than the quantization value applied to the original MPEG stream. As a result, the data amount of the block can be further reduced, and the bit rate of the MPEG stream can be more effectively reduced.

  In addition, as described in claim 6, the data amount of the block is adjusted so as to leave a high-order DCT coefficient in the vertical direction for the block area detected as causing image degradation by the block detection means. In this way, it is possible to effectively suppress image degradation on the reproduction side while suppressing the data amount of the block.

  Further, as in claim 7, if the problem block detection process is executed only for the I picture and / or P picture, or only for the intra block of the I picture and / or P picture, While simplifying the processing, it is possible to effectively suppress the occurrence of image degradation due to the problem block.

In addition, the effects of the invention described in the above claims will become more apparent from the following description of embodiments.

  Embodiments of the present invention will be described below with reference to the drawings.

  First, FIG. 1 shows a configuration of an image data conversion apparatus according to the embodiment.

  In the figure, a VLC decoder 201 decodes a VLC-processed MPEG data stream and derives a quantization coefficient for each block.

  Based on the DCT mode (frame DCT mode / field DCT mode) of the decoding target block and the quantization coefficient of the decoding target block, the VAD 202 determines whether the block is a block that causes image degradation. Note that the determination method will be described later.

  The adaptive DCT coefficient allocating unit 203 performs processing for converting an original MPEG stream into a stream having a lower bit rate. If the block is a block that does not cause image degradation, the DCT coefficient on the lower order side For example, when only 6 DCT coefficients are transmitted from the lower order side and the block is a block that causes image degradation, higher order DCT coefficients than the above, for example, all frequency components of the block The DCT coefficient is transmitted. The DCT coefficients transmitted in each case are not limited to those exemplified here, and can be changed as appropriate.

  FIG. 2 shows a specific example of DCT coefficient allocation processing in adaptive DCT coefficient allocation section 203. In the figure, there are shown cases in which 6, 9, and 12 DCT coefficients are left from the original MPEG stream.

  When the VLC data (original MPEG stream) in FIG. 6A is VLC decoded, a quantized coefficient sequence shown in FIG. Here, when six DCT coefficients (quantization coefficients) are left from the low order side, the uppermost VLC data in FIG. That is, the VCL data corresponding to the six DCT coefficients from the lower order side is left, and EOB (End of Block) is added thereafter. Similarly, when nine DCT coefficients (quantization coefficients) are left from the lower order side, the VCL data corresponding to the nine DCT coefficients is left from the lower order side, and thereafter, EOB (End of Block) ) Is added to generate VLC data in the middle of FIG.

  When twelve DCT coefficients (quantization coefficients) are left from the lower order side, as shown in FIG. 5B, the separation position of the twelve DCT coefficients from the lower order side is one VLC data. It is between “0” and “−1” in the quantization coefficient “0, 0, 0, −1” corresponding to the unit “000001”. In this case, since all of the quantization coefficients to be left are “0”, it is not necessary to leave them, and an EOB is added after the previous VLC data unit “1, 0”. That's fine. In this case, the generated VLC data is as shown in the lowermost part of FIG. 5C, and is the same as when nine DCT coefficients are left from the lower order side. In addition, when the DCT coefficient separation position is included in the quantization coefficient corresponding to one VLC data unit as described above, the VLC data unit is left as it is, in addition to the processing as described above, and then the EOB. May be added.

  FIG. 3 shows a specific example of DCT coefficient allocation processing for a block (problem block) determined to cause image degradation by the VAD 202 and a block (normal block) determined not to cause image degradation. In the figure, only the 6 low-order DCT coefficients from the original MPEG stream are left for the normal block, and all the DCT coefficients of the original MPEG stream are left as they are for the problem block.

  Note that the number of DCT coefficients to be left in the normal block is not limited to the example of FIG. 3, and may be 9 or 12 as shown in FIG. In addition to leaving all DCT coefficients for the problem block, an appropriate number (however, more than the number for the normal block) may be left. Thereby, the data amount after conversion can be further reduced, and the bit rate of the MPEG stream can be further reduced.

  Next, a problem block determination process in the VAD 202 will be described.

  First, FIG. 4 shows the layer structure of an MPEG stream according to the MPEG2 standard.

  As shown in the figure, the MPEG stream is composed of a sequence layer, a GOP (Group of Picture) layer, a picture layer, a slice layer, a macroblock layer, and a block layer. In the figure, I, P, and B in the GOP layer indicate I picture, P picture, and B picture, respectively, MB in the picture layer indicates a macroblock of 16 pixels × 16 lines, and BL in the macroblock layer is 8 A block of pixels × 8 lines is shown.

  The information for identifying the DCT mode is added as “DCT type” in the macroblock layer, and the DCT mode can be set for each macroblock. In other words, the DCT mode set for the macroblock is commonly applied to the four blocks constituting one macroblock.

  The VAD 110 extracts a DCT mode set for each macroblock from the macroblock layer, and recognizes the extracted DCT mode as a DCT mode of four blocks constituting the macroblock. When the recognized DCT mode is the frame DCT mode, the quantization coefficients of the four blocks constituting the macroblock are respectively referred to as follows, and each block corresponds to a block that causes image degradation. Is determined.

  FIG. 5 shows quantization coefficients that are referred to when determining whether or not to cause image degradation.

  In the present embodiment, as shown in the figure, the quantization coefficient at the position (0, 7) (1, 7) is referred to. A block that causes image degradation when the addition value (addition of absolute values) of these two quantization coefficients is equal to or greater than the threshold value S1, and a block that does not cause image degradation when the addition value is less than the threshold value S1. To do. The threshold value S1 is set to reflect experimental verification results. For example, as shown in FIG. 3, when the quantization coefficient of the frequency component at the position (0, 7) (1, 7) is to be determined, the threshold value S1 is set to S1 = 1.

  FIG. 6 shows a determination example in such a case.

  When the quantization coefficient of the decoding target block is as shown in FIG. 5A, the added value of the quantization coefficient at the position (0, 7) (1, 7) is “0”. The block is determined as a block that does not cause image degradation. On the other hand, when the quantization coefficient of the decoding target block is as shown in FIG. 5B, the added value of the quantization coefficient at the position (0, 7) (1, 7) is “3”. Therefore, this block is determined as a block that causes image degradation.

  By the way, as described above, the quantization coefficient at the position (0, 7) (1, 7) is set as the determination target because the DCT coefficient on the lower order side for an image having a large difference between the top field and the bottom field. This is because image degradation is likely to occur when the inverse DCT process is performed alone. That is, as shown in FIG. 14, since the top field and the bottom field are shifted by one line in the vertical direction, the image difference between the top field and the bottom field is reflected in the frequency component in the vertical direction. That is, the intensity of the image difference between the top field and the bottom field can be detected by the vertical frequency component. Specifically, the higher the difference between the two images, the higher the quantization coefficient on the higher-order side of the vertical frequency component. growing. Therefore, if the value of the quantization coefficient on the higher order side of the vertical frequency component is large, image degradation is likely to occur when the block is reproduced at high speed only with the lower order side DCT coefficient.

  In the above, paying attention to this point, the quantization coefficient at the position (0, 7) (1, 7) is used as an image quality degradation determination target. However, the quantization coefficient used for the determination target is not limited to this, and may be appropriately set according to the experimental verification result.

  For example, since the quantization coefficient at the position (0, 7) most directly indicates the intensity of the image difference between the top field and the bottom field, only the quantization coefficient is used to detect image deterioration. Even if this is done, it can be predicted that accurate determination results can be obtained. Alternatively, the quantization coefficient to be determined is set to the quantum at the position of (0, 1) to (0, 7), (1, 1) to (1, 7), (2, 1) to (2, 7). You may make it extend to a conversion factor. In this case, since the quantization coefficient on the lower side in the vertical direction in which the image difference between the top field and the bottom field is relatively gentle is included in the determination target, these values are added as they are (addition of absolute values) to obtain the threshold value. In the case of comparison, it is necessary to finely adjust the threshold value so that the detection accuracy of image degradation can be improved. At this time, weighting may be set at the time of addition so that higher-order quantization coefficients are emphasized than lower-order quantization coefficients.

  If a low-order quantization coefficient in the vertical direction and a high-order quantization coefficient in the horizontal direction are included in the determination target, an error component increases in the determination of image degradation, and there is a concern that the detection accuracy may decrease. For this reason, it is usually preferable to determine a frequency component having a low horizontal frequency component and a high vertical frequency component among the frequency components of the block. Specifically, reference is made in the determination within the range of (0, 4) to (0, 7), (1, 4) to (1, 7), (2, 4) to (2, 7). It is good to set the quantization coefficient. Also in this case, it is preferable to add each weighted coefficient with appropriate weighting.

  FIG. 7 shows a flowchart of data conversion processing executed for one picture.

  When the data conversion process for the picture is started, first, the DCT type of the head block is referred to (S101), and it is determined whether the DCT mode of the head block is the frame DCT mode (S102). If the determination result is NO, the process proceeds to S106, and conversion processing for the normal block, that is, processing for leaving only the low-order DCT coefficients (for example, six DCT coefficients from the low-order side) is executed.

  On the other hand, if YES in S102, the quantization coefficient of the specific frequency component in the block, for example, the quantization coefficient at the position of (0, 7) (1, 7) is added (S104), and this addition value is the threshold value. Whether it is S1 or more is determined (S104). If the determination result is NO, the process proceeds to S106, and the process of leaving only the low-order DCT coefficient is executed as described above. On the other hand, if the determination result in S104 is YES, the block is determined as a block that causes image degradation, and processing for leaving the higher-order DCT coefficients (for example, all DCT coefficients) is executed (S105).

  In this way, when the process of leaving the DCT coefficient for the normal block or the problem block is executed, the process of adding EOB is then executed (S107). Thereby, the data conversion process for the head block ends. When the process for the first block is completed, the process returns to S101, and the same process as described above is executed for the next block. The processing of S101 to S107 is repeated until the processing is executed for all the blocks included in the picture. When the process for all the blocks is completed (S108), the process for the picture is completed, and the process proceeds to the process for the next picture.

  Note that the processing shown in FIG. 7 is similarly applied to any of the I picture, P picture, and B picture. At this time, the threshold value S1 in S104 may be changed as appropriate for each picture type. Since the B picture is not referred to by other pictures, even if the B picture includes a problem block, the influence of image degradation due to this is relatively small. Therefore, the process shown in FIG. 7 may be performed only for I pictures and P pictures. Furthermore, only intra blocks of I and P pictures may be targeted. As a result, it is possible to effectively suppress the occurrence of image degradation on the reproduction side while simplifying the processing.

  As described above, according to the present embodiment, it is determined whether the block is a block that causes image degradation based on the DCT type and the quantization coefficient of the specific frequency component, and is left at the time of data conversion according to the determination result. Since the number of DCT coefficients is changed as appropriate, even when image reproduction is performed using the converted MPEG stream, the image quality of the reproduced image is not deteriorated, and therefore the bit rate is effectively reduced. Image quality deterioration can be suppressed at the same time.

  Needless to say, the present invention is not limited to such an embodiment, and various other modifications are possible.

  For example, in the above-described embodiment, the presence / absence of image degradation is determined by referring to the quantization coefficient of the specific frequency component. However, the DCT coefficient obtained by inverse quantization of the quantization coefficient of the specific frequency component is referred to. Thus, the presence or absence of image degradation may be determined. Here, when adding the DCT coefficient obtained by dequantizing the quantization coefficient and comparing it with a threshold value to determine the presence / absence of image degradation, naturally, the threshold value matches the magnitude of the DCT coefficient. It is necessary to make adjustments (for example, adjustment by multiplying the above S1 by a quantized value).

  Further, the configuration of the image data converter can be changed as shown in FIG.

  In the figure, 211 is an inverse quantization unit that inversely quantizes the quantization coefficient received from the VLC decoder 201, and 212 is a quantization coefficient according to the detection result from the VAD 202 (whether the block is a normal block or a problem block). For example, an adaptive DCT allocating unit 213 that executes a process of leaving 6 quantized coefficients on the low-order side in the case of a normal block and leaving all quantized coefficients as is in the case of a problem block, A requantization unit that requantizes the quantized coefficient received from the adaptive DCT allocating unit 212 to derive a quantized coefficient, and 214 performs VLC encoding on the requantized quantized coefficient and performs an EOB addition process It is a VLC encoder.

  Here, the quantization value in the requantization unit 213 can be set as appropriate according to the state of the reproduction system for reproducing the MPEG stream, the state of the communication system for transmitting the MPEG stream to the reproduction system, and the like. At this time, if the quantization value at the time of re-quantization is set larger than the quantization value of the original stream, the bit rate of the MPEG stream after conversion can be further reduced.

  Note that the quantization value set at the time of requantization is included in the SH (sheath header or macroblock layer) of the sequence layer shown in FIG. That is, the SH or macroblock layer quantization value in the original stream is rewritten to the quantization value set at the time of requantization.

  Further, when the target block is a block that causes image degradation, for example, as shown in FIG. 15, adaptively so as to leave a high-order DCT coefficient (a DCT coefficient with hatching in the figure) in the vertical direction. The DCT coefficient assignment units 203 and 212 may perform DCT coefficient assignment processing. As a result, it is possible to simultaneously suppress deterioration in the quality of the reproduced image while effectively reducing the bit rate.

In addition, the embodiment of the present invention can be variously modified as appropriate within the scope of the technical idea shown in the claims.

The figure which shows the structure of the image data converter which concerns on embodiment The figure which shows the specific example of the allocation process of a DCT coefficient The figure which shows the allocation state of the DCT coefficient with respect to a problem block and a normal block Diagram showing the structure of an MPEG stream The figure which shows the quantization coefficient referred in the case of image quality degradation block determination The figure which shows the example of determination of an image degradation block The figure which shows the image data conversion processing flow which concerns on embodiment The figure which shows the structure of the image data converter which concerns on other embodiment. The figure which shows the basic structural example of an image reproduction apparatus Diagram showing the arrangement of frequency components The figure which shows the DCT coefficient which remains at the time of data conversion The figure which shows the conventional image data conversion processing flow The figure explaining frame DCT and field DCT The figure explaining the image degradation phenomenon which arises in reproduction | regeneration using only a low-order side DCT coefficient The figure which shows the example of a change of the DCT coefficient left in the case of data conversion

Explanation of symbols

202 VAD
203 Adaptive DCT coefficient allocation unit 212 Adaptive DCT coefficient allocation unit 213 Requantization unit

Claims (1)

In an image data converter for converting a compression-encoded data stream into a low bit rate data stream,
The sum of absolute values of quantization coefficients or DCT coefficients corresponding to a specific frequency component having a low horizontal frequency component and a high vertical frequency component among the frequency components in the block region of the frame DCT mode constituting one picture is predetermined. A reference means for determining whether or not a threshold of
A block detection unit that detects the block region as a block region that causes image degradation when the reference unit determines that the threshold value has been exceeded;
For the block area detected by the block detection means as not causing image degradation, only the DCT coefficients from the lower order to the predetermined order are left among the DCT coefficients in the block area, and the block detection means For the block area detected to cause deterioration, the data amount of the block is set so as to leave the DCT coefficient from the lower order to the higher order than the predetermined order among the DCT coefficients in the block area. An image data conversion apparatus comprising: a data amount adjustment unit for adjustment.

Images