JP2005135249A - Image region dividing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a region dividing device capable of forming divided areas with high speed and accuracy by using information obtained through the partial decoding of compression-encoded data. <P>SOLUTION: A partial decoding part 1 partially decodes the compression-encoded data. First to third area dividing process parts 2-4 are hierarchized. The first area dividing process part 2 comprises a conversion coding information determining part 2-1, a coding mode information determining part 2-2, and a movement prediction information determining part 2-3 and provides rough area division. The second area dividing process part 3 comprises a movement prediction error information determining part 3-1 and subdivides the result of division by the first area division process part 2. The third area division process part 4 comprises an edge detection part 4-1 and subdivides the result of division by the second area division process part 3. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image area dividing device, and more particularly, to an area dividing device capable of forming an area divided at high speed and with high accuracy using information obtained by partially decoding compression-encoded data. Is.

Conventionally, methods based on Canny edge detection and a snake that is an active contour model are known as image segmentation techniques. Since a video has a very wide signal band and requires a huge amount of data, transmission and recording are generally performed in the form of compression-encoded data. When the conventional area division technique is applied to this video, the compression encoded data is once decoded to the pixel area, and an area divided by the pixel area is formed.
J. Canny “A Computational Approach to Edge Detection” IEEE Trans. On Pattern Analysis and Machine Intelligence 8 (6), 1986.

  However, in the conventional area division technique, the compression-encoded data is decoded in addition to the original area division process in order to form a divided area after decoding the compression-encoded data to the pixel area by a decoding process with a large processing load. There is a problem that a large amount of calculation cost is required. There is also a problem that the processing speed is slow.

  The snake-based method, which is an active contour model, excels in segmentation accuracy, but requires an iterative calculation with a large amount of calculation, and the segmentation accuracy tends to depend on the initial shape and parameter settings. There is a problem.

  An object of the present invention is to solve the above-described problems of the prior art and form an area that can be divided at high speed and with high accuracy using information obtained by partially decoding compression-encoded data. It is to provide a dividing device.

  In order to solve the above-described problem, the present invention provides a partial decoding unit that partially decodes compression-encoded data in a video region dividing device that divides a compression-encoded video into a plurality of regions, A plurality of area division processing means including at least one area division processing means for forming a divided area using information partially decoded by the decoding means, and forming areas divided at different resolutions. There is a first feature.

  Further, the present invention has a second feature in that the plurality of area division processing means are hierarchized from one having a low resolution of area division to one having a high resolution, and sequentially forming divided areas according to the hierarchy. is there.

  Further, the present invention has a third feature in that each of the plurality of area division processing means forms a divided area by using a division result of the upper layer area division processing means.

  In addition, the present invention has a fourth feature in that the plurality of area division processing means include area division processing means hierarchized according to an encoding information allocation unit.

  In addition, the present invention has a fifth feature in that the plurality of area division processing means includes area division processing means for forming a divided area using coding information for each encoding information allocation unit. .

  Further, the present invention is characterized in that the plurality of area division processing means include a low resolution area division processing means for forming an area divided by using transform coding information, coding mode information, and motion prediction information. There are 6 features.

  Further, according to the present invention, the low resolution region division processing unit determines whether or not the difference of the color signal DC component between the blocks is less than a threshold from the transform coding information about the color difference signal, and determines that the difference is less than the threshold. There is a seventh feature in that it has means for forming divided regions by integrating the blocks.

  Further, according to the present invention, the low resolution region segmentation processing means extracts the mode encoding mode information within a predetermined time from the encoding mode information for each position, and uses the identity of the mode encoding mode. There is an eighth feature in that it has means for forming divided regions.

  The ninth aspect of the present invention is that the low-resolution area dividing processing means includes means for forming a divided area by using the similarity of the direction and length of the motion vector obtained from the motion prediction information. There are features.

  In addition, the present invention has a tenth feature in that the plurality of area division processing means include area division processing means for forming an area divided using transform coding information.

  In the present invention, the region division processing means for forming a region divided using the transform coding information may determine whether the difference in luminance signal DC component between blocks from the transform coding information for the luminance signal is equal to or less than a threshold value. There is an eleventh feature in that it has means for forming divided areas by integrating blocks determined as having a difference equal to or less than a threshold.

  Further, according to the present invention, an area division processing unit that forms an area divided using the transform coding information estimates the presence / absence and direction of an edge of the luminance signal from the AC component of the transform coding information for the luminance signal. There is a twelfth feature in that it has an edge estimation means and forms a divided region using this estimation result.

  Further, according to the present invention, the edge estimation means estimates the presence / absence and direction of an edge of a luminance signal using the AC component partial absolute value sums in the horizontal direction and the vertical direction of the transform coding information for the luminance signal. There is a thirteenth feature.

  Further, according to the present invention, the edge estimator uses a comparison between the AC component partial absolute value sum and a threshold value, a magnitude relationship between the AC component partial absolute value sums in the horizontal direction and the vertical direction, or a ratio between the two. There is a fourteenth feature in that the presence / absence and direction of the edge are estimated.

  In addition, the present invention has a fifteenth feature in that the area division processing means for forming an area divided using the transform coding information forms an area divided at a resolution equal to or greater than a block unit.

  In addition, the present invention has a sixteenth feature in that the plurality of area division processing means includes area division processing means for forming a divided area using pixel information of a luminance signal and a color difference signal.

  Further, according to the present invention, an area division processing unit that forms an area divided using pixel information of the luminance signal and the color difference signal individually detects edges of the luminance signal and the color difference signal and uses the detected edges. Thus, there is a seventeenth feature in that a divided region is formed.

  In the present invention, divided areas are formed using information obtained by partially decoding compression-encoded data, and areas divided at different resolutions are formed, so that the amount of calculation is reduced. Thus, the processing cost can be greatly reduced, and the region segmentation result with an appropriate resolution can be appropriately selected in consideration of the required processing speed and resolution.

  In addition, by layering a plurality of area division processing means and using the upper division results in order to form divided areas, it is possible to form divided areas at high speed and efficiently. Furthermore, by performing edge estimation using the encoded information, it is possible to form a region divided with a resolution equal to or higher than the encoding unit despite using the encoded information.

  Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a video area dividing apparatus according to the present invention. In the following, an example in which segmentation processing is applied to a moving image encoded with the MPEG-1 video (ISO / IEC11172-2), which is an international standard, will be described. However, the present invention is limited to this encoding method. In addition, the present invention can be applied to other encoding methods.

  In FIG. 1, video compression-encoded data is given as input of the entire system. The compressed encoded data is input to the partial decoding unit 1. The partial decoding unit 1 partially decodes the compression-encoded data, and receives transform coding information (DCT coefficient), motion prediction (motion vector information) and coding mode information for each macroblock for the luminance signal and the color difference signal. Generate.

  Note that the transform coding information (DCT coefficient) here is not information obtained by transform coding an error between frames as in the P and B pictures of the MPEG system, but the frame itself as in the I picture. It means the information that is generated as a result. Therefore, in the case of P and B pictures, the information is reconstructed into transform coding information as used herein to be processed, or motion prediction (motion vector information) and coding mode for each macroblock in the frame. It is necessary to consider that only area division processing using information processing is performed.

  The first region division processing unit 2 includes a transform coding information determination unit 2-1, a coding mode information determination unit 2-2, and a motion prediction information determination unit 2-3, and uses information about color difference signals. And rough segmentation.

  The transform coding information determination unit 2-1 examines the difference in DC components of the transform coding information for the color difference signal, and forms roughly divided regions by integrating blocks whose difference is equal to or less than a threshold value.

  The encoding mode information determination unit 2-2 first accumulates the encoding mode information at the same position in frames before and after the time including the target frame, and sets an encoding mode having a high appearance frequency as a representative value. Next, a region divided more finely is formed by performing region division processing for each coding mode representative value on the division result of the transform coding information determination unit 2-1.

  The motion prediction information determination unit 2-3 examines the similarity of the direction and length of the motion prediction information of the macroblock in the divided area by the encoding mode information determination unit 2-2, and encodes the coding mode based on the similarity. By performing region division on the division result of the information determination unit 2-2, a more finely divided region is formed.

  In the MPEG system, since motion prediction (motion vector information) and coding mode information are information in units of macroblocks (16 × 16 pixels), areas divided by the first area division processing unit 2 with accuracy in units of macroblocks Is formed.

  The division result of the first area division processing unit 2 is output as the first area division result of the present apparatus. When a finer division result is required, the first division result is given to the second area division processing unit 3.

  In the case of a P picture or a B picture, as described above, the process in the transform coding information determination unit 2-1 is not performed, and the coding mode information determination unit 2-2 or the motion prediction information determination unit 2- It suffices to output the region divided by the processing in 3 as the division result.

  The second area division processing unit 3 includes a transform coding information determination unit 3-1, and forms a further finely divided area by using information about the first division result and the luminance signal.

  The transform coding information determination unit 3-1 examines the difference in DC component of the transform coding information for the luminance signal, and separates blocks whose DC component difference is equal to or less than a threshold value from the first division result. Then, a further finely divided region is formed. In the MPEG system, the transform coding information is information in units of blocks (8 × 8 pixels), so that an area divided with block unit accuracy is formed.

  Further, the edge direction for each block is estimated from the AC component of the transform coding information for the luminance signal, and a region finely divided into block units or more is formed based on the estimated edge direction.

  The division result by the second area division processing unit 3 is output as the second division result of the present apparatus. Further, when a finer division result is required, the second division result is given to the third area division processing unit 4.

  Hereinafter, the estimation of the edge direction in the second area division processing unit 3 and the area division based thereon will be described with specific examples. In order to estimate the edge direction in the block, first, the absolute value sum H of the AC component of the 0th row of the block and the 0th column of the 8 × 8 DCT coefficient (transform coding information) according to the following equations (1) and (2): The absolute value sum V of the AC components is obtained and compared with a threshold value.

Here, d _{j, i} represents a DCT coefficient of j rows and i columns.

  As a result of the comparison with the threshold value, when the absolute value sum H of the AC component in the 0th row is equal to or smaller than the threshold value and the absolute value sum V of the AC component in the 0th column is also equal to or smaller than the threshold value, there is no edge in the block portion. presume.

  When the absolute value sum H of the AC components in the 0th row is equal to or greater than the threshold value and the absolute value sum V of the AC components in the 0th column is equal to or less than the threshold value, it is estimated that a horizontal edge exists in the block portion. When the absolute value sum H of the AC components of the eyes is equal to or smaller than the threshold and the absolute value sum V of the AC components in the 0th column is equal to or larger than the threshold, it is estimated that a vertical edge exists in the block portion.

  When the absolute value sum H of the AC component in the 0th row is equal to or greater than the threshold and the absolute value sum V of the AC component in the 0th column is also equal to or greater than the threshold, the sign (positive / negative) of the low frequency element is checked and the absolute values H and V The edge direction is estimated by comparing the size of each other.

  FIG. 2 shows an example of the DCT coefficient with respect to the level of the stepped video image that rises to the right. FIG. 2A shows a case where the lower right portion is black and the upper left portion is white, and FIG. This is a case where the lower right portion is white and the upper left portion is black.

  FIG. 3 shows an example of the DCT coefficient with respect to the level of the step-down video image. FIG. 3A shows a case where the lower left portion is black and the upper right portion is white, and FIG. Is the case where the lower left part is white and the upper right part is black.

  As is clear from these figures, when there is an edge in the upward direction to the right, the low-frequency elements (0,1) and (1,0) of the DCT coefficient match and the edge in the downward direction is to the right Does not match the sign of the low frequency elements (0,1) and (1,0) of the DCT coefficient. In addition, depending on the magnitude relationship (or the ratio (H / V)) between the absolute value sum H of the AC components in the 0th row and the absolute value sum V of the AC components in the 0th column, the angle of rising to the right or falling to the right is 45. It can be estimated whether it is larger or smaller.

  Based on the above, as shown in FIG. 2, the low-frequency elements (0,1) and (1,0) have the same sign, and the absolute value sum H of the AC components in the 0th row is the AC in the 0th column. When the sum of absolute values of components is larger than V (ratio (H / V) is greater than 1), it can be estimated that the edge direction is in the range of 0 to 45 degrees.

  Further, the sign of the low frequency element (0,1) and (1,0) match, and the absolute value sum H of the AC component in the 0th row is smaller than the absolute value sum V of the AC component in the 0th column (ratio ( When H / V) is less than 1, it can be estimated that the edge direction is in the range of 45 to 90 degrees.

  Also, as shown in FIG. 3, the low frequency elements (0,1) and (1,0) do not match, and the absolute value sum H of the AC components in the 0th row is the absolute value of the AC components in the 0th column. When the sum is greater than V (ratio (H / V) is greater than 1), it can be estimated that the edge direction is in the range of 135 to 180 degrees.

  Further, the sign of the low frequency elements (0,1) and (1,0) do not match, and the absolute value sum H of the AC component in the 0th row is smaller than the absolute value sum V of the AC component in the 0th column (ratio) When (H / V) is smaller than 1, it can be estimated that the edge direction is in the range of 90 to 135 degrees.

  By smoothly reconnecting the region division results obtained so far with the edge direction estimated as described above, a region divided at a higher resolution than the encoded information unit (block) is formed. Can do.

  The third region division processing unit 4 includes a luminance signal and chrominance signal edge detection unit 4-1, and forms a region divided more finely in units of pixels using the second division result and pixel information.

  In luminance signal edge detection, for example, the edge of the luminance signal is detected using a Canny edge detector, and in color difference signal edge detection, the color difference signal is scaled to the same size as the luminance signal, and then the color difference signal is detected using the Canny edge detector. Detect edges of By superimposing these edge detection results on the second division result for optimization, it is possible to form a more finely divided region. The division result by the third area division processing unit 4 is output as the third division result of the present apparatus.

  In the above embodiment, since the first to third region division processing units 2 to 4 are hierarchically configured and the division results of each region division processing unit are output, respectively, the division result with the necessary resolution is appropriately selected. Can be obtained. Note that the third region division processing unit 4 needs information decoded to the pixel level, but the processing by the third region division processing unit 4 may be performed only when necessary, so that the processing burden on the entire apparatus is increased. Can be lightened. In addition, the third region division processing unit 4 is not required when the division result by the third region division processing unit 4 is not required.

  As is apparent from the above description, according to the present invention, a divided area is formed using information obtained by partially decoding compressed and encoded data of a video, thereby greatly reducing processing costs. Can be made. In addition, a plurality of area division processing units are hierarchically formed to form areas that are finely divided in stages, so that the division results at each stage can be obtained, so the processing speed and resolution are taken into account. Thus, the division result with an appropriate resolution can be appropriately selected and output.

It is a block diagram which shows one Embodiment of the area | region dividing device of the image | video which concerns on this invention. It is explanatory drawing of the estimation process of an edge direction (example of a step-up image to the right). It is explanatory drawing of the estimation process of an edge direction (an example of a step-down image to the right).

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Partial decoding part, 2 ... 1st area | region division process part, 2-1, 3-1 ... Conversion encoding information determination part, 2-2 ... Coding mode information determination part, 2-3 ... motion prediction information determination unit, 3 ... second region division processing unit, 4 ... third region division processing unit, 4-1 ... edge detection unit

Claims (17)

In an image area dividing device for dividing a compression-coded image into a plurality of areas,
Partial decoding means for partially decoding the compressed encoded data;
A plurality of area division processing means for forming areas divided at different resolutions, including at least one area division processing means for forming an area divided by using information partially decoded by the partial decoding means; An image area dividing device comprising: an image area dividing device; 2. The video according to claim 1, wherein the plurality of area division processing units are hierarchized from a low one to a high resolution of the area division, and sequentially form divided areas according to the hierarchy. Area dividing device. 3. The video area dividing device according to claim 2, wherein each of the plurality of area dividing processing means forms a divided area by using a division result of an upper layer area dividing processing means. 3. The video area dividing device according to claim 2, wherein the plurality of area dividing processing means includes area dividing processing means hierarchized according to an encoding information allocation unit. 5. The video area according to claim 4, wherein the plurality of area division processing means includes area division processing means for forming a divided area by using encoding information for each encoding information allocation unit. Splitting device. The plurality of area division processing means includes low resolution area division processing means for forming a divided area using transform coding information, coding mode information, and motion prediction information. The video image segmentation apparatus described. The low resolution region division processing unit determines whether or not the difference of the color signal DC component between the blocks is less than a threshold value from the transform coding information about the color difference signal, and integrates the blocks where the difference is determined to be less than the threshold value. 7. The image area dividing apparatus according to claim 6, further comprising means for forming a divided area. The low resolution region dividing processing means extracts the mode encoding mode information within a predetermined time from the encoding mode information for each position, and uses the identity of the mode encoding mode to divide the divided region. 8. The image area dividing device according to claim 6, further comprising means for forming the image. 8. The low-resolution area dividing processing means includes means for forming a divided area using similarity in direction and length of motion vectors obtained from motion prediction information. The image segmentation device described in 1. 6. The video area dividing device according to claim 5, wherein the plurality of area dividing processing means includes area dividing processing means for forming an area divided by using transform coding information. An area division processing unit that forms an area divided using the transform coding information determines whether or not the difference in luminance signal DC component between blocks is equal to or less than a threshold from the transform coding information about the luminance signal. 11. The video area dividing device according to claim 10, further comprising means for forming a divided area by integrating blocks determined to be equal to or less than a threshold value. The area division processing means for forming a divided area using the transform coding information has edge estimation means for estimating the presence and direction of the edge of the luminance signal from the AC component of the transform coding information for the luminance signal. 12. The image region dividing apparatus according to claim 10, wherein divided regions are formed using the estimation result. The edge estimation means estimates the presence / absence and direction of an edge of a luminance signal using the sum of absolute values of AC components in the horizontal direction and vertical direction of transform coding information for the luminance signal. 12. The image area dividing device according to 12. The edge estimation means compares the AC component partial absolute value sum with a threshold value and uses the magnitude relationship between the AC component partial absolute value sums in the horizontal direction and the vertical direction or the ratio of the two to determine whether or not there is an edge of the luminance signal. The image region dividing device according to claim 13, wherein the image region dividing device is estimated. 13. The video area dividing device according to claim 12, wherein the area dividing processing means for forming an area divided using the transform coding information performs area division with a resolution equal to or higher than a block unit. 2. The video area dividing device according to claim 1, wherein the plurality of area dividing processing means include area dividing processing means for forming a divided area using pixel information of a luminance signal and a color difference signal. An area division processing unit for forming an area divided using pixel information of the luminance signal and the color difference signal individually detects edges of the luminance signal and the color difference signal, and uses the detected edges to divide the area The image region dividing device according to claim 16, wherein:

Images