JP2005123921A - Apparatus and program of deciding composing order of image segment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that a conventional method for replacing the upper and lower positional relation between two overlapping segments to search a combination having the smallest error with a reference image causes an increase in the number of combinations resulting in the huge cost of error calculation. <P>SOLUTION: An intrusion quantity calculator 127 receives, as an input, overlap information to be outputted from a segment pasting section 126 and segment information of a reference image to be output from a segmentation section 122, and obtains mutual intrusion quantity thereof to output the quantity. A Z-Order deciding section 128 decides a composing order of overlapped images so that the image segment of a reference image having the smaller number of intrusion pixels may be arranged on the upper part. By doing this, even if the shape of a segment segmented from the standard image differs from the shape of a segment segmented from the reference image, the upper and lower positional relation can be decided by a simple inspection method. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a composition order determination apparatus and composition order determination program for image segments, and in particular, a segment of an image segment that divides a temporally continuous two-dimensional image such as a television image into segments of arbitrary shapes and generates a predicted image using the segments. The present invention relates to a composition order determination apparatus and a composition order determination program.

  Conventionally, the MPEG2 (Moving Picture Experts Group phase 2) system has been widely used as an image high-efficiency encoding system typified by digital recording devices and digital broadcasting. In this MPEG2 system, a two-dimensional image is divided into rectangular areas that do not overlap and encoded. Here, a processing method of this rectangular area will be described with a focus on a predictive encoding unit between frames which is related to the present invention.

  In inter-frame prediction, an area with a small pixel value error is selected as a predicted value from an image that is temporally mixed (hereinafter referred to as a reference image) with respect to the image of the rectangular area, and the position of the selected area is indicated. The prediction error between the information, the selected area, and the actual image (hereinafter referred to as a reference image) is encoded.

  As an example, consider creating the predicted image of FIG. 11B based on the image of FIG. FIG. 11A and FIG. 11B are temporally continuous frames. First, as shown in FIG. 11A, the circle 1a and the triangle 2a are positioned apart from each other, and the next frame (reference) In FIG. 11B, it is assumed that the circle has moved to the positional relationship where the circle overlaps the triangle 2b.

  In this case, first, rectangular blocks divided at regular intervals as shown by broken lines in FIG. This block is a unit to be encoded. Note that FIG. 12A is the same image as FIG. 11A, and FIG. 11B and FIG. 12B are also the same image. Next, when searching for the predicted value of block 3 in FIG. 12B from the image one frame before in FIG. 12A, block 5 and block 6 in FIG. 12A become prediction candidates. However, the block 5 with less error is selected as the predicted value. Similarly, as the predicted value of the block 4 in FIG. 12B, the block 7 and the block 8 in FIG. 12A can be considered, but the block 7 with less error is selected as the predicted value.

  Thereafter, the above-described search is performed for all the blocks in FIG. 12B, and a predicted image including the selected blocks 5 and 7 is obtained as shown in FIG. 12C. The vector information of each block for creating the predicted image and the prediction error between the predicted image of FIG. 12C and the reference image of FIG. 12B are DCT (Discrete Cosine Transform). After being subjected to orthogonal transformation and quantization in accordance with the above, variable length coding such as Huffman code is performed and output from the encoder. In the decoder, a prediction image is generated from the vector information, and the image is restored by adding the prediction error.

  On the other hand, a segment unit encoding technique is known. This is a method of dividing an area having the same feature amount in an image into an arbitrary shape. The feature amount includes, for example, a luminance value, a color, and a frequency. The difference between the segment unit encoding and the block unit encoding is that the predicted image search unit is changed from a block to a segment.

  As an example of this segment search method, a segment search method is known in which matching of segments is performed in both forward (in time order) and backward (in reverse time order) between two frames. By this segment search method, even if a part of the segment is concealed, either forward matching or backward matching succeeds, so that motion compensation can be performed on a segment basis.

  When this segment search method is applied to the example of FIG. 11, in FIG. 13 (a) showing the same image as FIG. 11 (a), the circle 1a is cut out along the outer periphery, and the triangle in FIG. 2a is also cut out along its outer periphery. The cut circle 1a is moved to the position of the circle 1c in FIG. 13B, and then the triangle 2a is arranged so as to overlap the circle 1c like the triangle 2b in FIG. 13C. FIG. 13C is a predicted image.

  As is clear from the prediction image in FIG. 12C and the prediction image in FIG. 13C, the prediction error with respect to FIG. 11B in which the prediction image using the segment-by-segment coding technique is the reference image. May be reduced. One reason is that segment overlap is allowed. In the above-mentioned patent document 1, the vertical relationship (hiding relationship) of this overlap is called Z order (Z-Order). However, the vertical relationship of the overlap of the two segments is that the triangle 2b is pasted on the circle 1c as shown in FIG. 14 (a) and the triangle 2b as shown in FIG. 14 (b). There are two ways in which the circle 1c is pasted. Patent Document 1 discloses a method of calculating an error with a reference image while switching the vertical relationship and selecting a vertical relationship with a small error.

International Publication No. 02/13127 Pamphlet

  In Patent Document 1, a segment is defined as a group of pixels that move in the same manner. However, strictly speaking, it is essential that one segment is a rigid body, and the size and shape of the segment do not change between two frames. Is based on the precondition. However, in the actual video, the movement of the subject causes a change in the distance from the camera that is shooting to the subject and a new concealment, so it is rare that the size and shape of the segment related to the subject image do not change, The technique of Patent Document 1 cannot be applied as it is. This is because if a predicted image is created on the assumption that the size or shape of the segment changes, a segment having a shape different from that of the reference image must be pasted.

  Therefore, when an image is to be handled practically, the vertical relationship (combination order) must be determined for a segment having a size or shape different from that of the reference image. However, Patent Document 1 discloses an invention for searching for a combination that minimizes an error from a reference image by exchanging the upper and lower relations of two overlapping segments. Considering that the shape change occurs, the number of combinations increases and the cost of error calculation becomes enormous.

  The present invention has been made in view of the above points, and overlaps by making motion compensation using a segment cut out from a reference image out of two consecutive frames and creating a prediction image with little error from the reference image. An object of the present invention is to provide an image segment composition order determination apparatus and composition order determination program capable of determining the composition order of image segments by easily determining the hierarchical relationship (hiding relation) of the segments with a small amount of calculation. .

  In order to achieve the above object, an image segment composition order determination apparatus according to a first aspect of the present invention uses one of the two time-continuous two-dimensional images as a reference image, and sets the other two-dimensional image as a reference image. When a two-dimensional image is used as a reference image, a first image segment having an arbitrary shape that is a set of pixels having the same characteristics cut out from the reference image, and an arbitrary shape that is a set of pixels having the same characteristics cut out from the reference image An image segment composition order determination device for determining a composition order with the second image segment, segmentation means for separately cutting out the first image segment and the second image segment from the reference image and the reference image, respectively Based on the first image segment and the second image segment, the first image segment is moved to move the reference image Motion vector detecting means for detecting a motion vector indicating a destination of the first image segment for similarity, and a segment boundary of the corresponding second image segment when the first image segment is moved according to the motion vector A segment pasting unit that generates overlap information indicating a concealment relationship of a plurality of first image segments that are in a relationship of overlapping at least part of the first image segment, and when the first image segment is moved according to a motion vector The portion of the second image segment corresponding to the first image segment crosses the segment boundary and overlaps with the other second image segment corresponding to the other first image segment in the concealment relationship. The area to be covered is the number of the first image segments that are concealed based on the overlap information. Erosion amount calculation means for calculating each, and the first image segment composition order so that the first image segment having a smaller area among the plurality of areas calculated by the erosion amount calculation means is displayed above. And a composition order determining means for determining.

  In the present invention, even when the image segment shape of the reference image is different from the image segment shape of the reference image, it can be determined that an image segment having a smaller erosion amount (erosion area) is displayed above.

  In order to achieve the above object, a composition order determining apparatus for image segments according to a second aspect of the present invention includes a segmentation means for separately cutting out the first image segment and the second image segment from the reference image and the reference image, respectively. Motion vector detection for detecting a motion vector indicating a destination of a first image segment for approximating a reference image by moving the first image segment based on the first image segment and the second image segment And a concealment relationship of a plurality of first image segments in which at least a part of the first image segment overlaps the segment boundary of the corresponding second image segment when the first image segment is moved according to the motion vector Segment pasting means for generating overlap information indicating the first image segment Another second image corresponding to another first image segment in a concealment relationship across the segment boundary of the second image segment corresponding to the first image segment when moving the image according to the motion vector The sum of the absolute value errors of the luminance values of the pixels of the first and second image segments in the overlapping portion in the segment is calculated based on the overlap information of the plurality of first and second image segments that are concealed. The first image segment having a smaller sum of absolute value errors among the sum of the error amount calculating means calculated for each and the plurality of absolute value errors calculated by the error amount calculating means is displayed on the top. And a composition order determining means for determining the composition order of one image segment.

  In the present invention, even when the image segment shape of the reference image and the image segment shape of the reference image are different, the sum of the absolute value errors of the luminance values of the pixels of the first and second image segments in the overlapping portion is small. The image segment can be determined to be displayed above.

In order to achieve the above object, an image segment composition order determination program according to a third aspect of the invention is a program for causing a computer to implement the image segment composition order determination device according to the first aspect of the invention. ,
Segmentation means for separately cutting the first image segment and the second image segment from the reference image and the reference image, respectively, and moving the first image segment based on the first image segment and the second image segment; Motion vector detecting means for detecting a motion vector indicating a destination of the first image segment for approximating the reference image, and a corresponding second image segment when the first image segment is moved according to the motion vector Segment pasting means for generating overlap information indicating the concealment relationship of a plurality of first image segments that are in a relationship of overlapping at least partially beyond the segment boundary, and moving the first image segment according to a motion vector The second corresponding to the first image segment The area eroded by the overlapping portion in another second image segment corresponding to another first image segment in the concealment relationship beyond the segment boundary of the image segment is changed to the concealment relationship based on the overlap information. The erosion amount calculation means for calculating each of a plurality of first image segments, and the first image segment having a smaller area among the plurality of areas calculated by the erosion amount calculation means, is displayed above. It is characterized by functioning as a composition order determining means for determining the composition order of the first image segments.

  In the present invention, even when the image segment shape of the reference image is different from the image segment shape of the reference image, it can be determined that an image segment having a smaller erosion amount (erosion area) is displayed above.

  Furthermore, in order to achieve the above object, the image segment composition order determination program of the fourth invention is a program for causing a computer to realize the image segment composition order determination apparatus of the second invention. To do.

  In the present invention, even when the image segment shape of the reference image and the image segment shape of the reference image are different, the sum of the absolute value errors of the luminance values of the pixels of the first and second image segments in the overlapping portion is small. The image segment can be determined to be displayed above.

In each of the above-described inventions, when the first image segment is moved based on the motion vector and pasted on the predicted image, the portion protruding from the segment boundary of the second image segment obtained by segmenting the reference image Finds how many pixels are eroding in which peripheral image segment. The above operation is performed for all the image segments, the number of pixels in which two overlapping image segments are eroded is compared, and the one with the smaller number of eroded pixels is determined as the upper layer. The comparison of the number of eroded pixels is obtained for all overlapping segments, and the composition order of the first image segments is determined by examining the vertical relationship. This is because a group of pixels having the same characteristics (that is, similar luminance values or color difference values) is an image segment, and therefore, the error per pixel of the eroded portion is substantially the same value.
Total error amount = (1 pixel error) x (number of pixels)
It is because it can be expressed. Here, the error of one pixel is an absolute value error of luminance values per pixel of the two first image segments that erode after moving based on the motion vector.

  Alternatively, the synthesis order of the first image segments can be determined based on the magnitude of the sum of absolute value errors of the eroded portions. As a result, according to the first to fourth inventions, it is possible to realize the composition order determination between segments by a simple method of obtaining and comparing the count of the number of pixels or the sum of absolute value errors.

  According to the present invention, even when the shape of the first image segment cut out from the reference image and the shape of the second image segment cut out from the reference image are different from each other, the ratio of the erosion area, that is, the ratio of the number of pixels, or the erosion Since the vertical relationship (the concealment relationship) can be determined by a simple inspection method that is the sum of the absolute value errors of only the portions, the overlap processing in the arbitrary shape coding can be realized at a low calculation cost.

  Next, the best mode for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of an image high-efficiency encoding apparatus (encoder) including an image segment composition order determining apparatus according to the present invention. This high-efficiency image encoding device (encoder) generates a prediction image from two-dimensionally input two-dimensional image signals that are temporally continuous, and uses segment shape information, motion vectors, image segments, etc. by Huffman code or arithmetic code The Z order (Z-Order) information indicating the synthesis order (hereinafter also simply referred to as “segment”) and the prediction error are encoded and output.

  Here, after the data constituting the reference image is encoded in advance, it is output from the output terminal 19 and transmitted to the decoding device (decoder), and also to the local decoder 18 in the segment unit image encoding device (encoder). In the following description, it is assumed that the decoded reference image signal is also output from the local decoder 18.

  First, a two-dimensional reference image signal to be encoded is input from the input terminal 11. The time-sequential input reference image signal and the reference image signal output from the local decoder 18 are respectively input to the Z-Order determiner 12 which is an image segment combination order determination device of the present invention. The Z-Order determiner 12 divides (cuts out) the reference image into image segments of arbitrary shapes from two temporally continuous image signals (reference image signal and reference image signal), and then extracts the reference image from the reference image. A motion vector indicating a movement destination of the image segment is detected, and the image segment obtained by the division is moved according to the detected motion vector.

  At this time, if there is a portion where overlap occurs between the image segments, the vertical relationship is determined (that is, the composition order of the image segments is determined). The segment shape information obtained by the above processing is output from the terminal 13, the Z-Order information is output from the terminal 14, and the motion vector information is output from the terminal 15.

  Next, the prediction error unit 16 generates a prediction image signal based on information input from the terminals 13 to 15, and then an error between the generated prediction image signal and the reference image signal input from the input terminal 11. And the prediction error signal is output to the variable length encoder 17. The variable length encoder 17 converts the segment shape information, motion vector, Z-Order information input from the terminals 13 to 15 and the prediction error signal input from the prediction error unit 16 into a known orthogonal transform such as DCT or quantum. After the encoding process, encoding is performed using a Huffman code or an arithmetic code.

  The encoded signal output from the variable length encoder 17 is output from the output terminal 19 and also input to the local decoder 18. The local decoder 18 decodes the encoded signal input from the variable length encoder 17 to reconstruct the reference image signal. The reconstructed reference image signal is output from the local decoder 18 as a reference image signal for encoding the reference image signal of the next time, and is input to the Z-Order determiner 12.

  Next, the configuration and operation of the Z-Order determiner 12 constituting the composition order determining apparatus for image segments according to the present invention will be described in detail. FIG. 2 is a block diagram of an embodiment of a composition order determining apparatus for image segments according to the present invention, and FIG. 3 is a flowchart for explaining the operation of an example of FIG. 2, the same components as those in FIG. 1 are denoted by the same reference numerals. In FIG. 2, a two-dimensional reference image signal to be encoded input to the input terminal 11 of the Z-Order determiner 12 constituting the image segment composition order determining apparatus according to the present invention is predetermined by an image memory 121. After being accumulated for the frame, it is divided into image segments by the segmentation unit 122. At the same time, the reference image signal output from the local decoder 18 in FIG. 1 is supplied to the image memory 123 via the input terminal 20 in FIG. Divided into units (step S1 in FIG. 3).

  At this time, the segment information output from the segmentation unit 124 is output to the outside from the output terminal 13 for use in encoding. Here, since the present invention can be considered independently of the segmentation method, a detailed description of the method is omitted. For example, a known image segmentation method disclosed in Japanese Patent Laid-Open No. 10-69544 is used. Can do.

  FIG. 4 is an example of two images for explaining the present embodiment, FIG. 4A shows an example of a reference image, and FIG. 4B shows an example of a reference image. The reference image shown in FIG. 4A is divided by the segmentation unit 124 into three segments of a quadrangle 41a, a triangle 42a, and a circle 43a. Also, the reference image shown in FIG. 4B is divided into three segments of a square 41b, a triangle 42b, and a circle 43b by the segmentation unit 122.

  Next, the segment of the reference image obtained in step S1 is moved based on the motion vector information (step S2 in FIG. 3). That is, in step S2, the segment information output from the segmentation units 122 and 124 is input to the motion vector detection unit 125 in FIG. 2, and the motion vector is obtained. This motion vector search method is not directly related to the gist of the present invention, and is well known and will not be described in detail. For example, the method described in Patent Document 1 can be used.

  The motion vector information obtained by the motion vector detection unit 125 is output to the outside from the output terminal 15 and also input to the segment pasting unit 126. The segment pasting unit 126 moves the segment of the reference image input from the segmentation unit 124 based on the motion vector information input from the motion vector detection unit 125. The above processing is step S2 in FIG.

  In the example of FIG. 4, the predicted image that approximates the segment of the reference image in FIG. 4B is created by moving the segment of the reference image in FIG. That is, the quadrangle 41a shown in FIG. 4A is changed to the quadrangle 41b shown in FIG. 4B, the triangle 42a shown in FIG. 4A is changed to the triangle 42b shown in FIG. 4B, and the circle 43a shown in FIG. Is moved to the position of the circle 43b in FIG. In the actual image, the triangle 42a partially concealed in the circle 43a is a triangle 42b that is partially concealed by the quadrangle 41b and has a large size. In addition, the quadrangular 41a is reduced in size as indicated by 41b, and partially hides the triangle 42b. Furthermore, the circle 43a that has concealed a part of the triangle 42a becomes a circle 43b that exists independently without changing its size or position.

  FIG. 5A shows segment boundaries as a result of segmenting the reference image of FIG. 4A by broken lines 51a, 52a, and 53a. FIG. 5B shows the reference of FIG. 4B. Segment boundaries as a result of segmenting the image are indicated by broken lines 51b, 52b, and 53b. As the predicted image generated by the motion vector, when the rectangle 41a and the triangle 42a in FIG. 4 (a) are arranged inside the segment boundaries 51b and 52b indicated by the broken line in FIG. 5 (b), FIG. 6 and FIG. 7 can be considered.

  That is, in FIG. 6, the quadrangular 41 a is hidden by 41 a ′, and the triangle 42 a is indicated by 42 a ′, so that the quadrangular 41 a ′ conceals a part of the triangular 42 a ′. It is a case. On the other hand, FIG. 7 shows that the quadrangular 41a is 41a "and the triangular 42a is indicated by 42a", so that a part of the quadrilateral 41a "is concealed by the triangle 42a" by being arranged inside the segment boundaries 51b and 52b. It is a case. The segment pasting unit 126 in FIG. 2 outputs the existence of the concealment relationship as overlap information.

  Next, based on the segment boundary of the corresponding reference image, the number of pixels eroding each other is counted based on the segment boundary of the corresponding reference image (step S3 in FIG. 3). That is, in this step S3, the erosion amount calculation unit 127 in FIG. 2 receives the overlap information output from the segment pasting unit 126 and the segment information of the reference image output from the segmentation unit 122 as inputs, and receives the mutual information. The amount of erosion is calculated and output. Here, the amount of erosion refers to an area protruding from a region of another segment that overlaps the reference image when the segment of the reference image is fitted into the segment boundary of the corresponding reference image.

  As an example, an operation for counting the number of eroded pixels of the rectangle 41a and the triangle 42a in the reference image of FIG. 4A will be described. In this case, the segment boundary 51b of the square broken line frame of FIG. 5B, which is the segmentation result of the reference image obtained in step S1 of FIG. 3, corresponds to the segment rectangle 41a of the reference image of FIG. The segment boundary 52b of the triangular broken line frame in FIG. 5B, which is the segmentation result of the reference image, corresponds to the triangle 42a of the segment of the reference image in FIG.

  Therefore, as shown in FIG. 8, the number of pixels in which the square 41a erodes into the segment boundary 52b of the triangular broken line frame exists between the segment boundary 51b of the square broken line frame and the outer frame 60 of the square 41a, and This is the number of pixels in the region 61 existing inside the segment boundary 52b of the triangular broken line frame. Further, as shown in FIG. 9, the number of pixels in which the triangle 42a erodes the segment boundary 51b of the square broken line frame exists between the segment boundary 52b of the triangular broken line frame and the outer frame 62 of the triangle 42a, and Although it is the number of pixels existing inside the segment boundary 51b of the square broken line frame, as can be seen from FIG. 9, there is no eroded pixel in this case.

  Finally, Z-Order, which is the image segment combination order, is determined so that the image segment with a small number of eroded pixels obtained in step S3 in FIG. 3 is arranged on top (step S4 in FIG. 3). That is, in this step S4, the Z-Order determining unit 128 in FIG. 2 arranges the image segment of the reference image with a small number of eroded pixels on the basis of the erosion amount input from the erosion amount calculation unit 127. The composition order of overlapping image segments is determined, and the determined composition order information (Z-Order information) of the image segments is output from the output terminal 14 to the outside.

  In the example of FIGS. 8 and 9, since the number of eroded pixels of the triangle 42a (in this case, zero pixels) is smaller, the triangle 42a is arranged on the quadrangle 41a. The arrangement shown by 42a ″ and 41a ″ in FIG. Are selected as predicted images. As described above, the synthesis order (Z-Order) of a set of overlapping image segments is obtained. By performing this operation for all overlapping image segments, a predicted image is completed.

  Next, another method for determining the vertical relationship of overlapping image segments according to the present invention will be described with reference to the flowchart of FIG. However, in FIG. 10, the same number is given to the step which performs the same operation as FIG. Since steps S1 and S2 in the figure have been described above, a description thereof will be omitted. The present embodiment is characterized in that in step S11 in FIG. 10, the sum of absolute value errors of pixels with the reference image is obtained instead of the number of pixels in step S3 in FIG. That is, the sum of the absolute value errors of the luminance value Y (x) of the pixel of the quadrangle 41a in the erosion area 61 of FIG. 8 and the luminance value D (x) of the pixel of the triangle 42a in the erosion area 61 of FIG. E is obtained by the following equation.

ただし、上式中、ｘは侵食領域６１の内部にある画素を示す値である。同様に、図９で三角形４２ａが四角形５１ｂに侵食している部分の絶対値誤差の和を求めるが、この例では零である。ここで、上記の数１の数式中、｜Ｙ（ｘ）−Ｄ（ｘ）｜は、ｘによらず略一定値になることが予想され、Ｑ＝｜Ｙ（ｘ）−Ｄ（ｘ）｜とすると、上式はＥ＝Ｑ・ｘと表すことができる。

However, in the above formula, x is a value indicating a pixel inside the erosion area 61. Similarly, in FIG. 9, the sum of absolute value errors of the portion where the triangle 42a erodes the quadrangle 51b is obtained. In this example, it is zero. Here, in the above mathematical formula 1, | Y (x) −D (x) | is expected to be a substantially constant value regardless of x, and Q = | Y (x) −D (x) If |, then the above equation can be expressed as E = Q · x.

  Next, Z-Order is determined so that a segment with a small sum of absolute value errors of the eroded area obtained in step S11 of FIG. 10 is arranged on top (step S12 of FIG. 10). Here, of two overlapping segments A and B, for example, assuming that the luminance value per pixel of A is Pa and that of B is Pb, the error of the portion where A erodes B is Qa. = | Pa−Pb |, and the error of the portion where B erodes A is Qb = | Pb−Pa |. However, since the absolute value is taken, Qa = Qb.

  On the other hand, the error amount of the portion where A erodes B can be expressed as Ea = Qa · xa based on Equation 1. Similarly, the error amount of the portion where B erodes A is Eb = Qb · xb. (Where xa and xb are the number of pixels in segments A and B in the eroded area). Here, since Qa = Qb as described above, the magnitude relationship between Ea and Eb is determined by the magnitude relationship between xa and xb. Therefore, in the above step S12, a segment having a small sum of absolute value errors of the eroded area is arranged on the upper side. In the embodiment of FIG. 10, although the amount of calculation increases, it is possible to obtain a more accurate image segment synthesis order than in the embodiment of FIG.

  In the above embodiment, two image segments are described as overlapping. However, the present invention is not limited to this. For example, three or more image segments overlap in the same region. Even in such a case, the present invention can be applied by determining the vertical relationship while selecting a pair of two image segments. In this case, the image segments are synthesized in such an order that the smaller the erosion area of the overlapping portion is displayed on the upper side. In addition, the present invention can be applied to a segment unit image encoding apparatus having a configuration other than that shown in FIG.

  The present invention also includes a computer program for causing a computer to realize the functions of the above apparatus. This computer program may be read from a recording medium and taken into a computer, or may be downloaded from a communication network and taken into a computer.

1 is a block diagram of an example of an image high efficiency encoding device (encoder) including an image segment composition order determination device of the present invention. FIG. It is a block diagram of one embodiment of a composition order determining apparatus for image segments according to the present invention. It is a flowchart for description of an example of a determination processing method of the vertical relationship of overlapping segments according to the present invention. It is an example of a reference image and a reference image. It is explanatory drawing of an example of a segmentation result. It is a figure explaining an example of a prediction picture by segment pasting. It is a figure explaining other examples of a prediction picture by segment pasting. It is explanatory drawing of an example of an erosion area. It is explanatory drawing of the other example of an erosion area. It is a flowchart of the other example of the determination method of the vertical relationship of the segment which overlaps by this invention. It is an example of the reference image and reference image for demonstrating the process which produces | generates a predicted image. It is explanatory drawing of the production process of the prediction picture in block unit coding. It is explanatory drawing of the production | generation process of the estimated image in segment unit coding. It is a figure explaining the difference in the prediction image produced | generated by the difference in the segment sticking order in segment unit encoding.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Reference image input terminal 12 Z-Order determiner 13, 14, 15, 19 Output terminal 16 Prediction error device 17 Variable length encoder 18 Local decoder 20 Reference image input terminal 41a, 41b Rectangle 42a, 42b Triangle
43a, 43b Circles 51a, 51b Square broken line frame boundary 52a, 52b Triangular broken line segment boundary 53a, 53b Circular broken line segment boundary 60, 62 Outer frame 61 Pixel erosion area 121, 123 Image memory 122, 124 Segmentation Unit 125 motion vector detecting unit 126 segment pasting unit 127 erosion amount calculating unit 128 Z-Order determining unit

Claims (4)

Of two time-continuous two-dimensional images, when one of the past two-dimensional images is used as a reference image and the other two-dimensional image is used as a reference image, pixels having the same characteristics cut out from the reference image are displayed. Image segment composition order determination apparatus for determining a composition order of a first image segment having an arbitrary shape as a set and a second image segment having an arbitrary shape as a set of pixels cut out from the reference image. Because
Segmentation means for separately cutting out the first image segment and the second image segment from the reference image and the reference image, respectively.
Based on the first image segment and the second image segment, a motion vector indicating a movement destination of the first image segment for moving the first image segment to approximate the reference image is detected. Motion vector detecting means for performing,
Concealing a plurality of first image segments in a relationship where at least a portion thereof overlaps across a segment boundary of the corresponding second image segment when the first image segment is moved according to the motion vector Segment pasting means for generating overlap information indicating the relationship;
When the first image segment is moved according to the motion vector, another first image segment that is in the concealment relationship exceeds the segment boundary of the second image segment corresponding to the first image segment. An erosion amount calculating means for calculating an erosion area of an overlapping portion in another second image segment corresponding to each of the plurality of first image segments in the concealment relationship based on the overlap information; ,
Composition order determining means for determining the composition order of the first image segments so that the first image segment having a smaller area among the plurality of areas calculated by the erosion amount calculating means is displayed above. An image segment composition order determination apparatus characterized by comprising: Of two time-continuous two-dimensional images, when one of the past two-dimensional images is used as a reference image and the other two-dimensional image is used as a reference image, pixels having the same characteristics cut out from the reference image are displayed. Image segment composition order determination apparatus for determining a composition order of a first image segment having an arbitrary shape as a set and a second image segment having an arbitrary shape as a set of pixels cut out from the reference image. Because
Segmentation means for separately cutting out the first image segment and the second image segment from the reference image and the reference image, respectively.
Based on the first image segment and the second image segment, a motion vector indicating a movement destination of the first image segment for moving the first image segment to approximate the reference image is detected. Motion vector detecting means for performing,
Concealing a plurality of first image segments in a relationship where at least a portion thereof overlaps across a segment boundary of the corresponding second image segment when the first image segment is moved according to the motion vector Segment pasting means for generating overlap information indicating the relationship;
When the first image segment is moved according to the motion vector, another first image segment that is in the concealment relationship exceeds the segment boundary of the second image segment corresponding to the first image segment. The sum of the absolute value errors of the luminance values of the pixels of the first and second image segments in a portion overlapping in another second image segment corresponding to the above-described concealment relationship based on the overlap information An error amount calculating means for calculating each of a plurality of first and second image segments;
Of the plurality of absolute value errors calculated by the error amount calculation means, the first image segment of the first image segment is displayed so that the first image segment having the smaller absolute value error sum is displayed higher. An image segment composition order determination apparatus comprising: composition order determination means for determining a composition order. Of two time-continuous two-dimensional images, when one of the past two-dimensional images is used as a reference image and the other two-dimensional image is used as a reference image, pixels having the same characteristics cut out from the reference image are displayed. Image segment composition order determination apparatus for determining a composition order of a first image segment having an arbitrary shape as a set and a second image segment having an arbitrary shape as a set of pixels cut out from the reference image. Is a program for realizing by a computer,
The computer,
Segmentation means for separately cutting out the first image segment and the second image segment from the reference image and the reference image, respectively.
Based on the first image segment and the second image segment, a motion vector indicating a movement destination of the first image segment for moving the first image segment to approximate the reference image is detected. Motion vector detecting means for performing,
Concealing a plurality of first image segments in a relationship where at least a portion thereof overlaps across a segment boundary of the corresponding second image segment when the first image segment is moved according to the motion vector Segment pasting means for generating overlap information indicating the relationship;
When the first image segment is moved according to the motion vector, another first image segment that is in the concealment relationship exceeds the segment boundary of the second image segment corresponding to the first image segment. An erosion amount calculating means for calculating an erosion area of an overlapping portion in another second image segment corresponding to each of the plurality of first image segments in the concealment relationship based on the overlap information; ,
Composition order determining means for determining the composition order of the first image segments so that the first image segment having a smaller area among the plurality of areas calculated by the erosion amount calculating means is displayed above. A program for determining the composition order of image segments, characterized in that it functions as an image segment. Of two time-continuous two-dimensional images, when one of the past two-dimensional images is used as a reference image and the other two-dimensional image is used as a reference image, pixels having the same characteristics cut out from the reference image are displayed. Image segment composition order determination apparatus for determining a composition order of a first image segment having an arbitrary shape as a set and a second image segment having an arbitrary shape as a set of pixels cut out from the reference image. Is a program for realizing by a computer,
The computer,
Segmentation means for separately cutting out the first image segment and the second image segment from the reference image and the reference image, respectively.
Based on the first image segment and the second image segment, a motion vector indicating a movement destination of the first image segment for moving the first image segment to approximate the reference image is detected. Motion vector detecting means for performing,
Concealing a plurality of first image segments in a relationship where at least a portion thereof overlaps across a segment boundary of the corresponding second image segment when the first image segment is moved according to the motion vector Segment pasting means for generating overlap information indicating the relationship;
When the first image segment is moved according to the motion vector, another first image segment that is in the concealment relationship exceeds the segment boundary of the second image segment corresponding to the first image segment. The sum of the absolute value errors of the luminance values of the pixels of the first and second image segments in a portion overlapping in another second image segment corresponding to the above-described concealment relationship based on the overlap information An error amount calculating means for calculating each of a plurality of first and second image segments;
Of the plurality of absolute value errors calculated by the error amount calculation means, the first image segment of the first image segment is displayed so that the first image segment having the smaller absolute value error sum is displayed higher. An image segment composition order determination program which functions as a composition order determination means for determining a composition order.

Images