JP6180583B1 - Method and program for associating respective closed regions of at least two images including a plurality of closed regions - Google Patents

Abstract

To create a two-dimensional animation more smoothly by reducing the human work burden in the process of creating a two-dimensional animation. A method for identifying a second closed region 2A corresponding to a first closed region 1A of an original image 1 from an original image 2, wherein each of the original image 1 and the original image 2 includes a plurality of pixels. Each of the plurality of pixels has a feature amount. The method includes the first closed region and the original image 2 based on the feature amount of each of the plurality of pixels included in the first closed region and the feature amount of each of the plurality of pixels included in the original image 2. Determining the degree of approximation with the plurality of closed regions 2A to 2C, and among the plurality of closed regions included in the original image 2, one closed region indicating the closest approximation is set as the second closed region. Identifying. The feature amount is a function of the distance from the boundary line of the closed region including the pixel to the pixel. [Selection] Figure 1

Description

  The present invention relates to a method and a program for associating each closed region of at least two images including a plurality of closed regions.

  In general, a two-dimensional animation is created by creating a plurality of two-dimensional images and sequentially displaying each of the plurality of two-dimensional images in time series. Two-dimensional animation is an expression technique for constructing a moving image by reproducing a still image created by handwriting in association with each frame on the time axis.

  In the old days, it was possible to create cartoons (two-dimensional animations) such as television and movies by creating a plurality of handwritten images, sequentially shooting the images on a movie film, and projecting the movie film. I came. Since each of the plurality of images is an image in which the motion of the character or the like is changed little by little, a moving image of a character with motion can be generated by continuously displaying the plurality of images. Further, as a more primitive technique, there is a flip book that expresses a moving character by flipping a plurality of images drawn on a plurality of papers with a finger.

A general two-dimensional animation is often created by the following steps.
Step (1) Creation of an original image: An image included in a key frame that is a key frame scattered at a predetermined timing among a series of moving image images is called an original image. Mainly designers create original drawings.
Step (2) Creation of intermediate image: An image that is located between a key frame and the next key frame on the time axis and smoothly interpolates the previous and subsequent original images (this image is called “intermediate image”) Create
Step (3) Coloring: Coloring of a plurality of closed regions existing in the original image and the middle image.
Step (4) Shooting: The original image and the background are combined, and the intermediate image and the background are combined to create each frame.

  The original picture of the process (1) is basically created by a designer. The original image is created by a designer handwriting or using computer graphics software or the like, which is a characteristic motion part of a moving character. Typically, the designer does not create all the frames of the video, but draws an original picture for the keyframes. The original picture is drawn across several frames.

  The process (2) is creation of a middle-sized image. In many cases, a person other than the designer creates an intermediate image between one original picture and the original picture positioned next in time. The intermediate image is an image used for a blank frame located between adjacent original images on the time axis. The purpose of creating an intermediate image is to fill a blank frame by creating an image obtained by interpolating key frames including adjacent original images on the time axis in order to create a smooth moving image.

  Therefore, the intermediate image is drawn so that the entire moving image is smooth. The half-cut image is often created by handwriting. However, since the number of images to be created is larger than that of the original image, many resources are required. In addition, although it has been attempted to automatically create an intermediate image, it is difficult to automatically create an intermediate image when creating a natural moving image.

  The process (3) is coloring. Since both the original image and the intermediate image are often drawn as line drawings, an operation (coloring) of coloring each closed region formed by the line drawing with an appropriate color is performed. Also for coloring, digitized processing is widespread.

  The step (4) is photographing. In the old days, each frame was shot on a silver salt film. In the photographing step (4), digitized processing is widespread, and a digital moving image is generated.

  In the step (2) and the step (3), there are many parts that require manual work, and there are many ratios where the operator performs complicated work. This is because the images of a plurality of objects existing in the original image are overlapped or deformed. In other words, the corresponding closed areas in the two original images are deformed or the corresponding intersections (intersections where the boundary lines of the closed areas intersect) move, so that automatic association becomes difficult. ing.

  The following techniques exist as techniques for finding corresponding intersections existing in two images.

  A control point for indicating the corresponding positional relationship between the two bitmap images is designated for each of the two bitmap images to be key frames, and determined on a straight line connecting the corresponding control points. There is a technique for generating an interpolation frame by changing a control point to an internal dividing point (for example, Patent Document 1).

In addition, in order to perform matching processing efficiently, there is a technique in which the matching unit obtains the contour point of the second shape data that matches the contour point of the first shape data by using dynamic programming (for example, Patent Documents). 2)
The following techniques exist as techniques for finding the corresponding closed regions existing in the two images.

  For example, there is a technique in which closed regions such as heads and hands are listed and stored as a chain and the characteristics of the closed region are tracked (see, for example, Non-Patent Document 1).

  In addition, there is a technique for performing contour matching and associating similar image closed regions (for example, see Non-Patent Document 2).

Japanese Patent Laid-Open No. 11-7539 JP2015-1115047 A

P. Garcia Trigo, H. Johan, T. Imagire, and T. Nishita. Interactive Region Matching for 2D Animation Coloring Based on Feature's Variation. IEICE ransactions (E92-D), No.6, pp.1289-1295 (2009). J. S. Madeira, A. Stork, and M. H. Gross. An approach to computer-supported cartooning. The Visual Computer, Vol.12, No.1, pp.1-17 (1996).

  As described above, in the middle image creating process (2), many resources are required for creating the middle image. In the coloring step (3), there is a coloring process that involves manual work. In this coloring process, it is necessary to apply the same coloring to the corresponding closed region for each of the original image and the intermediate image. Therefore, when performing coloring, it is necessary to specify a corresponding closed region existing in each frame of the plurality of original images and the plurality of intermediate images. Usually, the determination of the corresponding closed region requires visual inspection by the operator.

  An embodiment disclosed below aims to make it possible to create a two-dimensional animation more smoothly by reducing the work of an operator, particularly in an intermediate image creation process and a coloring process. And

  In one embodiment, the second region corresponding to the first closed region of one image is compared by comparing the plurality of closed regions included in one image with the plurality of closed regions included in the other image. In which the computer specifies the closed region from the other image, wherein each of the one image and the other image includes a plurality of pixels, and each of the plurality of pixels includes a feature amount. And the method uses the feature amount of each of the plurality of pixels included in the first closed region and the feature amount of each of the plurality of pixels included in the other image. A step of obtaining a degree of approximation between the closed region and a plurality of closed regions included in the other image; and the degree of approximation of the plurality of closed regions included in the other image being the closest One closed region indicating Comprising the steps of, a, the feature quantity is a function of the distance from the boundary line of the closed region including the pixel to the pixel, a method.

  By reducing the human work burden in the two-dimensional animation creation process, the two-dimensional animation can be created more smoothly.

It is a figure which shows the example which comprises a two-dimensional animation by the some flame | frame containing an original image and a middle-split image. It is a figure which shows the example of the feature extraction of each pixel which comprises an original picture. It is a figure which shows the example of the table | surface which manages each closed area | region which comprises an original picture. It is a figure which shows the example of the method of matching each closed area | region of two images containing a some closed area | region. It is a figure which shows the result of matching of each closed area | region of two images containing a some closed area | region. It is a flowchart which shows the example of matching of an area | region. It is a figure which shows the example of matching of the intersection which exists in each of two images containing a some closed region. It is a figure which shows the result of matching of each intersection of two images containing a some closed region. It is a figure which shows the example of the 1st subject which generate | occur | produces at the time of matching of the intersection which exists in each of two images containing a some closed region. It is a figure which shows the example of the process which corrects the intersection and boundary line which exist in the image containing a some closed region. It is a figure which shows the example of matching of an intersection based on the result of the process which corrects the intersection and boundary line which exist in the image containing a some closed region. It is a flowchart which discovers the case where the pattern of the several closed region which exists in them when the two intersections are caught as a unity is the same. It is a flowchart which shows the process which matches an intersection. It is a figure which shows the 1st example which creates a middle-split image automatically. It is a figure which shows the example which produces | generates a some middle image. It is a figure which shows the example which provides an intersection when there is no intersection on an isolated circuit. It is a figure which shows the example of the process of an intersection when one of two original pictures has an isolated cycle. It is a figure which shows the example which performs the process which divides | segments an original image into layers, and the process in case a part of closed region respond | corresponds. It is a figure which shows the example of the hardware constitutions of this embodiment.

  FIG. 1 is a diagram illustrating an example in which a two-dimensional animation is configured by a plurality of frames including an original image and a middle-sized image.

  FIG. 1 shows a time axis t. The designer draws original picture 1 and original picture 2. A frame including the original image 1 is displayed from the time t1 to t2. Then, a frame including the original image 2 is displayed from time t3. In this case, since the frame from time t2 to t3 cannot be a blank image, the intermediate image N is created and the frame including the intermediate image N is displayed from time t2 to time t3. Can be displayed smoothly. In the example of FIG. 1, a case where one intermediate image N is added is illustrated, but in reality, two or more intermediate images may be created between the original image 1 and the original image 2. . FIG. 1 shows one intermediate image N for the sake of simplicity.

  If an image expressing an intermediate movement between the original image 1 and the original image 2 can be generated, the intermediate image N can be automatically drawn. In order to draw the intermediate image N, for example, if the correspondence between main intersections in the original picture 1 and the original picture 2 is known, for example, by obtaining an intersection (for example, a middle point) obtained by interpolating the positions of the corresponding intersections. A corresponding intersection point may be obtained in the intermediate image. For example, it can be seen that the intersection 11 of the original picture 1 and the intersection 21 of the original picture 2 are corresponding intersections. Therefore, for example, the coordinates of the intersection N1 of the intermediate image N are obtained by obtaining the coordinates (for example, the coordinates of the middle point) obtained by interpolating the coordinates of the intersection 11 and the coordinates of the intersection 21 (for example, the pixel position of the frame). Can do.

  The correspondence between the intersection points in the original picture 1 and the original picture 2 is as follows. That is, the intersection point 11, the intersection point 12, the intersection point 13, and the intersection point 14 in the original image 1 correspond to the intersection point 21, the intersection point 22, the intersection point 23, and the intersection point 24 in the original image 2, respectively. If the correspondence between the plurality of intersection points in the original image 1 and the original image 2 can be found, the coordinates of the plurality of intersection points N1, N2, N3, and N4 of the intermediate image N are, for example, the original image 1 and the original image 2 Can be obtained by interpolating the corresponding intersection point.

  The above processing has been described on the assumption that the correspondence between each intersection of the original image 1 and each intersection of the original image 2 is known. However, in reality, relying on the operator's instructions for the correspondence between each intersection of the original picture 1 and each intersection of the original picture 2 imposes a great deal of labor on the operator. Therefore, it is required to automatically estimate the correspondence between each intersection of the original picture 1 and each intersection of the original picture 2.

  Turning to FIG. FIG. 7 is a diagram showing an example of association of intersections existing in each of two images including a plurality of closed regions in order to estimate that the intersection 13 of the original image 1 and the intersection 23 of the original image 2 are in a correspondence relationship. It is.

  FIG. 7C is an enlarged view of the vicinity of the intersection 13 in FIG. FIG. 7D is an enlarged view of the vicinity of the intersection 23 in FIG.

  As shown in FIG. 7C, it can be seen that the intersection 13 is an intersection on the boundary line formed by the background Y, the closed region 1B, and the closed region 1C. As shown in FIG. 7D, it can be seen that the intersection 23 is an intersection on the boundary line formed by the background Y, the closed region 2B, and the closed region 2C.

  Here, it is assumed that it is known that the closed region 2B has a corresponding relationship with the closed region 1B, and that the closed region 1C and the closed region 2C have a corresponding relationship. It is assumed that the background Y is the same kind of background in the original picture 1 and the original picture 2 and is known to have a correspondence relationship. A plurality of closed regions existing around the intersection 13 in FIG. 7C exist counterclockwise in the order of the closed region 1B, the background Y, and the closed region 1C. Similarly, the intersection 23 in FIG. The plurality of closed regions present in the surroundings exist in the order of the closed region 2B corresponding to the closed region 1B, the background Y, and the closed region 2C corresponding to the closed region 1C in the counterclockwise direction. From this, it is possible to automatically estimate that the intersection 13 and the intersection 23 are in a correspondence relationship.

  In the above estimation, it is assumed that the correspondence between the plurality of closed regions existing around the intersection 13 of the original image 1 and the plurality of closed regions existing around the original image 2 is known. However, relying on the operator's instructions for the correspondence between the plurality of closed regions of the original image 1 and the plurality of closed regions of the original image 2 imposes a lot of labor on the operator. Therefore, it is required to automatically estimate the correspondence between each intersection of the original image 1 and each intersection of the original image 2 as much as possible.

  Returning to FIG. 1, the above automatically indicates that the closed region 1A, closed region 1B, and closed region 1C of the original image 1 correspond to the closed region 2A, closed region 2B, and closed region 2C of the original image 2, respectively. To be estimated.

From the above, in order to automatically generate the intermediate image N located in the middle of the two original images 1 and 2 that are temporally continuous on the time axis, an estimation process opposite to the above procedure is performed. It is required to execute as follows.
(A) In the original picture 1 and the original picture 2, the correspondence between the closed areas existing in the original picture 1 and the original picture 2 is estimated.
(B) Estimating the correspondence of the intersections of the boundary lines that divide the closed regions existing in the original image 1 and the original image 2 using the information on the association of the closed regions.
(C) Finding the coordinates of intersections that interpolate the associated intersections.
(D) Generate an intermediate image using the shapes of the corresponding closed regions of the original image 1 and the original image 2 and the coordinates of the interpolated intersection point.

  Since the closed areas are associated in the process (a), for example, if the coloring of the original picture 1 is designated, the corresponding closed area in the original picture 2 can be colored automatically. Furthermore, since the closed area corresponding to the closed area in the original image can be found also in the middle image generated in the step (d), coloring of the closed area of the middle image can be performed almost automatically.

  Therefore, in the above description of the prior art, the coloring process is positioned after the process of creating the intermediate image, but in the present embodiment, the process of creating the intermediate image and the coloring process are in the reverse order. It turns out that it may be.

  Hereinafter, detailed embodiments will be described in order.

<1> Embodiment 1: Association of Closed Areas <1-1> Definition of Terms In Embodiment 1, an example of processing for automatically associating a plurality of closed areas existing in two original images is shown. . The following embodiment 1 is an example of processing, and is not limited to this example.

  FIG. 2 is a diagram illustrating an example of feature extraction of each pixel constituting the original image.

  Here, the following definitions are made with reference to FIG.

  “Closed area”: An area closed by a line in an image is called a “closed area”. For example, in FIG. 2A, the closed region 1A, the closed region 1C, and the closed region 1B are “closed regions” each closed by a line. The background of the original picture is not a closed area. Also, neither straight lines, points nor images are closed areas.

  “Boundary line”: A line surrounding a closed region is called a boundary line. The closed region and the closed region or the background are adjacent to each other across the boundary line.

  “Adjacent”: When the closed region A and the closed region B are in contact with each other at the boundary line X, it is said that the closed region A and the closed region B are adjacent to each other at the boundary line X.

  “Intersection”: A point where a boundary line intersects or an end point (end point) of the boundary line is called an intersection.

  “Pixel”: A mesh is defined on a two-dimensional plane existing in the image at equal intervals, and a component of the image existing at the coordinate position of each intersection of the mesh is called a pixel. A pixel may correspond to each pixel constituting the digital image.

  “Azimuth”: In FIG. 2A, four directions from the pixel P1, for example, up, down, left, and right are referred to as “azimuth”. Then, semi-straight lines U, D, L, and R extending in the vertical and horizontal directions are defined. Half straight lines U, D, L, and R are half straight lines extending upward, downward, left, and right from the pixel P1, respectively. In the example of FIG. 2A, half straight lines are defined in four directions, top, bottom, left, and right, respectively, but the number of directions is not limited to four.

  “Number of intersections”: In FIG. 2A, for example, a half line U extending in the upper direction from the pixel P <b> 1 intersects at a boundary line with the closed region 1 </ b> A constituting the original image 1 at U <b> 11. In this case, the number of crossings in the upper direction of the pixel P1 is defined as 1. Similarly, the half line L extending in the left direction of the pixel P1 intersects the boundary lines with the closed regions 1A, 1C, and 1B constituting the original image 1 at L11, L12, and L13. In this case, the number of crossings in the left direction of the pixel P1 is defined as 3. Similarly, the half line D extending in the lower direction of the pixel P1 intersects the boundary line with the closed region A1 constituting the original image 1 at D11. In this case, the number of crossings in the lower direction of the pixel P1 is defined as 1. Similarly, a half straight line R extending in the right direction of the pixel P1 intersects the boundary line with the closed region 1A constituting the original image 1 at R11. In this case, the number of crossings in the right direction of the pixel P1 is defined as 1.

  “Distance to boundary”: In FIG. 2A, for example, the distance from the pixel P1 to the first closed region including the pixel P1 is defined as “distance to the boundary”. In FIG. 2A, the distance to the upper boundary line of the pixel P1 is α1. The unit of the distance to the boundary line may be, for example, the number of pixels.

  For example, when a pixel belongs to a certain closed region, the pixel always crosses the boundary line of the closed region in any direction, so the number of intersections is 1 or more. If the pixel belongs to the background, the number of crossings may be zero.

<1-2> Components of Original Image The original image shown in FIG. 1 and FIG. 18 is a line image in many cases, and includes, for example, the following elements.
(A) Closed area (area closed at the boundary)
(B) Line branching from the closed region (branch line segment)
(C) An isolated line segment included in a closed region (isolated line segment)
A branch line segment and an isolated line segment existing between two original images can be associated by comparing relative coordinates with a boundary line of a closed region. Therefore, first, it is desirable to associate a closed region between two original images. Embodiment 1 shows an example of associating closed regions.

<1-3> Order of Coloring and Intermediate Image Generation In the above-described example of the prior art, the procedure for coloring the original image and the intermediate image after creating the intermediate image is shown. However, in the first embodiment, since each closed region of the original image is first associated, it is possible to color the corresponding closed region of the original image at this stage. First, the closed region is associated between the original images. In a plurality of original images, the corresponding closed areas are painted with the same color. In this way, it is possible to save the trouble of performing coloring for each original picture.

  In addition, if a closed region is already associated between two original images, even if a middle image located between the two original images on the time axis is subsequently generated, it is included in the middle image Can be easily associated with the closed area of the two original images.

  Therefore, after first associating the closed areas in the original image, coloring is performed, and then the closed areas included in the generated intermediate image can be substantially automatically colored.

<1-3> Definition of Feature Vector of Pixel of Original Image In this embodiment, the feature V (P) of the pixel P existing in the original image is defined as an 8-dimensional vector as follows.

V (P) = (c ₀ , c ₁ , c ₂ , c ₃ , c ₄ , c ₅ , c ₆ , c ₇ ) (Formula 1)
Here, c _{0 to} c ₇ are defined as follows. Note that β is a constant.
If the distances from the pixel P to the upper, lower, left, and right boundary lines are αu, αd, αl, and αr, respectively,
c ₀ = 1 / (αu + β) where the number of crossings in the upper direction is 1. Other cases c ₀ = 0
c ₁ = 1 / (αr + β) However, the number of crossings in the right direction is 1. In other cases, c ₁ = 0
c ₂ = 1 / (αd + β) where the number of crossings in the lower position is 1. In other cases, c ₂ = 0
c ₃ = 1 / (αl + β) However, the number of intersections in the left direction is 1. In other cases c ₃ = 0
c ₄ = 1 / (αu + β) However, the number of intersections of the four directions of up, down, left and right are all 2 or more, and the number of intersections of the upper direction is the smallest of the number of intersections of the four directions of up, down, left and right. In other cases c ₄ = 0
c ₅ = 1 / (αr + β) However, when the number of intersections of the four directions of up, down, left and right are all 2 or more, and the number of intersections of the right direction is the smallest of the number of intersections of the four directions of up, down, left and right. In other cases, c ₅ = 0
c ₆ = 1 / (αd + β) However, the number of intersections of the four directions of up, down, left and right are all 2 or more, and the number of intersections of the lower position is the smallest of the number of intersections of the four directions of up, down, left and right. In other cases c ₆ = 0
c ₇ = 1 / (αl + β) However, the number of intersections of the four directions of up, down, left and right are all 2 or more, and the number of intersections of the left direction is the smallest of the number of intersections of the four directions of up, down, left and right. In other cases c ₇ = 0
For c ₀ , c ₁ , c ₂ , and c ₃ , if there are a plurality of azimuths with the number of intersections of 1, the element of the feature vector corresponding to the azimuth with the number of intersections of 1 has a value. For c ₄ , c ₅ , c ₆ , and c ₇ , when the number of intersections is 2 or more and there are a plurality of minimum value orientations, the element of the feature vector corresponding to the plurality of orientations with the minimum number of intersections is used. Will have a value.

When the pixel P is a pixel belonging to the background or the boundary line, c ₀ = c ₁ = c ₂ = c ₃ = c ₄ = c ₅ = c ₆ = c ₇ = 0.

  In addition, the above-mentioned rule is only an example and is not limited to this. Examples of the method for determining other elements of the feature vector include the following rule (a) or (b).

  (A) In the above rule, when there are a plurality of the same number of intersections, it is determined that all of the plurality of directions have feature vector elements. However, instead of doing so, priorities are set in advance in the top, bottom, left, and right directions, and when there are multiple directions with the same number of intersections, the direction with the highest priority among the directions with the same number of intersections Only the distance to the boundary line may be adopted. The other elements of the feature vector may be zero.

(B) Among the feature vectors, the components c ₀ , c ₁ , c _2, and c ₃ are the distances to the boundary line among the azimuths with the number of intersections of 1 when there are azimuths with the number of intersections of 1. May be determined so that the feature amount 1 / (α + β) using α having the smallest value is included in the element in the direction of α having the smallest distance to the boundary line. The components of c ₄ , c ₅ , c _6, and c _{7 in} the feature vector have _four directions when there is no direction where the number of intersections is 1 and the number of intersections is 2 or more in all directions. Among them, the feature amount 1 / (α + β) using α having the smallest distance to the boundary line may be included in the element of the direction of α having the smallest distance to the boundary line. Then, the elements of the feature vectors in other directions may be zero.

In the above examples, both c ₀ and c ₄ represent upward components. c ₁ and _{c 5} represents a component of the right orientation. c ₂ and _{c 6} represent the component of the lower position. c ₀₃ and c ₇ represent left-direction components.

  The components of the feature vector of the pixels existing in the background may all be zero.

  Note that the above feature vector is an example as already pointed out, and the feature vector for achieving the task may be in other forms. Further, for example, the feature vector may be a four-dimensional vector whose dimensions are four directions of up, down, left, and right. A function based on the distance to the boundary line of the azimuth with the least number of intersections in the four azimuths may be set as the dimension element of the corresponding azimuth, and the values of the elements of the other dimensions may be zero.

<1-4> Properties of Feature Vector The above feature vector has the following properties.

  In the first embodiment, it is assumed that the original image is a line image composed of lines. A target on which an original picture is drawn is called an object. The object is drawn with a plurality of line drawings, and includes a plurality of closed regions.

  The boundary line between the object and the background is more discriminating than the boundary line between the closed regions existing in the object. This is because the boundary line that touches the background can also be called the outline that makes up the silhouette of the object, and it is a boundary line that expresses the characteristics of the object more strongly than the line that exists inside the object. This is because the rule of thumb is known.

  When the number of intersections is 1, it indicates that the boundary line existing in the direction is on the boundary with the background. On the other hand, a boundary line existing in a direction where the number of intersections is 2 or more is a boundary line that has entered the object and is not in contact with the background.

  If at least one of the feature vectors defined above has an azimuth with one intersection, the distance to the boundary line in that azimuth is used with priority. When there are a plurality of azimuths with the number of intersections of 1, a feature vector component is generated using the shortest distance to the boundary line among the azimuths. If there is no azimuth with the number of intersections of 1, the distance to the boundary line having the smallest value among the azimuths with the number of intersections of 2 or more is adopted.

  Thus, the reason why the distance to the boundary line with the number of intersections of 1 is used preferentially is to place importance on the boundary line in contact with the background.

  If the distance to the adopted boundary is α, 1 / (α + β) is calculated, and the value of a predetermined vector component of the 8-dimensional vector is set, and the values of the other vector components are set to zero. Here, β is a constant. The reason why the constant β is added to α is set in order to prevent the value of the vector component from becoming infinite because α is zero on the boundary line. Further, it depends on how much the size of the closed region is targeted and how much a component having a large value near the boundary is given as a feature. Since the distance α to the boundary line exists in the denominator, the vector component 1 / (α + β) takes a larger value (increasing function) as the pixel is closer to the boundary line in the closed region. Thus, by placing α in the denominator, it is possible to give a larger feature amount to the portion of the closed region that is close to the boundary line. As described above, giving a larger feature amount to a portion close to the boundary line can give the feature so as to give more importance to the shape of the boundary line. This is because the feature of the shape appears near the boundary rather than near the center of the closed region of the object. By using this feature vector, the correspondence between the closed regions of the original images can be achieved using the cost function described later. This is to make it possible to estimate the attachment with a high probability. The feature quantity 1 / (α + β) is an example, and is not limited to this feature quantity. The feature amount is desirably a function that emphasizes the vicinity of the boundary of the closed region by selecting a function that takes a large value in the vicinity of the boundary within the closed region.

<1-5> Example of Feature Vector of Pixel (1) Feature vector of pixel P1 (FIG. 2A)
The number of intersections of the four directions of the pixel P1 is as follows.

  Since the pixel P1 intersects at U11 in the upward direction, the number of intersections in the upward direction is 1.

  Since the pixel P1 intersects at R11 in the right direction, the number of intersections in the right direction is 1.

  Since the pixel P1 intersects at D11 in the downward direction, the number of intersections in the lower position is 1.

  Since the pixel P1 intersects at L11, L12, and L13 in the left direction, the number of intersections in the left direction is 3.

In the pixel P1, the azimuth with one crossing is present in the upper direction, the right direction, and the lower direction. The distance from the upper boundary is α1U. The distance from the rightward boundary is α1R. The distance from the lower boundary is α1D. Therefore, the feature vector V (P1) of the pixel P1 is as follows.
V (P1) = (1 / (α1U + β), 1 / (α1R + β), 1 / (α1D + β), 0, 0, 0, 0, 0)
(2) Feature vector of pixel P2 (FIG. 2B)
The number of intersections of the four directions of the pixel P2 is as follows.

  Since the pixel P2 intersects at U21 and U22 in the upward direction, the number of intersections in the upward direction is 2.

  Since the pixel P2 intersects at R21 and R22 in the right direction, the number of intersections in the right direction is 2.

  Since the pixel P2 intersects at D21 and D22 in the downward direction, the number of intersections in the lower position is 2.

  Since the pixel P2 intersects at L21 and L22 in the left direction, the number of intersections in the left direction is 2.

In FIG. 2B, there is no azimuth having the number of intersections of 1, and the pixel P2 has the smallest intersection number value of 2 among the azimuths having the number of intersections of 2 or more. All three directions. The distance from the upper boundary is α2U. The distance from the rightward boundary line is α2R. The distance from the lower boundary is α2D. The distance from the left azimuth boundary line is α2L. Therefore, the feature vector V (P2) of the pixel P2 is as follows.
V (P2) = (0, 0, 0, 0, 1 / (α2U + β), 1 / (α2R + β), 1 / (α2D + β), 1 / (α2L + β))
(3) Feature vector of the pixel P3 (FIG. 2C)
The number of intersections of the four directions of the pixel P3 is as follows.

  Since the pixel P3 intersects at U31 and U32 in the upward direction, the number of intersections in the upward direction is 2.

  Since the pixel P3 intersects at R31 and R32 in the right direction, the number of intersections in the right direction is 2.

  Since the pixel P3 intersects at D31 in the downward direction, the number of intersections in the lower position is 1.

  Since the pixel P3 intersects at L31 and L32 in the left direction, the number of intersections in the left direction is 2.

In FIG. 2C, since there is an orientation with the number of intersections of 1, when calculating the characteristics of the pixel P3, the distance from the lower boundary line is α3D of the orientation with the number of intersections of 1. Is adopted. Therefore, the feature vector V (P3) of the pixel P3 is as follows.
V (P3) = (0, 0, 1 / (α3D + β), 0, 0, 0, 0, 0)
<1-6> Creation of management table in which closed areas constituting two original pictures are arranged in descending order As pre-processing for associating a plurality of closed areas included in original picture 1 and original picture 2 shown in FIG. A table may be created in which a plurality of closed regions are arranged in descending order of area. By making this table, it is possible to evaluate the correspondence from a closed area having a large area. The reason for making such a table is that it is better to determine the correspondence relationship early for a closed region with a large area and remove it from the correspondence candidate than to determine the correspondence relationship of the closed region with a smaller area first. This is because there is an empirical rule that it is easy to find the correspondence of the closed region and the accuracy of the correspondence becomes high. Therefore, it is meaningful to create such a table in order to check the correspondence relationship in the order of high possibility of correspondence.

  FIG. 3A is a management table in which the three closed regions (closed region 1A, closed region 1B, and closed region 1C) of original image 1 in FIG. 1 are arranged in descending order of area. No. Indicates the descending order of the closed area, and the closed area index is assigned an index (for example, 1A) labeled on the closed area. The index may automatically give a unique identification for each closed region. The number of pixels represents the number of pixels included in each closed region, and is a numerical value proportional to the area. Therefore, the number representing the area may be substituted. Note that the number of pixels may be used, for example, when sorting in descending order of the area of the closed region. In the “association” column, all zeros (0) are stored as initial values. For example, 1 is stored in the closed region where the association of the closed region to be described later is completed. The closed region in which 1 is stored indicates that the association process has been completed, and may be excluded from the objects of the subsequent association process. Further, as will be described later, a closed region in which only a part of the association is completed is not changed to “1”, but is left as 0 and becomes a target of the subsequent association processing. Good. An example of this process will be described later.

  FIG. 3B is a management table in which the closed regions in the original image 2 in FIG. 1 are arranged in descending order of area.

<1-7> Association of Closed Regions An example of estimating the correspondence of closed regions existing in the original image 1 and the original image 2 is shown below.

  As described above, when associating closed regions, as a rule of thumb, a closed region having a large closed region in one original image (for example, original image 1) may correspond to the other original image. It can be said that it is easy to find a closed area with high accuracy and the accuracy of the correspondence is high. Therefore, in the following, first, the closed area having the largest area in the original image 1 is extracted in order, and the degree of approximation is calculated using the cost function E illustrated below for the extracted closed area. For example, the smaller the cost function E is, the closer the approximation is, the movement vector that minimizes the cost function E (optimal movement vector) is obtained, and the closed movement extracted from the original image 1 is further obtained using the optimal movement vector. The closed region of the original image 2 having the largest overlap with the region may be found, and this combination may be estimated as a corresponding closed region pair. Alternatively, the pair of closed areas found in this way may be displayed to prompt the operator for confirmation. At this time, the operator may cancel the association of the pair by inputting that the estimation of the association is incorrect. In this case, the corresponding closed region with the next largest overlap may be found and displayed as a pair candidate. Alternatively, the next optimal vector may be used to find the closed region of the original image 2 having the largest overlap with the closed region of the original image 1 and may be displayed as a candidate for the next pair.

For example, the closed region 1A occupying the largest closed region among the _i closed regions D _i (three closed regions 1A, 1B, 1C) existing in the original image 1 in FIG. For all the pixels p included in the region 1A, the cost function E (D _i , δ) for the closed region A1 is calculated by Equation 2 while moving the movement vector δ over the original image 2 in FIG. 3B. Also good.

Here, p + [delta] is shifted original 2 and its background by the movement vector [delta], when superimposed on the closed region D _{i (e.g.} closed region 1A), the pixel p in the closed region D _{i (e.g.} closed region 1A) This is the position of the original image 2 or background that overlaps the position. V1 (p) is a feature vector of the pixel at the pixel position p of the closed region D _i (eg, the closed region 1A) of the original image 1. V2 (p + δ) is a pixel feature vector at the pixel position p + δ of the original image 2 and its background. When the original image 2 that overlaps the pixel position p of the closed region D _i (eg, the closed region 1A) of the original image 1 is inside the closed region (eg, inside the closed region 2A), the square of the difference between the feature vectors of the original image 1 and the original image 2 Is calculated and integrated (Equation 2 top). When the original image 2 that overlaps the pixel position p of the closed region D _i (eg, the closed region 1A) of the original image 1 is not inside the closed region (eg, background or boundary line), the lower calculation of Expression 2 is performed and the constant γ is accumulated. To do. The reason for adding γ is that, for the overlap with the background portion, adding a relatively large constant γ increases the cost function and gives a penalty. For γ, an appropriate value is selected depending on the total number of pixels in the original image or the number of pixels in the closed region. For example, a constant such as γ = 2 may be set.

  If the position of p + δ of the original picture 2 is not within the closed area, the lower calculation of Equation 2 is performed. For example, the closed area in the original picture 2 that has already been associated is treated the same as the background. As shown in FIG.

  The feature vector has already been shown in Equation 1.

The movement vector δ is a two-dimensional movement vector, for example, in units of pixels. The closed area D _i (for example, the closed area 1A) of the original image 1 is taken out, and the original image 2 and its background are moved by the movement vector δ, and both are moved. A vector used for superposition. Then, a two-dimensional movement vector δ ^* _i that minimizes the cost function E of Equation 2 is obtained. For example, while changing the value of δ between the closed region 1A, the original image 2, and the background thereof, δ ^* _i that minimizes the cost function E is obtained as follows.
^{_{δ * i = argmin δ E (}} D i, δ) ( Equation 3)
Here, argmin _δ E (D _i , δ) means obtaining an argument δ that minimizes the value of E (D _i , δ).

Then, for example, for one closed region D _i of the original image 1 (for example, the closed region 1A), the cost function E is calculated by superimposing the original image 2 and the background by changing δ comprehensively, and the cost function E is minimized. Thus, δ ^* _i that satisfies the above Equation 3 is obtained. Alternatively, in the course of exhaustive calculation of the cost function E, if the minimum value of the cost function E (minimum value during the calculation) is not updated within a predetermined number of calculations, an exhaustive calculation of δ is performed. The minimum value of the cost function obtained up to that point may be adopted. Alternatively, in the calculation of the cost function E for a closed area having a large area, for example, the cost function E is limited to every other pixel in the closed area of the original image 1 to reduce the calculation amount. The calculation speed may be increased.

When this movement vector δ ^* _i is used, the closed region of the original image 2 that maximizes the area (for example, the number of pixels) of the closed region between the closed region D _i (for example, the closed region 1A) and the original image 2 is can be estimated is a closed area of the original 2 corresponding to the closed region D _i of the original 1.

  Hereinafter, an example of the correspondence estimation will be specifically described with reference to FIG.

  In order to make the explanation easy to understand, an example in which the closed region of the original image 2 corresponding to the closed region 1C of the original image 1 is obtained will be described with reference to FIG.

  In FIG. 4A, the closed area 1C of the original image 1 is taken out and is superimposed on the original image 2 as it is. In this state, the cost function E (C1, δ) is obtained for the closed region C1. The initial value of δ is a zero vector. The movement vector δ is drawn so as to move 1C in FIG. 4, but is added to the coordinate position of the term of the original picture 2 in Equation 2. Since the movement vector δ is a vector that defines the relative positional relationship of superposition, in the figure, for the sake of intuitive understanding, the movement vector δ is drawn as a vector for moving the closed region extracted from the original image 1. Should be noted. The descriptions in Equation 2 and FIG. 4 are synonymous with respect to relative expressions.

Thereafter, the movement vector δ is comprehensively changed to obtain δ ^* that minimizes the cost function E (C1, δ) as shown in FIG. 4B.

  As shown in FIG. 4B, among the closed regions of the original image 2, the closed region of the original image 2 having the largest area overlapping with the closed region C1 is obtained. In this case, the closed region 2C and the closed region 1C overlap in the region S1. The closed region 2A and the closed region 1C overlap in the region S2. It can be seen that the largest of these overlaps is S2. Therefore, it is estimated that the closed region 1C sharing the largest overlapping region corresponds to the closed region 2C.

  In this way, the correspondence between the closed region 1C of the original image 1 and the closed region 2C of the original image 2 is estimated. This estimate may be displayed to prompt the operator for confirmation. Alternatively, the association may be confirmed without obtaining operator confirmation. If the operator inputs that the estimation of the association is wrong, a pair of the closed region 1C and the closed region 2A in the region with the next largest overlapping area (in the case of FIG. 4B). May be displayed to estimate the correspondence of the next candidate.

Alternatively, when the operator inputs that the estimation of the association is wrong, the area where the cost function overlaps with C1 is the most, using the movement vector δ whose cost function is the next smaller than the movement vector δ ^*. The closed area of the large original image 2 may be estimated and displayed as a candidate for the next closed area corresponding to the closed area C1 to prompt the operator to confirm.

  FIG. 5 shows a correspondence table of closed regions. Based on the above results, the correspondence between the closed areas existing in the original picture 1 and the original picture 2 is determined, and the correspondence table in FIG. 5 is stored. If coloring is performed in the original picture 1 using this correspondence table, the coloring in the original picture 2 can be automatically performed.

<1-8> Region Association Operation FIG. 6 is a flowchart illustrating an example of region association.

  In step S100, parameters i and j are initialized. The parameter i may be, for example, the order (No) when the closed regions of the original image 1 are arranged in descending order as shown in FIG. i may be any number that can uniquely identify each closed region of the original image 1. The parameter j may be the order (No) when the closed regions of the original image 2 are arranged in descending order of area as shown in FIG. 3B, for example. j may be any number that can uniquely identify each closed region of the original image 2.

  In step S102, one closed region i (for example, closed region 1A) is extracted from one image (original image 1).

In step S104, a cost function E (D _i , δ) is calculated from the closed region D _i and the other original image.

  In step S106, it is checked whether δ has been changed comprehensively. If the check is “No”, the process proceeds to step S132. If the check is “Yes”, the process proceeds to step S108.

  In step S132, δ is changed by a predetermined value.

In step S108, the movement vector δ ^* ₁ when the cost function E (D _i , δ) is minimized is stored.

In step S110, [delta] ^* ₁ by using the area overlapping the closed region D _i is estimated as the maximum of the other original closed region D _j regions corresponding to the closed region D _i a.

  In step S152, the correspondence of the closed area may be displayed to prompt the operator to confirm. If it is input that the correspondence is wrong, the next pair of candidates may be displayed. Further, the correct correspondence of the closed region may be input according to an instruction from the operator. At this time, an instruction for coloring may be prompted.

In step S154, of the closed region D _j closed region D _i and original 2 of the original 1 correspondence relationship is determined, may be excluded from the scope of the subsequent processing of the correspondence. This operation may use the association flag of FIG.

  In step S112, it is checked whether i = I. That is, it is checked whether or not the association processing has been completed for all the closed regions in the original image 1. If this check is “NO”, the process proceeds to step S134. If this check is “Yes”, the entire process may be terminated.

  In step S134, i is incremented. By this operation, the next closed region of the original image 1 can be extracted in step S102.

  The outline of the processing flow of the first embodiment is as described above.

<2> Embodiment 2: Association of intersections <2-1> Details of processing of association of intersections The following describes the association of intersections existing in the original image 1 and the original image 2, respectively. The information on the association of the intersections can be used to generate an image obtained by interpolating the original image 1 and the original image 2.

  FIG. 7 is a diagram illustrating an example of association of intersections existing in each of two images including a plurality of closed regions.

  It is assumed that the association of each closed region between the original image 1 and the original image 2 has been completed by the processing described above. FIG. 7C shows an enlarged view of the closed region surrounded by a circle in the original image 1 in FIG. FIG. 7D shows an enlarged view of the closed region surrounded by a circle in the original image 2 in FIG. It is already known that the closed region 2B corresponds to the closed region 1B, and the closed region 2C corresponds to the closed region 1C. In addition, the same background Y exists in the original picture 1 and the original picture 2.

  As already described, the plurality of closed regions existing around the intersection 13 in FIG. 7C exist counterclockwise in the order of the closed region 1B, the background Y, and the closed region 1C. The plurality of closed regions existing around the intersection 23 in D) exist counterclockwise in the order of the closed region 2B corresponding to the closed region 1B, the background Y, and the closed region 2C corresponding to the closed region 1C. Therefore, it can be seen that the pattern of the positional relationship of the closed region in contact with the intersection 13 matches the pattern of the positional relationship of the closed region in contact with the intersection 23. From this, it is possible to automatically estimate that the intersection 13 and the intersection 23 are in a correspondence relationship. In this way, by using the information on the associated closed region, it is possible to automatically estimate the correspondence of the corresponding intersection. This estimation result may be displayed on the screen to prompt the operator to finalize the correspondence. Alternatively, the correspondence relationship between the estimated intersections may be automatically determined.

  FIG. 8 is a diagram illustrating a result of associating each intersection of two images including a plurality of closed regions. The information on the intersections associated with each other can be stored by the intersection correspondence table shown in FIG.

  FIG. 9 is a diagram illustrating an example of a first problem that occurs when associating intersections existing in two images including a plurality of closed regions. In FIG. 9, it is assumed that the closed regions 1A and 2A, the closed regions 1B and 2B, the closed regions 1C and 2C, and the closed regions 1D and 2D are the corresponding closed regions.

  In this case, in the original image 1 in FIG. 9A, the closed regions 1A, 1D, and 1B exist around the intersection P11 in the counterclockwise direction. On the other hand, in the original image 2 in FIG. 9B, closed areas 2A, 2D, and 2C exist at the intersection P21. In this example, the intersection point P11 and the intersection point P21 are determined not to be corresponding intersection points because there are no corresponding closed regions in the same pattern around them. An intersection corresponding to the intersection P11 of the original picture 1 does not exist in the original picture 2.

  However, when the two intersections (intersection point P11 and intersection point P12) in FIG. 9A and FIG. 9B (intersection point P21 and intersection point P22) are regarded as a group, the surroundings of the intersection point P11 and the intersection point P12 of the original picture 1 are obtained. There are closed areas 1A, 1D, 1C, and 1B in the counterclockwise direction, and closed areas 2A, 2D, 2C, and 2B in the counterclockwise direction around the intersection P21 and the intersection P22 of the original image 2, and the correspondence relationship of the closed areas Considering this, it can be seen that closed regions exist in the same pattern.

  Also in the example of FIG. 9, it is possible to associate the intersections by performing the following processing.

  FIG. 10 is a diagram illustrating an example of processing for correcting intersections and boundaries existing in an image including a plurality of closed regions.

  In FIG. 9A, the original image 1 is selected. Instead of the original image 1, the original image 2 may be selected. As shown in FIG. 10A, one of the two intersections (intersection P11, intersection P12) existing in the selected original picture 1 is deleted. In the case of FIG. 10A, the intersection P12 is deleted. The intersection to be deleted may be any intersection. As a result, boundary lines Ldb, Lbc, and Lcd of the closed region remain. Of the intersections, the intersection having the smaller angle of the intersection (that is, the angle formed by Lcd and Lbc on the region 1C side at the intersection P12 is smaller than the angle formed by Lad and Lab on the region 1A side at the intersection P11. In this case, the intersection P12) may be deleted with priority.

  As shown in FIG. 10B, the boundary line Lbc is extended to the intersection P11 along Ldb. The intersection point of the position of the extended intersection point P11 of the Lbc is defined as an intersection point P111.

  As shown in FIG. 10C, the boundary line Lcd is extended along the Ldb to the intersection P11. The intersection point of the extended intersection point P11 of the Lcd is defined as an intersection point P112.

  Note that Ldb may be deleted after the above operation.

  As shown in FIG. 10D, as a result, the intersection point P12 is deleted, the boundary line Lbc is extended, and the intersection point P111 is set at the position of the intersection point P11. The boundary line Lbc is extended, and the intersection point P112 is set at the position of the intersection point P11. For this reason, it can be considered that the intersection P11 is divided into two, the intersection P111 and the intersection P112. Note that the extended boundary line Lbc and the extended boundary line Lcd are drawn apart from each other, but are drawn apart to make the figure easier to understand and actually exist on the same line. To do. The same applies to the other drawings.

  FIG. 11 is a diagram illustrating an example of association of intersections based on a result of processing for correcting intersections and boundaries existing in an image including a plurality of closed regions.

  In the process of FIG. 10 described above, in FIG. 11A, the intersection point P12 is deleted, the boundary line Lbc is extended, and the intersection point P111 is set at the position of the intersection point P11. The boundary line Lbc is extended, and the intersection point P112 is set at the position of the intersection point P11. Therefore, the intersection point P11 is divided into two points, an intersection point P111 and an intersection point P112.

  As shown in FIG. 11B, the original picture 2 has two intersection points P21 and P22 as they are. As shown in the intersection correspondence table in FIG. 11C, the intersection point P111 of the original picture 1 and the intersection point P21 of the original picture 2 are associated with each other. Then, the intersection point P112 of the original picture 1 and the intersection point 22 of the original picture 2 are associated with each other. Then, it is assumed that a zero-length boundary line exists between the intersection point P111 and the intersection point P112 in FIG. The intersection P111 is connected to the boundary line Lab, and the intersection P112 is connected to the boundary line Lad. The intersection P11 may be deleted.

  By performing the above processing, even for an intersection that could not be associated as shown in FIG. 9, by deleting one intersection of any of the original images and performing the above processing, the association of the intersection can be performed. Yes.

[Intersection point mapping]
As shown in FIG. 9, FIG. 9 shows a case where the intersections of the original picture 1 and the original picture 2 cannot be associated with each other, and when the two intersections are regarded as a group, they exist around them. It is a flowchart which discovers the case where the pattern of the several closed area to perform is the same. In order to help understanding, in the flowchart of FIG. 12 and the following description, the reference numerals shown in FIG. Note that the flowchart of FIG. 12 is not limited to the example of FIG.

  In step S300, two adjacent intersections (Ldb) existing on the boundary line (Ldb) of two closed regions (1D, 1B) among a plurality of points that cannot be associated with each other in one original image (original image 1) ( A set Q of a set q of P11, P12) is extracted.

  In step S302, two adjacent intersections existing on the boundary line (Lac) of the two closed regions (2A, 2C) among the plurality of points that cannot be associated with each other in the other original image (original image 2) A set R of a set r of (P21, P22) is extracted.

  Step S304 shows an iterative process in which the element q included in the set Q is extracted and processed one by one with the step S314.

  Step S306 indicates a repetitive process in which the element r included in the set R is extracted and processed one by one with the step S312.

  In step S308, a plurality of closed region arrangement patterns (1A, 1D, 1C, 1D in the counterclockwise direction) in contact with at least one of the two intersections (P11, P12) included in q are included in r. A pattern of a sequence of closed regions of one original image corresponding to each of a plurality of closed regions (2A, 2D, 2C, 2D) in contact with at least one of the intersections (P21, P22) (counterclockwise 1A, 1D, 1C, 1D) is checked.

  In step S310, it is determined whether or not the checks in step S308 match. If the checks match, the process proceeds to step S320. If the checks do not match, the process proceeds to step S312.

  In step S320, it is recognized that r corresponds to q, and element s is added to set S and stored as s = (r, q).

  By performing the above processing, the correspondence between two intersections as shown in FIG. 9 can be extracted.

  FIG. 13 is a flowchart showing a process of taking out the elements s one by one from the set having the two intersection points extracted in FIG. 12 as the element s and associating the intersection points. In order to help understanding, in the flowchart of FIG. 12 and the following description, the reference numerals shown in FIG.

  Step S400 indicates that the element s is taken out from the set S one by one, and the process is repeated with step S412.

In step S402, one intersection is deleted. Note that the flowchart of FIG. 12 is not limited to the example of FIG. A set is defined as shown below.
s∈S
s = (q, r)
q∈intersection P11, intersection P12 (intersection included in one original picture (original picture 1))
r∈intersection P21, intersection P22 (intersection included in the other original picture (original picture 2))
In step S402, one of the intersections (P11, P12) included in the intersection set (q) of one original image is deleted.

  In step S404, two boundary lines (Lbc, Lcd) connected to the deleted intersection (P12) and not connected to the remaining intersection (P12) are specified.

  In S406, each of the two specified boundary lines (Lbc, Lcd) is left along the boundary line (Ldb) connecting the deleted intersection point (P12) and the remaining intersection point (P11). Extend to (P11).

  In step S408, a first intersection (P111) is generated at the extended end of one boundary line (Lbc) of the two extended boundary lines (Lbc, Lcd), and the one boundary line (Lbc) is defined. The point (P21) on the boundary line (L2bc) of the closed region (2B, 2C) of the other original image (original image 2) corresponding to the two closed regions (1B, 1C) adjacent to the boundary is generated. 1 point of intersection (P111). The intersection point P111 is connected to the boundary line Lab. Similarly, a second intersection (P112) is generated at the extended end of the other boundary line (Lcd) of the two extended boundary lines (Lbc, Lcd), and the one boundary line (Ldc) is defined as the boundary. The point (P22) on the boundary line (L2cd) of the closed region (2D, 2C) of the other original image (original image 2) corresponding to the two closed regions (1D, 1C) adjacent to the second image is generated. Corresponding to the intersection (P112). The intersection point P112 is connected to the boundary line Lad.

  In step S410, the remaining intersection (P11) is deleted. Assume that two generated intersections (P111, P112) are connected by a zero-length boundary line. The boundary line Ldb may be deleted. The intersection P11 may be deleted.

  By performing the above processing, intersection matching processing can be performed on the pattern of FIG.

  FIG. 14 shows an example of generating a middle-split image N for two original images to which the processes of FIGS. 11, 12, and 13 are applied.

  As shown in the middle image N in FIG. 14, the intersection point P111 of the original image 1 corresponds to the intersection point NP111 of the middle image, and the intersection point P112 of the original image 1 corresponds to the intersection point NP112 of the middle image. Then, it can be seen that the boundary line having a length of zero in the original image 1 is displayed as the boundary line Lac between the intersection point NP111 and the intersection point NP112 in the intermediate image N. In this way, it is possible to automatically generate an intermediate image by associating intersections.

  Note that when the intermediate image N is generated, the shape of each boundary line may also be interpolated to generate a smooth intermediate image. Alternatively, the intermediate image N may be generated by the morphing technique using the corresponding intersection point, the corresponding boundary line, the corresponding closed region information, and the like. In many cases, it is possible to infer the correspondence between the closed areas and the intersections of lines and points that do not constitute the closed areas that exist in the two original images. Of the closed areas, lines, points, and the like that exist in the two original images, the operator may be prompted to specify the corresponding relationship for which the corresponding relationship cannot be estimated. Finally, for closed areas, lines, points, etc. for which the operator has not instructed and the correspondence is unknown, the interpolation results such as surrounding closed areas, lines, points, etc. You may determine the positional relationship in an image. As seen in the morphing technology, it is possible to generate an intermediate image by using the morphing technology, etc., even if the correspondence between all the components present in the two original images is not known. is there.

  FIG. 15 is an example similar to the example illustrated in FIG. 14 and illustrates an example in which a plurality of intermediate images are generated. Between the original image 5 and the original image 6, ten intermediate images N01 to N10 are generated along the time axis. Each of the original image and the intermediate image corresponds to one frame of the moving image. In the case of FIG. 15, 12 frames of the original image 5, 10 intermediate images, and the original image 6 are displayed in succession, and a smooth moving image is reproduced.

  FIG. 16 is a diagram illustrating an example in which an intersection point is created by giving an end point to a boundary line of a closed region without an intersection point. In the case of this example, one intersection (intersection P51 and intersection P61) is shown for each of the original picture 7 and the original picture 8. The number of intersections of the closed regions in each original image may be plural. In this case, first, two closed regions are placed on one plane. Find the Hausdorff distance between these two closed regions. Next, for all the points on the boundary line of one closed region, find each point whose distance from the point on the boundary line of the other closed region is the smallest, and the point of one closed region and the point of the other closed region Create a set of two-point pairs that correspond to. In each pair, a pair having a distance between two points equal to the Hausdorff distance is assigned to each closed region as a corresponding intersection. In addition, when there are a plurality of pairs of points where the distance between the two points is equal to the Hausdorff distance, an arbitrary pair may be selected and set as a corresponding intersection.

  In general, the Hausdorff distance dH can reach any point in the set B by proceeding at least the distance dH at any point in the set A when there are two sets A and B. Similarly, any point in the set B can reach any point in the set A by proceeding at least the distance dH. Such a distance dH is called a Hausdorff distance. In the case of FIG. 16, in the original image 7 and the original image 8 placed on the same plane, a plurality of pixels included in the original image 7 correspond to the set A, and pixels included in the original image 8 correspond to the set B.

  Note that the Hausdorff distance is not necessarily employed, and a point having a similar shape in the closed region may be employed as a corresponding intersection. Moreover, it is good also considering the points which exist on a similar pattern as a corresponding intersection using methods, such as pattern recognition.

  FIG. 17 is a diagram illustrating an example of intersection processing when one of two original images has an isolated cycle. In the original picture 9, there are isolated cycles and straight lines. In the original image 10A, a straight line and a circle are in contact with each other, and an intersection point P101 and an intersection point P102 exist.

  In this case, the intersection of the original picture 9 and the original picture 10A cannot be associated. Therefore, it is desirable to do the following.

  That is, as shown in the original image 10B, the intersection point P101 and the intersection point P102 are deleted. Then, the straight line connected to the intersection point P101 and the straight line connected to the intersection point P102 are extended to form one straight line. Furthermore, the end point of the circle connected to the intersection point P101 and the end point of the circle connected to the intersection point P102 are connected to create one circular figure. Next, a new intersection is provided in the original picture 9 and the original picture 10B using the method shown in FIG. For example, an intersection point P093 and an intersection point P103 are provided. In this case, as described with reference to FIG. 16, when two circular figures are placed on a plane, an intersection may be given using the Hausdorff distance between these two circles. .

  FIG. 18 is a diagram illustrating an example in which an original image is divided into layers and processing is performed. The original picture 11 has an upper part 11A of the whistle and a lower part 11B of the whistle. The original picture 12 also has a whistle upper part 12A and a whistle lower part 12B. In the vicinity of such an image of the flute portion, there are cases where the intersection of the two original images cannot be simply associated. That is, there may be a closed region including an intersection that cannot be correlated even by the process of associating the intersection illustrated with reference to FIGS. 7 and 9. For such a closed region, it is effective to cut out the closed region and move it to another layer (layer), and perform the process of associating intersections separately for each layer.

  That is, the whistle part of the original picture 11 and the original picture 12 is copied to different layers, and the whistle image part is deleted from the original picture. When this operation is performed, the original picture is divided into the human part layer and the whistle part layer, and the process of associating the intersections can be performed. As a method for determining whether or not to divide into layers, it is effective to move a closed region where many patterns in FIG.

  Note that it is desirable to move a part of the closed area to another layer and delete the image transferred to the other layer from the original image. At this time, there may be a case where a portion around the deleted portion becomes blank and a portion where no closed region exists is generated. That is, there may be a branch line in which the closed area is opened, and the blank area of the remaining image obtained by moving a part of the closed area to another layer does not exist. In this case, the branch line may be left as it is, the end point (end point) of the branch line may be specified, and the end points may be associated with each other in the original image and the other original image. Alternatively, when an intersection that cannot be associated remains, the operator may be prompted to give an instruction.

Alternatively, in the above case, it is possible to form the closed region by performing a process of extending the line segments existing around the portion where the closed region is deleted and joining them together. When dividing an original image into layers and associating intersections, it may be necessary to perform a process of associating a newly generated closed region. This case can be dealt with in many cases by combining the closed regions that existed before moving to the layer. If it is not possible to cope with it, it is also effective to partially perform the association of the closed areas again.
<3> Third Embodiment: Modification Example of Formation of Middle-Split Image As described above, by associating the closed regions of the two original images and associating the intersections in order, the association of each part of the two original images Can be done.

  An intermediate image can be created by interpolating the position of the intersection of two original images. However, considering the correspondence of the closed region, the shape of the boundary of the closed region, the correspondence of the boundary connecting the corresponding intersections, etc. Thus, comprehensively interpolating the original image is advantageous in creating a more natural intermediate image.

  Further, by associating the operator with a coloring instruction when associating the images in the closed area, it is possible to prompt the operator to perform the coloring process and the confirmation of the closed area together.

  For example, the closed areas may be associated with the original image 1 and the original image 2 and then the closed areas of the original image 2 and the original image 3 may be associated with each other in order. In this case, if the operator instructs the coloring when the original image 1 and the original image 2 are associated with each other in the closed region, the coloring when the original image 2 and the original image 3 are associated with each other in the closed region can be performed substantially automatically. It becomes. In addition, since the coloring of the intermediate image can be easily estimated from the coloring of the original image, the coloring of the intermediate image can be performed almost automatically.

<4> Embodiment 4 (modified example of associating closed regions)
An example of processing when two closed regions of one original image correspond to one closed region of the other original image will be described below with reference to FIG.

  In FIG. 18, the closed region 110 and the closed region 111 in the original image 11 correspond to the closed region 121 in the original image 12. In this case, the association of the closed areas is in a many-to-one format. Such a situation is likely to occur in the case of an original image in which many objects overlap in front of the closed region. In order to deal with such a case, for example, the following processing may be performed.

  First, the closed region of the original image 12 corresponding to the closed region 110 of the original image 11 is estimated in the above-described first embodiment. With this estimation, 121 is estimated as a closed region corresponding to the closed region 110. At this point, the corresponding closed area 110 and closed area 121 are displayed on the screen to prompt the operator to confirm. The operator knows that the closed region 111 of the original image 11 remains as the closed region association. For this reason, the estimation enters that it is partially correct. In this input, the operator may instruct that the area 110 of the original image 11 is excluded from the corresponding target of the subsequent closed area, and the closed area 121 is not excluded from the target of the subsequent closed area. This instruction can be realized by the flag processing in the correspondence field of the table shown in FIG. That is, the association flag of the area 110 is set to 1, and is excluded from the subsequent association targets. On the other hand, in the association column of the area 121, the flag may be left as 0 so that it is taken into consideration in the subsequent association. At this time, the operator may also instruct coloring. Although the closed region 110 and the closed region 121 are colored with the same color according to the coloring instruction of the operator, the closed region 121 can hold a state as an estimation target of the closed region.

  Thereafter, when the process of associating the closed area 111 of the original image 11 is performed, it is estimated again that the closed area 121 of the original image 12 corresponds. When the instruction is prompted, the operator inputs that the correspondence is correct. According to the instruction of the operator, the association flag of the closed area 111 and the association flag of the closed area 121 shown in FIG. 3 are both replaced from 0 to 1, and these closed areas are the following closed areas. It is excluded from the target of the process of estimating the association.

  If the association field in FIG. 3 is not used, one-to-many, many-to-one, and many-to-many closed regions can be allowed to be associated as closed regions. Also in this case, the operator may be prompted to input an instruction as to whether or not the association is correct.

<5> Hardware Configuration FIG. 19 is a diagram showing a hardware configuration 1700 of the present embodiment.

  The hardware configuration 1700 includes a CPU 1710, a memory 1720, a communication control unit 1730, an input / output interface 1740, a display control unit 1750, a drive control unit 1760, and a print control unit 1770.

  The communication control unit 1730 is connected to a network such as the Internet. An input / output device 1742 is connected to the input / output interface 1740. A display device 1752 is connected to the display control unit 1750. The drive control unit 1760 can read and write the storage medium 1762. A printer 1772 is connected to the print control unit 1770.

  The storage medium 1762 may be a RAM, ROM, CD-ROM, DVD-ROM, hard disk, memory card, or the like.

  The method of the embodiment described above can be executed by a computer. In addition, the method of the embodiment may be implemented as a program that is executed by a computer. Part or the front of the program may be executed by the operating system. A part of the program may be realized by hardware. The program may be stored in the storage medium 1762.

  In the above-described embodiment, the method steps or the program steps may be executed simultaneously or in a different order as long as there is no contradiction.

  The above embodiments can be implemented as a hardware device.

  It goes without saying that the above embodiment does not limit the invention described in the claims, but is treated as an example.

  Note that the closed region and the background included in the original image and the intermediate image are examples of regions. The original image, the middle-split image, and the background are examples of elements constituting the image. The boundary line can be an element constituting a part of the outline of the closed region.

Lab boundary line Lac boundary line Lbc boundary line Lcd boundary line Ldb boundary line Lde boundary line P12 intersection point P112 set at the position of the deleted point P111 intersection point set at the position of P11

Claims (8)

By comparing the plurality of closed regions included in one image with the plurality of closed regions included in the other image, a second closed region corresponding to the first closed region of one image is obtained. A method in which a computer specifies from the other image, wherein each of the one image and the other image is composed of a plurality of pixels, and each of the plurality of pixels has a feature amount, and the method Is
From the feature amount of each of the plurality of pixels included in the first closed region and the feature amount of each of the plurality of pixels included in the other image, the first closed region and the other Obtaining a degree of approximation with a plurality of closed regions included in the image;
Identifying one closed region indicating that the degree of approximation is the closest among the plurality of closed regions included in the other image as the second closed region;
Have
The method is a method in which the feature amount is a function of a distance from a boundary line of a closed region including a pixel to the pixel.   The method according to claim 1, wherein the feature amount is a function of a distance from a point where a half line extending from a pixel in a predetermined direction intersects a boundary line of a closed region including the pixel to the pixel.   3. The method according to claim 2, wherein the feature amount is a function of a distance corresponding to an azimuth having the smallest number of intersections at which a plurality of the half-lines of the predetermined azimuth intersect the boundary line of the closed region.   The method according to claim 3, wherein the feature amount is a vector having a dimension corresponding to a combination of the plurality of predetermined azimuths and the number of intersections.   The method according to claim 1, wherein the distance function is a function that does not have a negative value and increases as the distance decreases.   The degree of approximation is the difference in feature amount between overlapping pixels in a portion where the first closed region and the other image overlap when the first closed region and the other image are overlapped. The method according to claim 1, wherein a sum of absolute values of or a square sum of the differences is used.   The method according to any one of claims 1 to 6, wherein the first closed region and the second closed region are displayed as mutually corresponding closed region candidates, and an operator is prompted to confirm.   The program which makes a computer perform the method of any one of Claims 1 thru | or 7.