JP2011170541A - Image deformation method, image processor, and computer program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve the accuracy of deformation, and to reduce the amount of calculation more than that of a conventional image deformation method. <P>SOLUTION: In an image deformation method, a movement amount of each apex in a control grid for deforming an image is acquired by a reference point set in an original image before deformation and a target point indicating a movement destination of the reference point, and the control grid in which the movement amount of each apex has been acquired is used to deform the original image. The method includes: a part setting step (step S1) of setting a plurality of parts in the original image; and a grid interval setting step (step S3) of setting a grid interval of the control grid with respect to each of the plurality of parts. The image deformation of each of the plurality of parts is performed using the control grid having the grid interval set in the grid interval setting step. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image deformation method, an image processing apparatus, and a computer program.

  In recent years, the need for image transformation has been increasing in many video works and a wide range of research fields. In this regard, many methods such as mesh warp, field morphing, radial basis function, and B-spline transformation (free-form deformation method) are used. Proposed.

  Among them, the free-form deformation method based on the B-spline transformation is a very useful deformation method that enables natural and smooth deformation, although some deformation errors occur. Further, a technique called multi-level B-spline conversion has been proposed as a technique for reducing the deformation error (see Non-Patent Document 1).

S. Lee, G. Wolberg, S.Y. Shin, “Scattered Data Interpolation with Multilevel B-spline”, IEEE Transaction on Visualization and Computer Graphics, Vol. 3, PP.228-244, 1997

An object of the present invention is to improve the deformation accuracy over the conventional image deformation method.
Another object of the present invention is to reduce the amount of calculation compared to the conventional image deformation method.

(1) The present invention acquires a movement amount of each vertex in a control grid for deforming an image from a reference point set in an original image before deformation and a target point indicating a movement destination of the reference point, An image deformation method for deforming the original image using a control grid from which the amount of movement of the vertex is acquired, a part setting step for setting a plurality of parts in the original image, and each of the plurality of parts A lattice interval setting step for setting a lattice interval of the control lattice, and performing image deformation of each of the plurality of parts using the control lattice of the lattice interval set in the lattice interval setting step, This is an image transformation method.

  According to the present invention, image deformation can be performed using a control grid in which a grid interval is set for each part.

(2) In the lattice interval setting step, it is preferable to determine the lattice interval according to the interval of the reference points included in the part for which the lattice interval is to be set.

(3) In the lattice interval setting step, it is preferable to determine the lattice interval in the lattice direction according to the maximum interval in the lattice direction of adjacent reference points in the part.

(4) In the lattice interval setting step, it is preferable to determine a value substantially equal to the maximum interval as the lattice interval in the lattice direction.

(5) For locations where the error between the position of the reference point and the position of the target point after moving by the image deformation using the control grid of the set grid spacing is greater than a predetermined threshold, Image deformation can be performed again using a control lattice having a lattice interval smaller than the lattice interval.
In this case, since calculation of a portion with a small error can be omitted, an increase in processing time can be suppressed even if image deformation is performed again.

(6) Viewed from another viewpoint, the present invention provides a moving amount of each vertex in a control grid for deforming an image from a reference point set in the original image before deformation and a target point indicating the movement destination of the reference point. And an image processing device that transforms the original image using the control grid from which the movement amount of each vertex is acquired, and a part setting unit that sets a plurality of parts in the original image; Grid interval setting means for setting a grid interval of a control grid for each of a plurality of parts, and image deformation of each of the plurality of parts is performed using the control grid of the grid interval set by the grid interval setting means An image processing apparatus configured to be performed.

(7) The present invention from another viewpoint is a computer program for causing a computer to function as the image processing apparatus.

  According to the present invention, the lattice spacing of each of the plurality of parts can be made different so that the image can be deformed for each part. Therefore, the deformation accuracy can be improved.

It is a figure which shows a control grid. It is a conceptual diagram of an image deformation process. It is a figure which shows the control grid used for the conventional multi-level B-spline conversion. It is a functional block diagram of an image processing apparatus. It is a figure which shows parts. It is a flowchart of an image deformation process. It is a figure which shows the lattice spacing determination method of a control lattice. It is a figure which shows the lattice spacing determination method of a control lattice. It is explanatory drawing of the movement of a reference point, and an error. It is a figure which shows the example of the lattice division | segmentation in an adaptive lattice method. It is a figure which shows the influence area | region of a reference | standard point, and an actual deformation | transformation area | region. It is a chart which shows the unnatural rate of a shape.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.
Hereinafter, first, basic B-spline conversion and multi-level B-spline conversion described in Non-Patent Document 1 will be described, and then the configuration and processing method of the image processing apparatus according to the present embodiment will be described.

[1. Basic B-spline conversion]
When deforming the shape of the original image, it is necessary to perform coordinate transformation uniquely from the point p (x, y) as the deformation source to the point p ′ (x ′, y ′) as the deformation destination. This coordinate transformation can be expressed by two deformation functions x ′ = f _x (x, y) and y ′ = f _y (x, y) for each of the x and y coordinate axes.
In the B-spline conversion, image transformation is realized using a B-spline basis function as the conversion function.

In the orthodox B-spline transformation, deformation is performed using a control grid as shown in FIG. Specifically, a movement amount is assigned to each vertex (hereinafter referred to as “control point”) in the grid, and a unique shape deformation is realized in the entire original image using this.
In FIG. 1, the original image is divided into m × n, and a pseudo local region is arranged around the original image. In the entire original image, the number of control points is (m + 3) × (n + 3).

Image deformation is performed so that the reference point set on the original image moves to the position of the target point as shown in FIG.
The amount of movement of each control point in the control grid is determined based on the positions of the reference point and the target point.

  In the B-spline transformation, first, the movement amount of 16 control points existing in the vicinity of a certain local region is used to deform the local region existing in the center. In the control lattice shown in FIG. 1, when the deformation of the local region A in the range of m = 0 to 1 and n = 0 to 1 is performed, 16 exists in the region near the local region A (shaded portion in FIG. 1). The deformation process of the local region A is performed using the movement amount of each control point.

  Then, by repeating the above deformation processing for all local regions in the original image, unique shape deformation is realized for the entire original image.

Let Ω = {(x, y) | −1 ≦ x ≦ m + 1, −1 ≦ y ≦ n + 1} be a lattice plane arranged in the xy coordinate space. Then, transformation is performed using a set P = {(x, y, z)} of points scattered in the xyz three-dimensional space. Here, {(x, y)} represents a point on Ω, and {z} represents the amount of movement of the pixel of interest at this point.
In order to realize the interpolation of the point group P at all coordinates in the image, the B-spline transformation defined by the control point group Φ existing on Ω is used as the deformation function f.

Here, φ _ij is a weight parameter of the ij-th control point in Φ. The deformation function f can be expressed as follows by using the weight parameter of the control point.

B _k and B _l represent the following B-spline basis functions.

In performing image deformation, a reference point group {(x ₁ , y ₁ ), (x ₂ , y ₂ ),..., (X _n , y _n )} and a target point group {(x _n , y _n )} The coordinates of x ₁ ', y ₁ '), (x ₂ ', y ₂ '), ..., (x _n ', y _n ')} are acquired in advance.
The weight parameters (φ ^x _kl and φ ^y _kl ) in the direction of each coordinate axis of x and y given as parameters to each control point can be obtained by minimizing the cost function shown below.

  Here, the regions affected by the movement of each reference point are nine local regions near the point. If the areas affected by multiple reference points overlap, the control points that exist in those areas will be affected by the influence of each reference point, and this area may undergo natural shape changes. It becomes impossible. Therefore, in order to realize an accurate shape change even when a plurality of reference points exist in the vicinity, it is necessary to solve this problem.

Let φ _ij be a weight parameter to be obtained at the ij-th control point on Φ (weight parameter used for actual deformation), and φ ^c _ij be a weight parameter given individually from the reference point c at the ij-th control point. The weight parameter φ _ij to be obtained can be obtained by minimizing the following cost function.

  The B-spline transformation realized by the above formula realizes very smooth and natural deformation, but the deformation result has some deformation error.

[2. multi-level B-spline conversion]
The B-spline transformation described above can be smoothly and naturally deformed, but the deformation includes some deformation error (error with respect to the target point). This can be reduced by performing deformation using a small-sized grid, but in that case, local rapid deformation occurs, which is far from natural and smooth deformation. In the conventional B-spline conversion, it is difficult to achieve both the accuracy of deformation and the naturalness / smoothness.

  In the aforementioned Non-Patent Document 1, a technique called multi-level B-spline conversion is proposed as a technique for reducing deformation errors while maintaining the naturalness and smoothness of deformation.

This approach deformation performed using a plurality of control grid _{Φ 1, Φ 2, ···,} Φ h. Of these, Φ ₁ is a control lattice used in the conventional B-spline transformation, and Φ ₂ is a control lattice having a lattice interval smaller than Φ ₁ . Although the reduction ratio of the lattice spacing can be determined arbitrarily, here, a 1 / 2-fold lattice is used in order to simplify later calculations.
Here, when Φ _k is a control lattice having (m + 3) × (n + 3) lattice vertices (control points), Φ _{k + 1} has (2m + 3) × (2n + 3) lattice vertices. [Phi arrangement of ij-th control points in _k is, [Phi _{k + 1} in the (2i, 2j) coincide with the arrangement of the first control point.

In order to perform multi-level B-spline conversion, first, normal B-spline conversion using a coarse control lattice Φ ₁ is performed as shown in FIG. Then, the error generated here is compensated by using a deformed lattice Φ ₂ having a lattice interval smaller than Φ ₁ as shown in FIG. Actually, an error at the time of deformation using a coarse grid is set as a new movement amount, and the deformation using a one-step fine grid is repeated. This makes it possible to reduce the error each time the deformation is repeated.

Since the B-spline transformation is smoothly deformed so as to be drawn by the amount of movement, the error does not increase from the previous stage by repeating the deformation. However, the calculation time increases according to the number of deformations.
The multi-level B-spline transformation is a very useful deformation method, but requires a lot of calculation time. There is also room for improvement in deformation accuracy.

[3. Configuration of image processing apparatus and processing method thereof]
The image processing apparatus according to the present embodiment further improves the deformation accuracy over the conventional multi-level B-spline conversion and achieves a reduction in conversion calculation time.
The image processing apparatus 1 according to the present embodiment is configured by installing in a computer a computer program that executes image deformation processing described below. Various functions in the image processing apparatus 1 are performed by this computer program. It has been realized.
The computer includes an arithmetic processing unit, a storage unit, an input / output device, and the like.

  FIG. 4 shows functional blocks of the image processing apparatus 1. The image processing apparatus 1 functions as a reference point setting unit 11, a target point setting unit 12, a parts setting unit 13, a lattice interval setting unit 14, an image deformation processing unit 15, an error determination unit 16, and a lattice interval adjustment unit 17. have.

The reference point setting unit 11 is for setting a plurality of reference points (see FIG. 2B) that are deformation references on an image (original image) that is a deformation target. The reference point setting unit 11 may receive information indicating the position of the reference point from the user and store the position of the reference point in the storage unit, or may receive reference point information given from other software. May be stored in the storage unit.
Note that the reference point can be set at an arbitrary position of a deformation target (for example, a human face) in the image.

  Arrangement intervals between adjacent reference points are set so as to be substantially aligned within one part. Although it is not necessary for the arrangement intervals to be completely coincident with each other, it is easy to set an appropriate lattice interval of the parts by being substantially aligned.

  The target point setting unit 12 is for setting a target point (see FIG. 2B) indicating the destination of the reference point for each of the set reference points. The target point setting unit 12 may also receive information indicating the position of the target point from the user and store the position of the target point in the storage unit, or may receive target point information given from other software May be stored in the storage unit.

The parts setting unit 13 is for dividing and setting a region to be deformed in the original image into a plurality of parts (regions). For example, when the region to be deformed is a human face, the plurality of parts include, for example, regions (eyes, eyebrows, nose, mouth, contour) surrounded by dotted lines in each figure in FIG. Can be set. One part may be composed of a plurality of regions separated from each other.
The part setting unit 13 may be a part for setting a region to be a part by an input from a user, or may be a part that automatically cuts out a part by image processing on an original image.

  The lattice interval setting unit 14 is for setting the lattice interval of the control lattice used for image deformation of each part set by the part setting unit 13. In the lattice interval setting unit 14, an appropriate lattice interval is set for each of a plurality of parts.

  The image deformation processing unit 15 performs image deformation processing of the original image using the control grid having the grid interval set by the grid interval setting unit 14. The image deformation processing unit 15 performs image deformation for each part by B-spline conversion using a control lattice having a different lattice interval for each part. Since a control lattice having a different lattice interval is used for each part, appropriate deformation for each part can be realized.

  The error determination unit 16 is for determining an error between the position of the reference point and the position of the target point after moving by image deformation using a control grid set for each part. This error determination is performed for each of a plurality of reference points.

  The lattice interval adjustment unit 17 has a lattice smaller than the lattice interval of the control lattice used in the previous image deformation at a location where a reference point having an error calculated by the error determination unit 16 larger than a predetermined threshold exists. This is for setting the interval (for example, a lattice interval that is ½ of the previous lattice interval). For a portion (reference point) adjusted by the lattice interval adjustment unit 17 so that the lattice interval is reduced, the image deformation processing unit 15 uses the newly set lattice lattice control lattice to perform image deformation again. I do.

  FIG. 6 shows image deformation processing by the image processing apparatus 1. First, a setting process for a part to be deformed is performed (step S1). In the part setting process, the part setting unit 13 performs a process of receiving an input for designating an area to be a part from the user. In the later-described image deformation (step S4), an independent image deformation process is performed for each part.

  Subsequently, processing for setting a reference point and a target point to which the reference point is moved is performed in each part (step S2). In the reference point and target point setting process, the reference point setting unit 11 and the target point setting unit 12 perform a process of receiving an input for designating the reference point and the target point from the user. Note that the execution order of steps S1 and S2 may be reversed.

  Then, the process which sets the lattice space | interval of the control lattice applied for every set part is performed (step S3). In the lattice interval setting process, the lattice interval setting unit 14 determines the lattice interval according to the interval between the reference points included in each part. That is, if the arrangement of the reference points in the part is dense, the grid interval is reduced, and if the arrangement of the reference points in the part is sparse, the lattice interval is increased.

In the present embodiment, the lattice interval setting unit 14 determines the lattice interval by using the maximum interval in the lattice direction of adjacent reference points in the part.
Here, it is assumed that a plurality of reference points C in a certain part P are arranged as shown in FIG. Further, it is assumed that the X-axis direction and the Y-axis direction shown in FIG. 7A coincide with the respective lattice directions of the control lattice (in the two-dimensional case, the vertical direction and the horizontal direction).
In this case, the lattice spacing setting unit 14 determines two reference points C _X1 and C _X2 which are adjacent to each other in the X-axis direction among the plurality of reference points C (see FIG. 7B). ). An interval X between these reference points C _X1 and C _X2 is the maximum interval in the X axis direction between adjacent reference points in the part P.

Further, the lattice interval setting unit 14 determines two reference points C _Y1 and C _Y2 that are adjacent to each other in the Y-axis direction among the plurality of reference points C (see FIG. 7B). The interval Y between these reference points C _Y1 and C _Y2 is the maximum as the Y-axis direction interval between adjacent reference points in this part P.

The lattice interval setting unit 14 uses the maximum interval X in the X-axis direction as the lattice interval in one lattice direction (lateral direction), and the maximum interval Y in the Y-axis direction as the lattice interval in the other lattice direction (vertical direction). Set (see FIG. 7C).
As described above, the lattice interval of the control lattice for a certain part is determined. The lattice spacing determined in this way is appropriate for each part, and is suitable for performing appropriate image deformation for each part.
The reason why the method of determining the lattice spacing as described above is preferable will be described later.
In the present embodiment, the maximum intervals X, Y and the lattice interval are completely matched, but may be slightly different.

  The lattice spacing setting process in step S3 as described above is performed for each of the set plurality of parts. Therefore, the grid interval is set for each part.

Further, as shown in FIG. 8 (a), a first reference point group responsible for deformation of a certain part P1 and a second reference point group responsible for deformation of another part P2 are present close to each other. In such a case, it is difficult to perform deformation by setting different lattice intervals for each part.
Therefore, for a plurality of parts P1 and P2 in which the reference point group is close, a grid interval common to the plurality of parts P1 and P2 is set.
That is, among the reference point groups in both parts P1, P2, the maximum interval in the X-axis direction is the lattice interval in one lattice direction (lateral direction), and the maximum interval in the Y-axis direction is the other lattice direction (vertical direction). ).

More specifically, as shown in FIGS. 8B1 and 8B2, for each of the parts P1 and P2, the lattice spacing is determined by the same method as shown in FIG.
Thereafter, for the parts P1 and P2 in which the reference point groups are close to each other, as shown in FIG. 8C, the respective lattice intervals are compared, and a larger lattice interval is obtained in each of the X-axis direction and the Y-axis direction. , And determined as a grid interval common to both parts P1 and P2 (see FIG. 8D).
In the case of the control grid of FIG. 8D, the grid spacing in the X axis direction is the grid spacing in the X axis direction of the part P2, and the grid spacing in the Y axis direction is the grid spacing in the Y axis direction of the part P1. ing.

  Subsequently, image deformation of each part (eyes, eyebrows, nose, mouth, contour, etc.) is performed using a control grid having a grid interval set for each part (step S4). In the present embodiment, even for one image, local deformation for each part is performed independently (hereinafter, a method of deforming for each part is referred to as “partial deformation method”). Therefore, it is possible to use a control grid having a different grid interval for each part. In addition, about the part which a reference point group adjoins, a deformation | transformation is collectively performed by the integrated common control grid.

  After the image deformation in step S4, an error is determined for each movement of each reference point accompanying the image deformation (step S5). In the error determination process, as shown in FIG. 9, the error determination unit 16 obtains a difference between a position after a certain reference point is moved by image deformation using a control grid and a target point. Further, the error determination unit 16 compares the difference (error) with a predetermined threshold value.

  For a portion (reference point) where the error is larger than a predetermined threshold value, image deformation is performed again using a control grid having a grid interval smaller than the set grid interval (step S6).

Here, in the conventional multi-level B-spline transformation, first, deformation is performed using a coarse control grid as shown in FIG. 3A, and then the entire image is changed to FIG. 3B. As shown in the figure, a control grid having a fine grid interval is used to perform image deformation again.
In the multi-level B-spline conversion, image deformation by a control lattice having a finer lattice interval is repeated, so that the deformation error can be reduced, but the amount of calculation increases because image deformation of the entire image is repeated.

  Therefore, in the partial deformation method in which image deformation is performed for each part, when the conventional multi-level B-spline conversion is simply applied, the above-described problem that the image deformation is performed again for the entire part and the amount of calculation increases. Will remain.

  Therefore, in the present embodiment, image deformation using a control lattice having a finer lattice interval (hereinafter referred to as “adaptive lattice”) is applied only to a portion (reference point) where the deformation error is larger than a predetermined threshold. The process is speeded up (step S6).

  In the adaptive grid method, conversion with a control grid with a relatively large grid spacing (low level conversion) can be performed at a location where deformation error is small, and conversion with a control grid with a relatively small grid spacing (high level) is performed only at a location with large deformation error. Level conversion). Thereby, in the case of the image deformation again, the area for calculation for the image deformation can be reduced, and unnecessary repeated calculation can be reduced.

  FIG. 10 shows how to reduce the lattice spacing in the adaptive lattice method. After the first-stage image deformation (step S4) is performed on a certain part using the control lattice of the size shown in FIG. The lattice spacing is reduced (for example, reduced by 1/2). FIG. 10B shows a control lattice in which the lattice spacing is partially reduced due to the second stage image deformation.

  Then, the image deformation processing unit 15 sets the position of the reference point after the movement by the first-stage image deformation (step S4) as a new reference point for a portion with a large deformation error, and uses the new reference point to the target point. Perform image transformation to move.

  Such a second image deformation is repeated for all the reference points until the deformation error becomes smaller than a predetermined threshold value. Therefore, in the second stage of image deformation, the lattice spacing is further reduced as shown in FIG. In the adaptive lattice method, even if the reduction of the control lattice is repeated, the area that needs to be calculated becomes smaller each time the control lattice is repeated, so that an increase in the amount of calculation can be suppressed.

In the adaptive grid method, from the viewpoint of error reduction, the same processing as the multi-level B-spline conversion is performed, and therefore, good image deformation accuracy can be obtained as in the multi-level B-spline conversion.
Moreover, in the adaptive grid method, the multi-level processing is not performed on the entire image as in the multi-level B-spline conversion, but the multi-level processing is performed only on the portion where the deformation error is large. Can be reduced.

In the present embodiment, the processing after calculation of the movement amount of each pixel such as pixel operation is omitted for a portion where the deformation does not occur (a portion where the calculated moving distance is less than one pixel). .
For example, as shown in FIG. 11 (a), when there are two reference points C and C, the area where the calculation for deformation is performed (the influence area of the reference point C) is the filled range in FIG. 11 (a). (Peripheral area including the reference point).
However, at a position away from the reference points C and C, the calculated movement amount may be less than one pixel. For such a range, even if a calculation for an image moving operation (image deformation process) is actually performed, the image is not deformed as a result.

Therefore, in the present embodiment, for each pixel, the calculated movement amount is compared with a predetermined threshold value (for example, 0.5 pixel), and when the movement amount is less than the threshold value, no apparent deformation occurs. And the pixel movement operation process is omitted.
The filled range in FIG. 11B indicates a range where the movement operation process is actually performed. Here, in the pixel movement operation process, the calculated movement distance is rounded off, so that when the movement amount is equal to or greater than the threshold value (0.5), movement for one pixel occurs. .

[4. Consideration of reference point arrangement and grid spacing]
When the reference points are arranged on the deformation target, the arrangement interval of the reference points changes according to the shape of each part in order to realize high-precision deformation. When deformation using a control grid of the same size is performed on a plurality of parts having different fineness, the lattice spacing of the lattice is not necessarily optimal for deformation, and causes deterioration in deformation accuracy.

The cause of the deterioration of the deformation accuracy is roughly classified into a case where the lattice spacing is too large and a case where the lattice spacing is too small.
When the lattice interval is too large, problems such as (i) occurrence of deformation in an unintended region and (ii) deterioration of deformation accuracy occur.

When the lattice spacing of the control lattice used for deformation is increased, the range affected by the movement of each reference point is also increased accordingly. When this becomes extreme, the locality of deformation is lost, and a large deformation occurs outside the original deformation target area.
In addition, when deformation is performed using a control grid having a large grid interval, the area affected by each reference point becomes large as described above, and the area affected by other reference points existing in the vicinity tends to overlap. At this time, even if care is taken to minimize the error according to equation (1), this error becomes large and blurring tends to occur when a large number of affected areas overlap.

  Further, when the lattice spacing is too small, problems such as (iii) excessive localization of shape change and (iv) failure of shape change occur.

When the deformation of one part is realized using a plurality of reference points, if the deformation is performed using a grid that is too small in size, the deformation by each reference point is localized more than necessary. As a result, the respective deformations are not smoothly connected and natural deformation cannot be performed.
Also, the B-spline transformation is affected by the movement of points only in nine regions around the reference point. However, when deformation is performed using a small-size grid (or when the movement amount of the reference point is significantly larger than the grid size), the movement destination of the reference point does not fit in this region. Normal deformation is not realized. This unnaturalness becomes more prominent than the unnaturalness generated in the above (i) to (iii).

Considering these, in order to perform B-spline conversion with high accuracy, the influence areas of adjacent reference points responsible for deformation of the same part need to overlap to some extent.
On the other hand, it can be seen that it is necessary to adjust so as not to overlap with the influence area of the reference point responsible for deformation of other parts.

In addition, even when the reference points responsible for deformation of the same part are overlapped with each other at the reference points, blurring and deformation errors tend to occur depending on the number of reference points.
The affected areas are overlapped to the extent that no unnaturalness occurs in the deformation only at adjacent reference points that carry the deformation of the same part, and in other cases, the lattice spacing is adjusted so that the affected areas do not overlap as much as possible. Is preferred.

  In addition, among the above four problems, when the problem (iv) occurs, the unnaturalness of deformation becomes the largest. Therefore, it is considered most important that the problem (iv) does not occur, and it is preferable to define the size of the lattice so that the other three problems do not occur as much as possible.

  Here, in order to examine the number of divisions of the control grid at the time of execution of deformation by B-spline transformation, a questionnaire survey was conducted in association with the distance between the reference points (maximum spacing) and the grid spacing of the control grid. In the questionnaire survey, 100 images obtained by changing the lattice spacing of the face images of 25 men and women were used, and responses were obtained from 32 men and women. The results of this investigation are shown in FIG.

  It can be seen from FIG. 12 that when the lattice spacing is larger than the distance between the reference points, 80% or more of the respondents answered that they do not feel unnaturalness. Further, it was found from the subsequent hearing that the above problems (iii) and (iv) were hardly recognized in the images having the same distance between the lattice and the reference point. Most of the reasons for the unnaturalness are the shape of the contour of the face being deformed, but some respondents felt unnaturalness in the shape of the hair arranged around it.

  In addition, from FIG. 12, after the problems (iii) and (iv) are not recognized, the number of answers that the shape is unnatural has not converged so much even if the size of the control grid (lattice interval) is increased. . Further, as described above, if the size of the lattice is too large, the influences of the problems (i) and (ii) are greatly reflected. Therefore, in the partial deformation method, the maximum distance between adjacent reference points among the reference point groups responsible for deformation of the same part is defined as the lattice spacing of the control lattice and used for deformation. As a result, the problems (iii) and (iv) can be reduced as much as possible, and the influence of the problems (i) and (ii) can be suppressed.

[5. Experimental result]
Table 1 below shows a) calculation time of the conventional multi-level B-spline transformation, b) calculation time when the entire image is processed only by the adaptive grid method (partial deformation method is not adopted), and c) partial deformation. The calculation time when combining the method and the conventional multi-level B-spline transformation, and d) the calculation time when combining the partial deformation method and the adaptive lattice method (in the case of FIG. 6) are shown.

As shown in Table 1, it can be seen that in b) the adaptive lattice method, a) the conventional multi-level B-spline conversion process can be reduced, so that the process is accelerated. Further, in the partial deformation method of c), the calculation time is almost the same as that of the conventional multi-level B-spline transformation, but the error can be reduced.
d) When the partial deformation method and the adaptive lattice method are combined, the error can be reduced and the calculation time can be reduced.

[6. Addendum]
In addition, the matter disclosed above is an exemplification, and does not limit the present invention, and various modifications are possible.
The present specification also discloses other inventions other than those described in the claims. For example, the following invention is described in this specification.

  The movement amount of each vertex in the control grid for deforming the image is acquired from the reference point set in the original image before deformation and the target point indicating the movement destination of the reference point, and the movement amount of each vertex is acquired. An image deformation method for deforming the original image using the control grid, comprising: determining a grid interval of the control grid in accordance with an interval between reference points included in the original image, Image transformation method.

  The movement amount of each vertex in the control grid for deforming the image is acquired from the reference point set in the original image before deformation and the target point indicating the movement destination of the reference point, and the movement amount of each vertex is acquired. An image deformation method for deforming the original image using the control grid, wherein the position of the reference point and the position of the target point after being moved by the image deformation using the control grid with a predetermined grid interval An image deformation method characterized by performing image deformation again for a portion where the error is larger than a predetermined threshold by using a control lattice having a lattice interval smaller than the lattice interval.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 11 Reference point setting part 12 Target point setting part 13 Parts setting part 14 Lattice space | interval setting part 15 Image deformation | transformation processing part 16 Error determination part 17 Lattice space | interval adjustment part

Claims (7)

The movement amount of each vertex in the control grid for deforming the image is acquired from the reference point set in the original image before deformation and the target point indicating the movement destination of the reference point, and the movement amount of each vertex is acquired. An image deformation method for deforming the original image using a control lattice,
A parts setting step for setting a plurality of parts in the original image;
For each of the plurality of parts, including a lattice interval setting step for setting a lattice interval of the control lattice,
An image deformation method, wherein image deformation of each of a plurality of parts is performed using a lattice control lattice set in the lattice interval setting step. The image deformation method according to claim 1, wherein, in the lattice interval setting step, the lattice interval is determined according to the interval between reference points included in a part for which the lattice interval is to be set. The image deformation method according to claim 2, wherein, in the lattice interval setting step, the lattice interval in the lattice direction is determined according to the maximum interval in the lattice direction of adjacent reference points in the part. The image deformation method according to claim 3, wherein in the lattice interval setting step, a value substantially equal to the maximum interval is determined as a lattice interval in the lattice direction. For a location where the error between the position of the reference point and the position of the target point after moving by the image deformation using the control grid of the set grid interval is larger than a predetermined threshold, the set grid interval The image deformation method according to claim 1, wherein the image deformation is performed again using a control lattice having a smaller lattice interval. The movement amount of each vertex in the control grid for deforming the image is acquired from the reference point set in the original image before deformation and the target point indicating the movement destination of the reference point, and the movement amount of each vertex is acquired. An image processing apparatus that performs deformation of the original image using a control grid,
Parts setting means for setting a plurality of parts in the original image;
For each of the plurality of parts, including lattice interval setting means for setting the lattice interval of the control lattice,
An image processing apparatus configured to perform image deformation of each of a plurality of parts using a control lattice having a lattice interval set by the lattice interval setting means.   A computer program for causing a computer to function as the image processing apparatus according to claim 6.

Images