JP4767305B2 - Image deformation device, image deformation method, and image deformation program - Google Patents

Description

  The present invention relates to an image deformation apparatus, an image deformation method, and an image deformation program that deform an image based on a particle method.

In the field of computational fluid dynamics, in solving the Navier-Stokes equations governing fluid motion (hereinafter referred to as “NS equations”), stable numerical values that are in line with the non-linear and non-stationary properties inherent to fluids The solution is important and necessary. Expression (1) is an expression that represents the NS equation in a vector format. The first term on the left side is a time term (differential term), and the second term is an advection term (convection term). The first term on the right side is a pressure term, the second term is a viscosity term (diffusion term), and the third term is an external pressure term.

  As a conventional numerical solution method for NS equations, a finite difference method, a finite element method, a boundary element method, and the like are widely used after defining a discretized calculation grid. In addition, in order to obtain vortex generation as a solution of the NS equation in a fluid phenomenon, not limited to gas and liquid, it is one of the issues to deal with instability in the numerical solution by the convection term. It is known that the handling of terms greatly affects the calculation accuracy.

  On the other hand, in the field of computer graphics, computer vision, etc., on the other hand, changes in fluids such as flames, waves, clouds, fog, etc. over time, and rigid bodies and elastic bodies when cars or rubber balls collide or negotiate. A numerical method called a particle method (particle method) is actively used to express the deformation and the like (see Non-Patent Document 1). For example, in aerospace engineering, it is used to analyze the flow around extremely high-speed objects exceeding the speed of sound, and in coastal engineering, it is used to model overtopping, boiling, steam explosion, etc. that collide with a dike.

  The particle method is a modeling technique that approximates the shape change of various continuums such as fluids, rigid bodies, and elastic bodies using a plurality of particles. Specific solutions based on this particle method include SPH (Smoothed Particle Hydrodynamics) method and MPS (Moving Particle Semi-implicit) method, which is an extension of SPH method. Since it is a complete Lagrangian method, there is an advantage that it can be easily handled even if the deformation of the interface is large.

In particular, the MPS method is a solution obtained by introducing an incompressible flow calculation algorithm into the particle method, and equations (2) and (3) are used as governing equations.

  Equation (2) is a mass conservation law in the MPS method. Ρ on the left side is the fluid density. By making the time derivative of ρ zero, the density is made constant, which is an uncompressed condition in the MPS method. Equation (3) is a momentum conservation law in the MPS method. Since the complete Lagrangian method means that the convection term of the NS equation shown in equation (1) is not discretized, the Lagrangian derivative can be applied to the time derivative of the equation of motion as shown in equation (3), The convection term will not be explicitly described. Note that u represents velocity, p represents pressure, ν represents diffusion coefficient, and f represents external force.

Here, since the particle method does not use a calculation grid as one of its features, it becomes a problem how to discretize each differential equation shown in the equations (2) and (3). For, we prepare a particle interaction model for the differential operators of differential equations such as gradient (∇) and Laplacian (∇ ² ), and discretize the differential equation using the particle interaction model. Become. Specifically, each particle exists in the Lagrangian coordinate system because it is a complete Lagrangian method, and does not exist discretely on the lattice as in the finite difference method or the like. As shown in FIG. A weighting function (Equation (4)) that considers the interaction with a particle s located at a predetermined interparticle distance r from k is defined, and this equation (4) is used as an interparticle interaction model.

That is, equation (4), for particles s present in the range of particle k of constant radius r _e, it is assumed to provide a volume weighted by exponential functions for inter-particle distance r between the particle k. On the other hand, since it is assumed that there is no interaction with particles outside the range of the radius r _e , the value of w (r) is zero when r _e <r.

Then, for a plurality of particles s which are present in a range of particle k of radius r _e, called particle number density was taken the sum of the weighting function shown in equation (4), (5) Define in.

Here, as described above, since the differential equations shown in the equations (2) and (3) include the gradient (∇) and the Laplacian (∇ ² ) as differential operators, the gradient value, the Laplacian value, It is necessary to define a calculation formula for calculating each of.

First, the equation for calculating the gradient value of the scalar value (physical quantity) φ at the position of the particle k is defined as in equation (6). As shown in FIG. 7, a simple gradient vector is defined between the particle k and the particle s and averaged by multiplying by the weighting function w (r). d is the number of spatial dimensions. For example, in the case of two dimensions, 2 is substituted. Since the particle number density from the uncompressed condition is constant, and this constant value n ^0.

Next, a formula for calculating the Laplacian value is defined as shown in Equation (7). As shown in FIG. 8, this means that the scalar value of the particle k is distributed to the particle s with a distribution of weighting functions.

  Therefore, by using the equations (6) and (7) and calculating the simultaneous equations of the equations (2) and (3), the particle velocity u and the interparticle distance (= position) r can be obtained. it can.

Next, a specific calculation algorithm will be described. By calculating the terms other than the pressure term in the equation (3), in other words, by the equations (8) and (9), the velocity u and the position r of the particle at the intermediate time * between the time n and the time n + 1. Can be obtained. The following scheme is based on the MAC (Market-And-Cell) method generally used in numerical calculation.

Here, it is necessary to set the particle number density to a constant value n ⁰ from the non-compressed condition. However, since the particle number density n ^* does not satisfy the uncompressed condition at the stage of the equation (9), the provisional particle number density is given using the equation (10).

Further, since it is necessary to consider the pressure term to satisfy the incompressible condition, the Poisson equation relating to pressure is defined as Equation (11), and the iterative calculation is performed so that the right side of Equation (11) converges to zero. Calculate about.

Then, the pressure at a certain number of iterations is substituted into equation (12) to obtain a speed correction amount u ′. Expression (12) is a calculation expression for calculating the correction amount u ′ for the speed u, and is assumed to be caused by the pressure term.

Since the correction amount u ′ in the equation (12) is the velocity fluctuation amount that fluctuates from the time n to the time n + 1, the velocity and position of the particle at the new time can be determined from the equations (13) and (14). It becomes.

The calculation of Expression (11) is continued until the change in the absolute value of the pressure around the number of iterations converges to a predetermined minute value or less in the vicinity of each particle. If it is not less than the minute value, the calculation result at time n is repeated as time n + 1 until calculation is completed.
“Numerical analysis of flow by particle method”, [online], [October 27, 2008 search], Internet <URL: http://www.nagare.or.jp/nagare/21-3/21-3 -t02.pdf> Supervised by Hideyuki Tamura, “Introduction to Computer Image Processing, Chapter 5 Image Feature Extraction, Analysis, and Recognition”, Soken Publishing, 1985, p.118-125

  However, there are almost no cases where the particle method is applied to an object called an image, and since an image is usually configured with a limited number of pixels, there is a problem that it is difficult to simply apply the particle method. In addition, since particle attributes in the particle method are assumed to be uniform, in order to correspond to various attributes such as color and texture in the image, an approximate calculation is performed by increasing or decreasing the density of the number of particles. When it is increased, there is a problem that the calculation cost increases rapidly in proportion to the increase in density.

  The present invention has been made in view of the above, and an object of the present invention is to provide an image deformation apparatus, an image deformation method, and an image deformation program that deform an image based on a particle method.

The invention according to the first claim is a storage means for storing a certain image, and the RGB color components forming the pixel for each pixel of the image by reading the image from the storage means. Each value is multiplied by a predetermined weighting coefficient, and the difference between the image intensities obtained by adding the respective component values after the multiplication and the image intensity of two adjacent pixels is multiplied by 0.5. Edge strength as a value, color model conversion of the RGB into HSV, a color that is an average value of any component of H, S, and V in a plurality of pixels within a certain range from the target pixel, and the target pixel the sum of components of said H calculated for a plurality of pixels within a predetermined range, analyzing at least two or more image feature amount of the texture strength is a value obtained by averaging by a fast Fourier transform on the total solution And means, said pixel and particles in the particle method, weighting amount = exp (image feature quantity / normalization factor) (normalized coefficient, as the weighting amount calculated in the equation is distributed in the range of 0 to 1 Each of the plurality of image feature amounts is converted into a plurality of weighting amounts using a formula (which is a coefficient to be adjusted), each of the plurality of weighting amounts is multiplied by a predetermined weighting factor, and the value after the multiplication is added. A motion calculation means for calculating the motion of the particle based on the particle method, using a value obtained by dividing the value by a predetermined adjustment coefficient so that the value after the addition falls within 1.0 at a maximum , And a display means for displaying a deformed image based on the pixels moved by the movement of the particles.

The gist of the second aspect of the invention is that a plurality of image feature amounts analyzed for the pixel are analyzed based on image feature amounts of a plurality of pixels included in a certain range from the pixel. To do.

According to a third aspect of the present invention, there is provided a first step of storing a certain image in the storage means by a computer, and reading out the image from the storage means to form a pixel for each pixel of the image. The component value of each RGB color is multiplied by a predetermined weighting coefficient, and the difference between the image intensity which is a value obtained by adding the respective component values after the multiplication and the image intensity of two adjacent pixels is calculated. The edge strength, which is a value obtained by multiplying 0.5 with respect to the color, and RGB color model conversion into HSV, and the average value of any component of H, S, and V in a plurality of pixels within a certain range from the target pixel And the texture intensity, which is a value obtained by calculating the sum of the H components for a plurality of pixels within a certain range from the target pixel and averaging the sum by fast Fourier transform. Also a second step of analyzing two or more image feature amount, the pixel with the particles in the particle method, weighting amount = exp (image feature quantity / normalization factor) (normalized coefficients are calculated in the formula The plurality of image feature amounts are converted into a plurality of weight amounts using a formula of a weighting amount to be distributed in a range of 0 to 1, respectively, and a predetermined weighting is applied to the plurality of weight amounts. A value obtained by multiplying each coefficient, adding the value after the multiplication, and dividing by a predetermined adjustment coefficient so that the value after the addition falls within 1.0 is used as the particle weighting amount. And a fourth step of displaying a deformed image based on a pixel moved by the movement of the particle.

In the invention according to the fourth aspect, the plurality of image feature amounts analyzed for the pixel are:
The gist is that the analysis is performed based on image feature amounts of a plurality of pixels included in a certain range from the pixel.

The gist of the fifth aspect of the invention is to cause each step in the image deformation method according to the third or fourth aspect to be executed by a computer.

  According to the present invention, it is possible to provide an image deformation device, an image deformation method, and an image deformation program that deform an image based on a particle method.

  FIG. 1 is a functional configuration diagram illustrating a functional configuration of the image deformation apparatus according to the present embodiment. The image deformation apparatus 100 includes an input unit 11, an analysis unit 12, a motion calculation unit 13, a display unit 14, and a storage unit 31.

  The input unit 11 has a function of receiving an input of an image in which, for example, a wavefront of swell, wave splash, and the like are captured.

  The accumulation unit 31 has a function of accumulating images input after being received by the input unit 11. As such a storage unit 31, for example, a storage device such as a memory or a hard disk is generally used, and not only the inside of the image transformation device 100 but also an external storage that can be electrically connected via a communication network. It is also possible to use a device.

  The analysis unit 12 has a function of reading an image from the storage unit 31 and analyzing a plurality of image feature quantities characterizing the image for each pixel. Specifically, image feature amounts of image intensity, edge intensity, color, and texture intensity are calculated for each pixel constituting the image.

First, a method for calculating the image intensity will be described. The two-dimensional coordinate position of the pixel is (i, j), the image intensity at that position is I (i, j), and one pixel is formed with three colors R (red) G (green) B (blue). In this case, the analysis unit 12 multiplies each color component value by a predetermined weighting coefficient (a _{1 to} a ₃ ), as shown in Expression (15), and calculates each component value after multiplication. The added value is used as the image intensity of the pixel (i, j). Regarding the weighting used in the equation (15), the ratio (a ₁ : a ₂ : a ₃ ) may be set to 1: 2: 1, for example, but the sum (a ₁ + a ₂ + a ₃ ) is always set to 1. Thus, an average average image intensity I _ave (i, j) in one pixel is calculated.

Next, a method for calculating edge strength will be described. An edge is a portion where the intensity value of a pixel has changed remarkably, and is a portion where the visual variation seen in the contour line, texture, etc. of a wavefront taken is very large. Analyzer 12, as shown in equation (16), the average image intensity value _{I ave (i + 1, j} ) and the average image intensity value _I ave (i-1, j adjacent pixels B neighboring pixel A adjacent to the right and left The value obtained by averaging the difference from) is the edge intensity I _edge (i, j) of the pixel (i, j).

  Note that Equation (16) calculates edge strength using pixels adjacent to the left and right, but it is also possible to use pixels adjacent vertically and to use four adjacent pixels vertically and horizontally. . Equation (16) is calculated using the average image intensity value calculated using Equation (15), but the image intensity value of the image read from the storage unit 31 may be used as it is. Is possible.

Next, a color calculation method will be described. The analysis unit 12 performs color model conversion of the RGB image read from the storage unit 31 into an HSV (Hue (Hue), Satulation (saturation), Value (brightness)) image, and, as shown in Expression (17), the pixel ( The average value of only the H component (hue component) of the set of neighboring pixels (si, sj) located in the vicinity of i, j) is the color of the pixel (i, j). NN is the number of pixels in the set. For example, when 3 × 3 is a set of neighboring pixels, the number of pixels NN is nine.

  Note that Equation (17) calculates the hue using the hue, but it is also possible to use the saturation or brightness, or to use the RGB component values as they are.

Finally, a method for calculating the texture strength will be described. Here, the spatial frequency component is used when calculating the texture intensity. Specifically, as shown in Expression (18), the analysis unit 12 calculates the sum of H components (hue components) for a set of neighboring pixels (si, sj) located in the vicinity of the pixel (i, j). Then, a value obtained by averaging the sum by fast Fourier transform (FFT) is set as the texture intensity of the pixel (i, j).

  Note that the image strength, edge strength, color, and texture strength are examples of image feature amounts, and other image feature amounts may be used. Further, it is not always necessary to use these four image feature amounts, and two or three image feature amounts may be used. Furthermore, each calculation formula shown in Expression (15) to Expression (18) is an example of calculating each image feature amount, and it is also possible to calculate using other components characterizing each image feature amount. It is.

The motion calculation unit 13 regards pixels constituting the image as particles in the particle method, and uses a plurality of image feature amounts analyzed for the pixels as particle weighting amounts to calculate the particle motion based on the particle method. Has the function to calculate. Specifically, as shown in FIG. 2, image intensity, edge intensity, color, and texture intensity analyzed by the analysis unit 12 can be converted into a certain range on the same basis (linear equation or the like) or nonlinear ( Each is converted into a predetermined weighting amount using a conversion function of an exponential function, a quadratic function, or the like. For example, in the case of image intensity, a conversion function for converting the image intensity so as to be distributed in the range of 0 to 1 is used as shown in Expression (19). The normalization coefficient is a coefficient that is adjusted so that the calculated weighting amount is distributed in the range of 0 to 1.

Thereafter, the motion calculation unit 13 calculates, for each pixel, a weighting amount obtained by applying a predetermined weighting to each weighting amount in the range of 0 to 1 and converging the weighting amount to a certain range. That is, the weighting amounts in the image strength, edge strength, color, and texture strength are set to w _int (i, j), w _edge (i, j), w _hue (i, j), and w _tex (i, j), respectively. In this case, as shown in the equation (20), the weighting coefficients (b _{1 to} b ₄ ) are respectively multiplied with the respective weighting amounts, and a value w _ave (i, j) obtained by adding the values after the multiplication is calculated. To do. Note that WN is a coefficient that is adjusted so that the calculated weighting amount falls within 1.0 at the maximum.

Then, the motion calculation unit 13 sets the calculated w _ave (i, j) as the weighting amount of the particle in the particle method, that is, as shown in FIG. 3, the weighting of each pixel corresponding to the particle k and the particle s, respectively. The time-series movement of each particle according to the calculation method of the particle method of Equations (6) to (14) described above using Equation (21) _obtained by multiplying the amount _rs and rk of Equation (5) by each amount To calculate the velocity and position of the particles (pixels) at the new time.

  The display unit 14 has a function of displaying a deformed image based on the pixel moved by the particle motion calculated by the motion calculation unit 13.

  Subsequently, a processing flow of the image deformation apparatus according to the present embodiment will be described. FIG. 4 is a flowchart showing a processing flow of the image transformation apparatus according to the present embodiment. First, the input unit 11 receives an input of an image obtained by photographing a predetermined target (step S101). Next, the storage unit 31 stores the input image after receiving it (step S102).

  Subsequently, the analysis unit 12 analyzes the image feature amount relating to the image strength, edge strength, color, and texture strength characterizing the input image for each pixel (step S103).

  Then, the motion calculation unit 13 converts the analyzed image intensity, edge intensity, color, and texture intensity into weighting amounts using a conversion function that can be converted into a certain range based on the same reference, and the converted weightings. Calculates the amount of weight that is converged to a certain range by applying a predetermined weight to the amount for each pixel, and calculates the movement of particles based on the particle method using the weighted amount after convergence as the particle weight in the particle method. (Step S104).

  Finally, the display unit 14 displays the target image deformed based on the pixels moved by the movement of the particles (step S105).

FIG. 5 is a diagram illustrating a state in which an image is deformed by the image deformation device according to the present embodiment. For an image in which a complex texture as shown in FIG. 5 (a) is expressed, a deformation target range is selected using a mouse as shown in FIG. 5 (b), and an initial speed around the selected target is displayed. Define multiple initial velocity vectors as a field. The external force f gravity (9.8), the range of radius r _e is 10 pixels, given an initial value zero for pressure, based image intensity, the edge intensity, color, texture intensity in accordance with the processing flow shown in FIG. 4 As a result of calculating the weighting amount and further repeatedly calculating using the equations (13) and (14), it becomes possible to realize large displacement and movement of the particles at high speed as shown in FIG. As a result, it was possible to confirm that the shapes of the selected target ranges changed. Although it is a complete Lagrangian method, it can be said that it is possible to realize vortex and rotation deformation and color change at the same time, and it is also characterized by the realism.

  According to the present embodiment, a plurality of image feature quantities characterizing an image are analyzed for each pixel, the pixel is a particle in the particle method, and the plurality of image feature quantities analyzed for the pixel are the particle features. As the weighting amount, the movement of each particle is calculated based on the particle method, so that it is possible to deform the object of the image using a single image and to express the deformation easily and at high speed. Become.

  Finally, the image transformation apparatus described in each embodiment is configured by a computer, and each process of each functional block is executed by a program. In addition, each processing operation of the image deformation apparatus described in each embodiment is recorded as a program on a recording medium such as a compact disk or a floppy (registered trademark) disk, and this recording medium is incorporated in a computer or a recording medium. The program recorded in the above can be downloaded to a computer via an arbitrary communication line, or installed from a recording medium, and the computer is operated by the program, thereby causing each processing operation described above to function as an image transformation device. Of course you can.

  It should be noted that the image transformation apparatus described in the present embodiment can be applied particularly in the technical fields of the multimedia field, the coding field, the communication field, and the video surveillance field.

It is a functional block diagram which shows the functional structure of the image deformation apparatus which concerns on this Embodiment. It is explanatory drawing for demonstrating the process in a motion calculation part. It is a figure which shows the state which matched the particle | grains of the particle method, and the pixel of the image. It is a flowchart which shows the processing flow of the image transformation apparatus which concerns on this Embodiment. It is a figure which shows the state in which an image is deform | transformed by the image deformation apparatus which concerns on this Embodiment. It is a figure which shows the interaction between particle | grains in a particle method. It is a figure which shows the gradient value of particle | grains k with respect to particle | grains s. It is a figure which shows the Laplacian value of particle | grains k with respect to particle | grains s.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Input part 12 ... Analysis part 13 ... Motion calculation part 14 ... Display part 31 ... Accumulation part 100 ... Image deformation device S101-S105 ... Step

Claims (5)

Storage means for storing an image;
The image is read from the storage means, and for each pixel of the image, each RGB component color forming the pixel is multiplied by a predetermined weighting coefficient, and each component value after the multiplication is obtained. The image intensity that is the added value, the edge intensity that is a value obtained by multiplying the difference between the image intensities of two adjacent pixels by 0.5, and the RGB color model converted to HSV, Calculate the sum of H, S, and V of a component within a certain range as the average value of any component of H, S, and V, and the sum of the H components for a plurality of pixels within a certain range from the target pixel. An analysis means for analyzing at least two or more image features out of the texture intensity which is an averaged value by fast Fourier transform ;
The pixel is a particle in the particle method, and weighting amount = exp (image feature amount / normalization coefficient) (the normalization coefficient is a coefficient that is adjusted so that the weighting amount calculated by the equation is distributed in the range of 0 to 1. Each of the plurality of image feature amounts is converted into a plurality of weighting amounts using a formula (1)), the predetermined weighting coefficients are respectively multiplied by the plurality of weighting amounts, and the value after the multiplication is added, A motion calculation means for calculating a motion of the particle based on the particle method, using a value obtained by dividing the value by a predetermined adjustment coefficient so that the value after addition falls within a maximum of 1.0, as a weighting amount of the particle;
Display means for displaying a deformed image based on pixels moved by the movement of the particles;
An image transformation device comprising: The plurality of image feature amounts analyzed for the pixel are:
2. The image deformation device according to claim 1 , wherein the image deformation device is analyzed based on image feature amounts of a plurality of pixels included within a certain range from the pixel. By computer
A first step of storing an image in the storage means;
The image is read from the storage means, and for each pixel of the image, each RGB component color forming the pixel is multiplied by a predetermined weighting coefficient, and each component value after the multiplication is obtained. The image intensity that is the added value, the edge intensity that is a value obtained by multiplying the difference between the image intensities of two adjacent pixels by 0.5, and the RGB color model converted to HSV, Calculate the sum of H, S, and V of a component within a certain range as the average value of any component of H, S, and V, and the sum of the H components for a plurality of pixels within a certain range from the target pixel. A second step of analyzing at least two or more image feature quantities of texture intensity which is a value averaged by fast Fourier transform ;
The pixel is a particle in the particle method, and weighting amount = exp (image feature amount / normalization coefficient) (the normalization coefficient is a coefficient that is adjusted so that the weighting amount calculated by the equation is distributed in the range of 0 to 1. Each of the plurality of image feature amounts is converted into a plurality of weighting amounts using a formula (1)), the predetermined weighting coefficients are respectively multiplied by the plurality of weighting amounts, and the value after the multiplication is added, A third step of calculating the movement of the particle based on the particle method, with a value obtained by dividing by a predetermined adjustment coefficient so that the value after addition is within 1.0 at maximum ,
A fourth step of displaying a deformed image based on the pixels moved by the movement of the particles;
An image deformation method characterized by comprising: The plurality of image feature amounts analyzed for the pixel are:
4. The image deformation method according to claim 3, wherein analysis is performed based on image feature amounts of a plurality of pixels included within a certain range from the pixel. 5. An image deformation program that causes a computer to execute each step in the image deformation method according to claim 3 or 4 .