JP5008772B2 - Still image animation method, apparatus and program - Google Patents

Description

  The present invention relates to a technique for converting some or all of still images into moving images in the field of computer graphics (CG) and the like.

  Various objects are shown in the still image, but if you can add movement to the object, you can find added value beyond the original information of the still image. Conventionally, an image editing tool is used to map an image to a mesh, and the movement is mainly performed by deforming the lattice points of the mesh with a mouse. For this reason, it is necessary to increase the density of lattice points for fine deformation, and it has been pointed out that the motion between image frames becomes discontinuous during animation. In particular, for a subject with a remarkable three-dimensional composition, it has been necessary to transform the depth information with much effort and time.

  Non-Patent Document 1 proposes a method for imparting motion to a wave in a single picture, and a complex motion is created through random numbers using a spectral method. However, since random numbers are used, there is a problem that fine movement cannot be artificially set and the expression is strongly limited to the wave phenomenon.

  Non-Patent Document 2 discloses a tool that can be interactively set so that a waterfall image can be repeatedly expressed. However, like Non-Patent Document 1, there is a problem that applicable materials are limited to a specific target (here, a waterfall). Moreover, it is only two-dimensional and does not include the depth effect.

Y-Y.Chuang, B.D.Goldman, K.C.Zheng, B.Curless, D.H.Salesin, and R.Szeliski "Animating pictures with stochastic motion textures" ACM SIGGRAPH, pp.853-860,2005. K.S.Bhat, S.M.Seitz, J.K.Hodgins, and P.K.Khosla, "Flow-based video synthesis and editing", ACM SIGGRAPH, pp. 360-363, 2004.

  An object of the present invention is to introduce a depth when moving a two-dimensional image into a moving image so that an arbitrary fine movement can be set.

により設定し、

の有限差分法による離散化表現により時間発展解を求めることで、前記動きを加えたい画像を前記速度場に応じて変形する、ことを特徴とする静止画の動画化方法である。
請求項２にかかる発明の静止画の動画化装置は、１枚の２次元画像を取り込むデータ入力手段と、該取り込んだ２次元画像を蓄積するデータ蓄積手段と、前記取り込んだ２次元画像を遠くの速度と手前の速度とが奥行き情報に基づいて非線形になるように３次元座標に変換する座標変換手段と、前記２次元画像の内の動きを加えたい一部又は全部の画像に１又は２以上の同一又は異なる３次元の速度場を、奥行きを算出する項を加えた３次元移流方程式

により設定する動き設定手段と、

の有限差分法による離散化表現により時間発展解を求めることで、前記動きを加えたい画像を前記速度場に応じて変形するデータを得る動画化計算手段と、を備えることを特徴とする静止画の動画化装置である。
請求項３にかかる発明の静止画の動画化プログラムは、１枚の２次元画像を取り込むデータ入力ステップと、該取り込んだ２次元画像を蓄積するデータ蓄積ステップと、前記取り込んだ２次元画像を遠くの速度と手前の速度とが奥行き情報に基づいて非線形になるように３次元座標に変換する座標変換ステップと、前記２次元画像の内の動きを加えたい一部又は全部の画像に１又は２以上の同一又は異なる３次元の速度場を、奥行きを算出する項を加えた３次元移流方程式

により設定する動き設定ステップと、

の有限差分法による離散化表現により時間発展解を求めることで、前記動きを加えたい画像を前記速度場に応じて変形するデータを得る動画化計算ステップと、をコンピュータに実行させるための静止画の動画化プログラムである。 In order to achieve the above object, a still image moving image method according to a first aspect of the present invention includes a moving image capturing apparatus that captures and accumulates a single two-dimensional image, and that captures the captured two-dimensional image at a distant speed. Is converted into a three-dimensional coordinate so that the front speed is nonlinear based on the depth information, and one or two or more of the same or different three or more images to be added to the motion in the two-dimensional image are added. 3D advection equation with a 3D velocity field and a term to calculate depth

Set by

This is a still image animation method characterized in that a time evolution solution is obtained by a discretization expression using the finite difference method, and the image to which the motion is to be added is deformed according to the velocity field.
According to a second aspect of the present invention, there is provided a still image moving image conversion apparatus comprising: a data input unit that captures a single two-dimensional image; a data storage unit that stores the captured two-dimensional image; Coordinate conversion means for converting the speed and the previous speed into three-dimensional coordinates so as to be nonlinear based on the depth information, and 1 or 2 for some or all of the images to which motion in the two-dimensional image is to be added. Three-dimensional advection equation with the same or different three-dimensional velocity field plus a term to calculate depth

Movement setting means set by

By determining the time evolution solution by the discrete representation by finite difference method, a still image, characterized in that it comprises a moving value calculating means for obtaining data to be modified in accordance with the image to be added to the motion to the velocity field This is an animation device .
Still image video programs of the invention according to claim 3, a data input step of capturing a two-dimensional image of one, and a data蓄Sekisu step of storing a two-dimensional image captured said, the captured two-dimensional image coordinate variable 換Su steps of converting the 3-dimensional coordinates to be non-linear based on distant velocity and before the speed and the depth information, the two-dimensional part want to make movements or all of the images in the image 3D advection equation with one or two or more same or different 3D velocity fields plus a term to calculate depth

A motion setting step set by

A still image for causing a computer to execute a moving image calculation step for obtaining data for deforming the image to which the motion is to be added according to the velocity field by obtaining a time evolution solution by a discretization expression of the finite difference method of This is an animation program .

  According to the present invention, conversion of a two-dimensional image into a three-dimensional coordinate system, setting of a three-dimensional velocity field to an image to which motion is to be added, and speed of an image of a portion to which motion is to be added using a partial differential equation By transforming according to the field, it is possible to obtain data as a result of time evolution of natural animation. Even a hand, facial expression, and other movements can be directly applied to animation processing of a two-dimensional image by detecting the three-dimensional velocity field.

  The present invention captures and accumulates a two-dimensional image that has already been taken through a camera or a camera, converts the captured two-dimensional image into a three-dimensional coordinate, and adds a motion within the two-dimensional image. Alternatively, one or two or more same or different three-dimensional velocity fields are set for all the images, and the moving image is deformed according to the velocity field using the partial differential equation (three-dimensional advection equation). The time-developed data is obtained by calculating the conversion, and this is displayed as a moving image.

  An embodiment of the device configuration for realizing this is shown in FIG. The apparatus includes a data input unit 101 that captures a two-dimensional image that has already been captured through a camera, a data storage unit 102 that stores the captured image, and a coordinate conversion that converts the two-dimensional image into three-dimensional coordinates. Means 103, motion setting means 104 for setting one or two or more same or different three-dimensional velocity fields in a part or all of the images to be subjected to motion in the two-dimensional image, and partial differential equations (three-dimensional A moving image calculation means 105 that obtains time-developed data by calculating a moving image that deforms the image to be added according to the velocity field using an advection equation), and a display means 106 that displays the result. .

  These means can be realized by a computer having an arithmetic processing unit, an external storage device, a memory, and the like, and the lattice point model deformation / movement processing of this embodiment is executed by a program. This program is stored in an external storage device of a computer, and can be recorded on a recording medium or provided via a network.

  The program includes a step of capturing a two-dimensional image that has already been photographed through a camera or a step, a step of converting the two-dimensional image into a three-dimensional coordinate, The step of setting one or two or more same or different three-dimensional velocity fields in all the images and the partial differential equation (three-dimensional advection equation) are used to transform the image to which the motion is to be added according to the velocity field. And a step of calculating animation to obtain time-developed data.

  FIG. 2 is an explanatory diagram of an example in which a two-dimensional image is converted into three-dimensional coordinates and a three-dimensional velocity field is set in a part of the image. By applying the concept of weak perspective projection and parallel perspective projection (reference: OpenCV programming book production team, “OpenCV”, Mainichi Communications, 111 pages, September 21, 2007) to a two-dimensional image. Enable to add 3D information. Use the mouse to select one of the two-dimensional images shown in Fig. 2 (a) from the 3D velocity field prepared in advance using the GUI menu. And set. Here, the vortex field 201 is selected. The two-dimensional image shown in FIG. 2B is converted into a three-dimensional coordinate system. After this, the image and the velocity field are blended, and then the three-dimensional advection equation is used as the partial differential equation, and according to the velocity field (vortex field 201), as shown in FIG. If the image is deformed, a time-developed moving image can be easily created as shown in FIGS. 3 (d) and 3 (e). In FIG. 3 (e), part of the cloud in FIG. 3 (d) is deformed according to the velocity field. Note that the three-dimensional velocity field can be set not only for a part of the two-dimensional image to which a motion is desired but also for all the images. Further, the present invention is not limited to one three-dimensional velocity field, and two or more same or different three-dimensional velocity fields can be set in different regions.

と表される。ｔは時間、ｘ、ｙは画素上の位置、ｚは奥行き、ｕ＝（ｕ，ｖ，ｗ）は速度ベクトル項である。画像濃淡値Ｉ_i,j,kがベクトルｕによって移流式により画像に伝播される。 The three-dimensional advection equation is

It is expressed. t is time, x and y are positions on a pixel, z is depth, and u = (u, v, w) is a velocity vector term. The image gray values I _{i, j, k} are propagated to the image by the advection equation by the vector u.

ここで、ｎは離散化時間を表し、ｎ＝０を初期値とする。Δｔは時間ステップ幅で1.0とした。右辺は空間の１次微分である。なお、格子幅は1.0とした。 In the advection equation, an image is propagated stepwise from the outside to the inside by obtaining a time evolution solution by a discretization expression by the finite difference method as in the following equation (2). The initial image is an external known area.

Here, n represents the discretization time, and n = 0 is an initial value. Δt is 1.0 in terms of time step width. The right side is the first derivative of space. The lattice width was 1.0.

  By converting the two-dimensional image into the three-dimensional coordinates, the far speed and the near speed are naturally converted nonlinearly on the two-dimensional image based on the depth information. Until now, it has been a two-dimensional velocity field, so such animation has been difficult.

  FIG. 3 is an explanatory diagram of an example of a three-dimensional velocity field. 3 (a) shows the vortex field, and FIG. 3 (b) shows the divergent field. The user can freely change the size and density by using the mouse and using the GUI menu. Other free curves can also be set three-dimensionally. Not only fluid phenomenon but also face deformation can be applied.

  FIG. 4 is an explanatory diagram of an example in which a motion vector (three-dimensional velocity field) is detected from the images captured by the cameras 401 and 402 by a motion estimation method. By moving the hands 403 and 404 in the video, it is possible to obtain three-dimensional motion information (reference: Xugang, Saburo Tsubame, “3D Vision”, Kyoritsu Shuppan, pages 112-113, 2001). Reference numeral 405 denotes a motion vector. Thus, depth information can be easily acquired by using the two cameras 401 and 402. That is, an optical flow can be obtained three-dimensionally. This is stored as input information so that it can be selected on the GUI menu, and by applying it as a three-dimensional velocity field, some or all of the images in the two-dimensional image are subjected to complicated movements according to the user's intention. It can be animated to make it happen.

It is a block diagram of the Example of the apparatus structure of this invention. It is explanatory drawing which sets the three-dimensional coordinate system to the two-dimensional image, sets the three-dimensional velocity field, and obtains the image developed in time. It is explanatory drawing which shows the example of a three-dimensional velocity field. It is explanatory drawing which detects a 3-dimensional velocity field from the 2-dimensional image imaged with the camera by the motion estimation method.

101: Data input means, 102: Data storage means, 103: Coordinate conversion means, 104: Motion setting means, 105: Animation calculation means, 106: Display means 201: Vortex field 401, 402: Camera, 403: 404: Hand 405: Motion vector

Claims (3)

An animation device captures and accumulates a single 2D image, converts the captured 2D image into 3D coordinates so that the distant speed and the near speed become nonlinear based on depth information, A three-dimensional advection equation in which one or two or more same or different three-dimensional velocity fields are added to a part or all of the two-dimensional images to which motion is to be added, and a term for calculating the depth is added.

Set by

By transforming the image to which the motion is to be added according to the velocity field by obtaining a time evolution solution by a discretized representation by the finite difference method of
A still image animation method characterized by the above. Data input means for capturing a single two-dimensional image;
Data storage means for storing the captured two-dimensional image;
Coordinate conversion means for converting the captured two-dimensional image into three-dimensional coordinates so that the distant speed and the near speed become nonlinear based on the depth information;
A three-dimensional advection equation in which one or two or more same or different three-dimensional velocity fields are added to a part or all of the two-dimensional images to which motion is to be added, and a term for calculating the depth is added.

Movement setting means set by

An animation calculation means for obtaining data for deforming the image to which the motion is to be added according to the velocity field by obtaining a time evolution solution by a discretization expression of the finite difference method of
An apparatus for converting a still image into a moving image. A data input step for capturing one two-dimensional image;
A data蓄Sekisu step of storing a two-dimensional image captured said,
Coordinate variable 換Su steps of converting the 3-dimensional coordinates to be non-linear based on distant velocity and before the speed and the depth information a two-dimensional image captured above,
A three-dimensional advection equation in which one or two or more same or different three-dimensional velocity fields are added to a part or all of the two-dimensional images to which motion is to be added, and a term for calculating the depth is added.

A motion setting step set by

An animation calculation step for obtaining data for transforming the image to which the motion is to be added according to the velocity field by obtaining a time evolution solution by a discretized representation by the finite difference method of
A still image animation program that allows a computer to execute the program.

Images