JP2003069900A - Device and program for generating video - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video generating device and its program which generate a continuous-motion video where the motions of a subject are smoothly connected between single-motion videos when the single-motion videos of the actually shot subject are connected so as to obtain the continuous-motion video by synthesis. SOLUTION: The video generating device 1 is provided with a main control part 2 having a single-motion deviation correcting means 2a for correcting the deviation of the subject in the single-motion video, an inter-motion deviation correcting means 2b for correcting the deviation of the subject between the single-motion videos and a slight deviation correcting means 2c, a RAM 3, a ROM 4, a VRAM 5, an external storage part 6, an image compositing part 7, an external input/output part 8, a display part 9, an external signal input part 10 and a bus 11.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image generating apparatus and an image generating program for generating a motion image of a person, which is used when a real person is synthesized on computer graphics.

[0002]

2. Description of the Related Art Generally, when a live-action image of a moving person (subject) is synthesized on a CG (computer graphics) image, the person performs a series of motions in front of a background of a uniform specific color (chroma keyback). The video image being recorded is recorded, the foreground and the background are separated from each video frame of the recorded video information based on the gradation value of the specific color, and the image of the separated foreground person is converted into a CG video image. By synthesizing each video frame, a live-action video of a moving person can be CG.
It was composited on the video. Here, the series of motions of the person represents a continuous motion from the start to the end of the motion of the person combined on the CG image.

[0003]

However, in the above-mentioned conventional technique, the basic motion of a person, for example, a single motion image obtained by photographing a single motion such as "bowing" or "looking sideways" is connected. Then, when synthesizing on the CG image to generate a continuous motion image, the position and the shape of the person are deviated for each single motion, and thus the generated continuous motion image is
There is a problem in that the images become discontinuous during a single motion, resulting in unnatural motion images.

In order to avoid this problem, conventionally, instead of connecting the images of each single motion of the person, the images of a series of motions of the person are used for synthesis on the CG image. Therefore, the image of the person to be combined with the CG image needs to be a continuous motion image in which the entire series of motions is photographed at once, and the continuous motion image can be reused in another scene on the CG image. There was a problem that it could not be done.

The present invention has been made in view of the above-mentioned problems, and a series of motions of a person actually photographed on a CG image is obtained by connecting single motion images obtained by photographing a single motion of the person. It is an object of the present invention to provide a video generation device and a video generation program that generate a motion video that smoothly connects a motion of a person between the single motion videos when they are combined.

[0006]

DISCLOSURE OF THE INVENTION The present invention is proposed to achieve the above-mentioned object. First, the image generating apparatus according to claim 1 is configured so that a plurality of images are taken for each type of motion of a subject. In a video generation device for generating a continuous motion video of the subject by connecting the single motion video of, based on a characteristic point that is a feature of the shape of the subject set in advance,
A configuration is provided in which a single motion shift correction unit that corrects the shift of the subject in each video frame of the single motion video is provided, and the positions of the subject in the start video frame and the end video frame of the single motion video are matched.

According to such a configuration, the image generation apparatus causes the single motion deviation correction means to detect the object in each image frame of the single motion image based on the characteristic points that are characteristics of the shape of the object set in advance. Displacement correction is performed to match the positions of the subject in the start video frame and the end video frame of the single motion video. At this time, the video generation device determines whether the feature point of the subject in the start video frame and the feature point of the subject in the end video frame in the single motion video are shifted between the start video frame and the end video frame based on the amount of deviation. Correct the displacement of the subject in the frame.

Further, in the image generating apparatus according to claim 2, in the image generating apparatus according to claim 1, the single motion shift correcting means shifts a feature point between the start video frame and the end video frame. Based on the amount of deviation of the characteristic points of each video frame calculated from the amount and the number of video frames of the single motion video, the deviation of the subject is corrected by performing warping on the object of each video frame. It was configured.

According to this structure, in the video generation device, the single motion shift correction means calculates the shift amount between the feature points of the start video frame and the end video frame and the number of video frames of the single motion video. Based on the amount of deviation of the characteristic points of each video frame, the object of each video frame is warped to correct the deviation of the object.
At this time, the video generation device corrects the shift of the subject in each frame by increasing the shift amount of the feature points of each video frame as the distance from the start video frame increases (as the end video frame approaches).

According to a third aspect of the present invention, there is provided a video generation device,
The image generation apparatus according to claim 1 or 2, wherein a reference image in which the subject is arranged at an initial position in advance and feature points of the subject are set, and a single image corrected by the single motion deviation correction unit. It is configured to include an inter-motion deviation correction unit that corrects the deviation of the subject in all the video frames of the single motion video based on the amount of deviation of the subject from the start video frame of the motion video.

According to this structure, the image generation apparatus uses the motion difference correction means and the single motion difference correction means, and the reference image in which the object is previously arranged at the initial position and the feature points of the object are set. Based on the amount of displacement of the subject from the start image frame of the corrected single motion image,
Correct the deviation of the subject in all video frames of a single motion video. At this time, the video generation device corrects the subject of all the video frames of the single motion video with the same shift amount based on the shift amount of the subject between the reference image and the start video frame.

Further, in the image generating apparatus according to claim 4, in the image generating apparatus according to claim 3, the motion gap correcting means determines the characteristic points of the subject between the reference image and the start image frame. Based on the shift amount, the shift of the subject is corrected by performing warping on the subject in all video frames of the single motion video.

According to such a configuration, the video generation apparatus causes the motion gap correction means to detect the subjects of all the video frames of the single motion video based on the shift amount of the feature points of the subject between the reference image and the start video frame. With respect to, the warping is performed with the same shift amount to correct the shift of the subject.

According to a fifth aspect of the present invention, there is provided a video generation device,
Based on the two video frames in which the subject is arranged, a fine deviation correction means is provided for adding an interpolation frame for smoothly deforming the shape of the subject of one video frame to the subject of the other video frame over time. , Adding the reference image before the start video frame and after the end video frame of the single motion video corrected by the motion gap correction means, and by the fine shift correction means, the start video frame and the front thereof. The interpolating frame is added between the reference image added to the above and the interpolating frame is added between the end video frame and the reference image added thereafter.

According to such a configuration, the video generation apparatus provides the reference image before the start video frame and after the end video frame of the single motion video corrected by the motion shift correction means by the fine shift correction means. In addition, an interpolation frame in which the shape of the subject is sequentially converted is added between the start video frame and the reference image added before the start video frame, and the interpolation frame is added between the end video frame and the reference image added thereafter. An interpolation frame in which the shape of the subject is sequentially converted is added.

Further, in the image generating device according to claim 6, in the image generating device according to claim 5, the fine deviation correction means forms an interpolation frame by morphing between two image frames in which a subject is arranged. The configuration was added.

According to this structure, the image generation device adds the interpolation frame in which the shape of the subject is sequentially converted by morphing between the two image frames in which the subject is arranged by the fine shift correction means.

According to a seventh aspect of the present invention, there is provided a video generation program for connecting a plurality of single motion video images taken for each type of motion of a subject to generate a continuous motion video of the subject. Based on a preset characteristic point that is a feature of the shape of the subject, the unit is made to function as a single action shift correcting unit that corrects the shift of the subject in each video frame of the single action video, The positions of the subject in the start video frame and the end video frame of the video frame are matched.

According to such a configuration, the image generation program causes the single motion deviation correction means to detect the object in each image frame of the single motion image based on the characteristic points that are characteristics of the shape of the object set in advance. Displacement correction is performed to match the positions of the subject in the start video frame and the end video frame of the single motion video. At this time, the video generation program sets the distance between the start video frame and the end video frame based on the amount of deviation between the feature points of the subject in the start video frame in the single motion video and the feature points of the subject in the end video frame. Correct the displacement of the subject in the frame.

Further, the image generation program according to claim 8 is the image generation program according to claim 7, wherein the subject is arranged in advance at an initial position and a feature point of the subject is set, As inter-motion shift correction means for correcting the shift of the subject in all the video frames of the single motion video based on the shift amount of the subject from the start video frame of the single motion video corrected by the single motion shift correction means It is configured to function.

According to such a configuration, the image generation program is configured such that the motion shift correction unit causes the motion image shift correction unit to preliminarily place the subject at the initial position and sets the feature point of the subject, and the single motion shift correction unit. The deviation of the subject in all the video frames of the single motion video is corrected based on the corrected amount of the deviation of the subject from the start video frame of the single motion video. At this time, the video generation program corrects the subject in all video frames of the single motion video with the same shift amount based on the shift amount of the subject between the reference image and the start video frame.

Further, in the image generation program according to claim 8, the image generation program according to claim 8 is based on the two image frames in which the object is arranged, from the object of one image frame to the object of the other image frame. Before the start image frame of the single motion image corrected by the motion gap correction unit is caused to function as a fine shift correction unit that adds an interpolation frame that smoothly deforms the shape of the subject of the video frame with time. The reference image is added after and after the end video frame, an interpolation frame is added between the start video frame and the reference image added before by the fine shift correction unit, and the end video frame and An interpolated frame is added between the reference image added to the above and the reference image.

According to this structure, the video generation program provides the reference image before the start video frame and after the end video frame of the single motion video corrected by the motion shift correction means by the fine shift correction means. In addition, an interpolation frame in which the shape of the subject is sequentially converted is added between the start video frame and the reference image added before the start video frame, and the interpolation frame is added between the end video frame and the reference image added thereafter. An interpolation frame in which the shape of the subject is sequentially converted is added.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described in detail below with reference to the drawings. (Structure of video generation device)

FIG. 1 is a block diagram showing the configuration of a video generation apparatus which is an embodiment of the present invention. As shown in FIG. 1, the image generation device 1 includes a main control unit 2, a RAM 3, and an R.
The OM 4, the VRAM 5, the external storage unit 6, the image composition unit 7, the external input / output unit 8, the display unit 9, the external signal input unit 10, and the bus 11 are configured.

The image generation apparatus 1 uses a CG image and a single action image obtained by photographing a basic action of a person (subject), for example, a single action such as "bowing" or "looking sideways". The continuous motion image obtained by connecting the single motion images is CG.
This is a device that synthesizes images.

The main control unit 2 is specifically composed of a CPU, is connected to each hardware described later via a bus 11, and controls them, as well as a single operation deviation correction means described later. The software functions such as 2a, movement difference correction means 2b, and fine deviation correction means 2c are executed.
The single motion deviation correction means 2a, the movement deviation correction means 2b, and the fine deviation correction means 2c may be recorded in a recording medium as a video generation program and distributed for each individual function or each function. Is.

A RAM (Random Access Memory) 3 is a work area used when the main control unit 2 executes a program, and is composed of a general RAM such as SRAM. The main control unit 2 sends temporary data to the R via the bus 11.
Write to AM3 and read from RAM3 as needed. In particular, in the present invention, mainly the single motion deviation correction means 2a, the inter-motion deviation correction means 2b, and the fine deviation correction means 2c.
It is used as a work area when executing image processing.

A ROM (Read Only Memory) 4 stores a program necessary for the function of the image generating apparatus 1, and the main control unit 2 operates based on this program. The ROM 4 is an EPROM (Erasable and Programmable).
ble ROM) and flash memory.

A VRAM (Video RAM) 5 is written with data for display on a display device (not shown) such as a CRT, and a display unit 9 described later outputs the data written in the VRAM 5 to the display device. The VRAM 5 is composed of, for example, a dual port DRAM (Dynamic Random Access Memory) capable of simultaneously inputting and outputting data.

The external storage unit 6 is a storage unit for accumulating a single motion image 6a, a CG image 6b, and feature point data 6c input from the outside via an external input / output unit 8 which will be described later.
Specifically, it is composed of a hard disk or the like.

The single motion image 6a stored in the external storage unit 6 is a still image file divided into video frames in which the basic motion of a person is a single motion. Also, C
The G image 6b is a still image file obtained by dividing the CG image to be combined with the motion image of the person into image frames.
The feature point data 6c is a data file in which the positions that are the features of the shape of the person are plotted for each still image of the single motion image 6a.

Here, the feature point data 6c will be further described. The feature point data 6c is developed by expanding each still image of the single motion image 6a in the VRAM 5, displaying it on a display device (not shown) such as a CRT through a display unit 9 described later, and inputting a device such as a mouse ( The position designated by (not shown) is analyzed via an external signal input unit 10 described later, and stored as feature point data 6c. For example, a feature point is set on the outline of a person for each still image of the single motion image 6a. The number of feature points varies depending on the size and complexity of the human image. In addition, the feature point data 6c is
It is also possible to store data created externally in the external storage unit 6 via the external input / output unit 8.

The image synthesizing unit 7 separates the foreground and the background based on the gradation value of the specific color from the image of the person existing on the uniform background of the specific color, and the image of the separated foreground person. Is combined with the CG image. For example, a composite CG image can be generated by combining the single motion image 6a and the CG image 6b.

The external input / output unit 8 stores the single motion image 6a, the CG image 6b, and the feature point data 6c stored in the external storage unit 6.
Is an interface unit that externally inputs data and outputs data generated by operating the video generation device 1. In this configuration, the image data is input / output to / from the outside as a still image file, but the external input / output unit 8 has a video dividing means for dividing the video signal into still images for each video frame. Generate a still image file from the external video signal by dividing means,
Further, a configuration may be adopted in which a video reproduction means for generating a still picture file into a video signal is provided and the still picture file is outputted to the outside as a video signal by this video reproduction means.

The display unit 9 outputs the data written in the VRAM 5 as a display signal to a display device (not shown) such as a CRT. This allows the operator to create the feature point data 6c while confirming on a screen such as a CRT,
It is possible to confirm the composite CG image.

The external signal input section 10 inputs an input signal from an input device (not shown) such as a mouse or a keyboard. The operator can instruct a feature point or execute a video generation program by using a mouse or a keyboard. (Operation of video generation device)

Next, the operation of the image generating apparatus 1 will be described with reference to FIGS. FIG. 2 shows an image generation apparatus 1
It is a schematic diagram showing the flow of processing of.

Here, the single motion image 6 in the external storage unit 6
a is composed of an original single motion image n1, an original single motion image n2, an original single motion image n3, ... Which are different for each type of motion, and the corrected single motion is performed by the operation of the video generation device 1. It is assumed that the image n1 ′, the corrected single motion image n2 ′, the corrected single motion image n3 ′, ... Are generated.

The operation of the image generating apparatus 1 is roughly divided into three steps. First, the single motion deviation correction means 2a simply matches the position of the person in the start video frame (hereinafter referred to as the start point frame) of the original single motion image 1 with the end video frame (hereinafter referred to as the end point frame). One motion deviation correction is performed (S1). In the original single-motion image 2, the original single-motion image 3, ...

Next, the movement difference correction means 2b selects one of the starting point frames of each single movement corrected by the single movement movement correction means 2a as the reference image Fc,
Inter-motion displacement correction is performed so that the human positions in all the starting point frames of the single motion image corrected by the single motion displacement correction match the human positions of the reference image (S2).

Then, the minute shift correcting means 2c causes a minute shift between the reference image and the starting point frame and between the reference image and the ending point frame with respect to the frame corrected by the motion difference correcting means 2b. To correct
Corrected single motion image n1 ′, corrected single motion image n2 ′,
The corrected single motion image n3 ′, ... Is generated (S3).

Next, referring to FIGS. 3 to 5, the single motion deviation correcting means 2a and the inter-motion deviation correcting means 2 which are the respective means described above.
b, the operation of the fine deviation correction means 2c will be described in detail.

FIG. 3 is a conceptual diagram for explaining the operation of the single operation deviation correction means 2a. FIG. 3A is a sequence diagram showing the contents of each frame of the single motion image 6a. Here, a single motion image in which a person faces the front (starting point frame Fs), turns right, and finally faces the front again (end point frame Fe) will be described as an example. In addition, it is assumed that a feature point that is a feature of the shape of the person is preset in each frame. This feature is
Although shown in the figure for convenience of explanation, it is actually stored as feature point data 6c in the external storage unit 6.

The single motion deviation correcting means 2a corrects the deviation within the single operation. The single motion image 6a is a real image of a person, and since the position of the person is displaced between the starting point frame Fs and the ending point frame Fe, the characteristic point of the person in the ending point frame Fe and the starting point frame Fs are compared. The position of the person in the end point frame Fe is corrected so that the feature points match. Along with this, the intermediate frame Fn between the starting point frame Fs and the ending point frame Fe is also corrected for the person position so that the person moves smoothly.

Specifically, first, the starting point frame F
The shift amount of each feature point is calculated for the person s and the person of the end point frame Fe. For example, in the single motion image shown in FIG. 3A, the characteristic points of the person in the starting point frame Fs are represented by Ps (xs,
ys) and the feature point of the person in the end point frame Fe, which is the same position as the feature point, is Pe (xe, ye), the feature point Pe is moved to the feature point Ps in order to match the position of the person. The amount of deviation can be represented as the magnitude of the motion vector from the characteristic point Pe to the characteristic point Ps. Therefore, the shift amount is calculated in the same manner as above for all the characteristic points of the person in the end point frame Fe.

Next, the corrected positions of the characteristic points of the intermediate frame Fn between the starting point frame Fs and the ending point frame Fe are obtained. First, the shift amount of the feature points of the intermediate frame Fn may be a value obtained by multiplying the maximum shift amount by the shift amount between the starting point frame Fs and the end point frame Fe and the ratio according to the number of frames. Good.

For example, in FIG. 3A, in the nth intermediate frame Fn counted from the starting point frame Fs (Fs is the first), the feature point Ps of the person in the starting point frame Fs.
The shift amount of the feature point Pn (xn, yn) corresponding to (xs, ys) is ((x
e−xs) × (n−1) / (f−1)), and the Y direction is ((ye−ys) × (n−1) / (f−1)).
That is, the feature point position Pn ′ to which the feature point Pn (xn, yn) should move is Pn ′ = (xn− (xe−xs) × (n
-1) / (f-1), yn- (ye-ys) x (n-
1) / (f-1)). Then, the intermediate frame Fn
The corrected position is obtained for all the characteristic points of the person in the same manner as described above. The corrected position of the feature point of the end point frame Fe is the feature point of the person in the starting point frame Fs itself. By the above procedure, the corrected positions of the feature points other than the starting point frame Fs are obtained.

Then, in each frame other than the starting point frame Fs, based on the motion vector from the pre-correction feature point to the post-correction feature point, transformation mapping conversion is performed to move the human image to the post-correction feature point position. Let It should be noted that this modified mapping conversion can be realized using the existing technology of warping. As a result, the starting point frame F
s and the person position of the end point frame Fe match. Figure 3
(2) is a sequence diagram showing the contents of each frame of the single motion image after the correction of the single motion shift, and shows that the positions of the person in the starting point frame Fs and the ending point frame Fe match. .

Then, by performing the above procedure for each single motion image of the person photographed individually, the single motion image 6a is obtained.
Is updated to an image in which the deviation between the starting point frame Fs and the ending point frame Fe is corrected.

Next, with reference to FIG. 4, the operation of the inter-motion deviation correcting means 2b will be described. FIG. 4 is a conceptual diagram for explaining the deviation correction between the operations for connecting the single operation image 6a generated by the single operation deviation correction unit 2a to another different operation.

The movement displacement correcting means 2b sets the person position in the starting point frame Fs of the single movement image 6a after the single movement displacement correction and the person's characteristic point in advance by arranging the person at the initial position. The deviation from the person position in the reference image Fc is corrected.

Specifically, first, one is selected from the starting point frames Fs of the plurality of single motion images 6a corrected by the single motion shift correcting means 2a, and is set as the reference image Fc.
As the reference image Fc, it is also possible to use an image in which the person is arranged in the initial position in advance.

Next, from the person feature points of the starting point frame Fs of the reference image Fc and each single motion image, the person feature points of the starting point frame Fs are moved to the person positions of the reference image Fc. The motion vector Ve is calculated.

Then, the transformation mapping conversion is performed based on this motion vector Ve for all the frames of the single motion image 6a after the single motion shift correction shown in FIG. 4A.
Starting point frame Fs of reference image Fc and single motion image 6a
Match the person positions in. It should be noted that this modified mapping conversion can be realized using the existing technology of warping. FIG. 4 (2) shows the single motion image 6a after the motion gap correction.

Then, by performing the inter-motion displacement correction on all the single motion images 6a after the single motion displacement correction, the persons in all the single motion images 6a become the reference images Fc.
The movement is corrected so that it starts from the position of the person.

Next, the operation of the fine deviation correction means 2c will be described with reference to FIG. FIG. 5 shows a single motion image (FIG. 5 (1)) after the motion gap correction, and a single motion image corrected by the fine shift correction (FIG. 5 (2) single motion image after the fine shift correction). FIG. 7 is a conceptual diagram for explaining an operation of generating a.

The position of the person in the starting point frame Fs and the ending point frame Fe of each operation is roughly the reference image Fc by the single operation deviation correction and the movement deviation correction.
It matches the position of the person in. However, there are differences other than the position of the person, such as slight differences in facial expressions. Therefore, the fine deviation correction unit 2c adds the reference image Fc before the starting point frame Fs of the single motion image after the movement deviation correction, and sequentially converts the person in the reference image Fc into the person in the starting point frame Fs. The reference image Fc and the starting point frame F so that the shape is smoothly deformed with the passage of time.
An interpolation frame is inserted between s and s (FIG. 5 (2)). The insertion of the interpolation frame can be realized by using morphing which is an existing technique.

Further, the fine shift correcting means 2c adds the reference image Fc after the end point frame Fe of the single motion image after the motion shift correction, and sequentially from the person of the end point frame Fe to the person of the reference image Fc. Conversion is performed and the end point frame Fe is set so that the shape is smoothly deformed with the passage of time.
An interpolation frame is inserted between the reference image Fc and the reference image Fc.

By executing the above procedure for each single motion image, a corrected single motion image n1 ', a corrected single motion image n2', a corrected single motion image n3 ', ... Are generated. .

The corrected single motion image n generated as described above
1 ', corrected single motion image n2', corrected single motion image n
3 ′, ... Are CG images 6b by the image composition unit 7 of FIG.
It is possible to generate a composite CG image in which a real-life person is composited on a CG video image by combining with. At this time, various continuous motion images can be generated by combining the individual motions of the corrected single motion image 1, the corrected single motion image 2, the corrected single motion image 3, ...

Further, in the present embodiment, the single motion video in which the motion of the person is photographed has been described, but the subject of the video does not have to be a person, and is not limited to a person as long as it is a moving subject video. Not something.

[0063]

As described above, the video generating apparatus and the video generating program according to the present invention have the following excellent effects.

According to the first aspect of the present invention, in the image generation device, the single motion deviation correction means causes each of the single motion images to be generated on the basis of the characteristic points that are the characteristics of the shape of the object set in advance. It is possible to correct the shift of the subject in the video frame and match the positions of the subject in the start video frame and the end video frame of the single motion video.

Thus, for example, "bow",
When a person's basic movement, such as “looking sideways,” is taken as a single action, even if the person moves to a position different from the initial position after the end of the operation, it can be corrected to the initial position. It is possible to easily perform the shooting of the basic operation and reduce the shooting time.

According to the second aspect of the invention, in the video generation device, the single motion shift correction means causes the shift amount of the feature points of the start video frame and the end video frame and the number of video frames of the single motion video. Based on the shift amount of the characteristic points of each video frame calculated from the above, the shift of the subject can be corrected by performing warping on the subject of each video frame.

As a result, when the position shift of the subject between the start image frame and the end image frame of the single motion image is corrected to match the positions of the subjects, the objects in the frames other than the start image frame are warped. Since the positional deviation is corrected, it is possible to realize a smooth movement of the subject without discontinuity from the start frame to the end frame.

According to the third aspect of the present invention, in the image generation device, the motion image shift correcting means sets the reference image in which the subject is arranged in the initial position in advance and the feature points of the subject are set, and the single image. It is possible to correct the deviation of the subject in all the video frames of the single motion video based on the amount of deviation of the subject from the start video frame of the single motion video corrected by the motion deviation correction means.

As a result, since the position of the subject in the end video frame of the single motion video matches the position of the subject in the start video frame of the other single motion video with reference to the subject of the reference image, each single motion video is It becomes possible to create continuous motion images by connecting them. It is also possible to create different continuous motion images by changing the combination of individual single motion images. Even when creating images in which the continuous motion of the subject is different, it is possible to take a new motion of the subject. Instead, since the single motion image can be reused, the time for creating the continuous motion image can be shortened.

According to the fourth aspect of the invention, in the video generation device, the motion shift correction means, based on the shift amount of the feature point of the subject between the reference image and the start video frame,
The subject shift can be corrected by performing warping on the subject in all video frames of the single motion image with the same shift amount.

As a result, in the single motion image, the position of the subject is moved so that the motion of the subject starts at the position of the subject in the reference image and ends at the position of the subject in the reference image.
Further, since all the video frames of the single motion image are warped based on the same displacement amount, it is possible to realize a smooth motion of the subject without discontinuity in the single motion image.

According to the invention described in claim 5, in the image generation device, before the start video frame and after the end video frame of the single motion video corrected by the motion shift correction means by the fine shift correction means. And a reference image added to the start video frame, and an interpolation frame in which the shape of the subject is sequentially converted between the start video frame and the reference image added before the start video frame, and the end video frame and the reference image added after that. An interpolation frame in which the shape of the subject is sequentially converted can be added between and.

As a result, between the start image frame of the reference image and the single motion image and between the end image frame of the reference image and the single motion image, for example, a difference in facial expression of a person, a fine pattern other than the position of the subject is displayed. The single motion image in which the movement of the subject can be corrected from the position of the reference image and the motion of the subject ends at the position of the reference image enables smooth movement even in the case of minute changes such as facial expressions of the subject. Can be realized.

According to the sixth aspect of the present invention, the image generation apparatus adds the interpolation frame in which the shape of the subject is sequentially converted by morphing between the two video frames in which the subject is arranged by the fine shift correction means. be able to.

Thus, it is possible to add interpolation frames in which the shape of the subject is sequentially converted by morphing between the reference image and the start video frame of the single motion video and between the reference image and the end video frame of the single motion video. So
The single motion video in which the motion of the subject starts from the position of the reference image and ends at the position of the reference image can realize smooth motion without discontinuity even in the case of minute changes such as facial expressions of the subject. it can. In addition, when connecting the single motion images, it is possible to realize smooth movement without discontinuity even in minute movements such as the position and facial expression of the subject.

According to the seventh aspect of the present invention, the image generation program is based on the characteristic points which are the characteristics of the shape of the object preset by the single motion deviation correction means.
It is possible to correct the shift of the subject in each video frame of the single-action video, and match the positions of the subject in the start video frame and the end video frame of the single-action video.

Thus, for example, "bow",
When a person's basic movement, such as “looking sideways,” is taken as a single action, even if the person moves to a position different from the initial position after the end of the operation, it can be corrected to the initial position. It is possible to easily perform the shooting of the basic operation and reduce the shooting time.

According to the eighth aspect of the present invention, the image generation program includes the reference image in which the object is arranged in the initial position in advance and the feature points of the object are set by the motion deviation correcting means, and the single image. It is possible to correct the deviation of the subject in all the video frames of the single motion video based on the amount of deviation of the subject from the start video frame of the single motion video corrected by the motion deviation correction means.

As a result, since the position of the subject in the end video frame of the single motion video matches the position of the subject in the start video frame of the other single motion video with respect to the subject of the reference image, each single motion video is It becomes possible to create continuous motion images by connecting them. It is also possible to create different continuous motion images by changing the combination of individual single motion images. Even when creating images in which the continuous motion of the subject is different, it is possible to take a new motion of the subject. Instead, since the single motion image can be reused, the time for creating the continuous motion image can be shortened.

According to the ninth aspect of the present invention, the image generation program is configured such that the fine shift correcting unit corrects the single motion video corrected by the motion shift correcting unit before the start video frame and after the end video frame. And a reference image added to the start video frame and the reference image added in front of the end video frame and the reference image added in front of the start video frame. An interpolation frame in which the shape of the subject is sequentially converted can be added between and.

As a result, between the start image frame of the reference image and the single motion image and between the end image frame of the reference image and the single motion image, for example, a difference in facial expression of a person, a fine pattern other than the position of the subject is displayed. The single motion image in which the movement of the subject can be corrected from the position of the reference image and the motion of the subject ends at the position of the reference image enables smooth movement even in the case of minute changes such as facial expressions of the subject. Can be realized.

[Brief description of drawings]

FIG. 1 is a block diagram showing an overall configuration of a video generation device that is an embodiment of the present invention.

FIG. 2 is a schematic diagram showing a flow of processing of the video generation device according to the embodiment of the present invention.

FIG. 3 is a conceptual diagram for explaining an operation of single operation deviation correction in the present invention.

FIG. 4 is a conceptual diagram for explaining an operation of correcting a movement gap in the present invention.

FIG. 5 is a conceptual diagram for explaining an operation of fine deviation correction in the present invention.

[Explanation of symbols]

1 ... Image generation device 2 ... Main control unit 2a ... single motion deviation correction means 2b ... Operation gap correction means 2c ... Fine deviation correction means 3 ... RAM 4 ... ROM 5 ... VRAM 6 ... External storage 6a ... Single motion image 6b ... CG image 6c ... Characteristic point data 7: Image composition section 8: External input / output section 9 ... Display 10 ... External signal input section 11 ... bus

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/262 H04N 5/262 (72) Inventor Kei Yokoyama 1-8-17 Daido, Kanazawa-ku, Yokohama-shi, Kanagawa F term (reference) 5B050 AA08 BA06 BA12 DA07 EA05 EA13 EA19 EA24 FA02 5B057 AA20 CA12 CB12 CC01 CD06 CE08 CH01 CH12 CH14 DA07 DA16 DB02 DC05 DC32 5C023 AA16 AA37 BA04 BA11 CA01 DA02 DA03 DA04 EA03 5A0 FA33 FA33 5L096 FA0

Claims (9)

[Claims] 1. An image generating apparatus for generating a continuous motion image of a subject by connecting a plurality of single motion images photographed for each type of motion of the subject, wherein the shape feature of the subject is set in advance. A single motion shift correction means for correcting the shift of the subject in each video frame of the single motion video based on the feature point, and the position of the subject in the start video frame and the end video frame of the single motion video is determined. An image generation device characterized by matching. 2. The feature of each video frame calculated by the single motion shift correction means from the amount of shift between feature points of the start video frame and the end video frame and the number of video frames of the single motion video. The image generation apparatus according to claim 1, wherein the object shift is corrected by performing warping on the object of each of the video frames based on the amount of shift of the points. 3. A reference image in which the subject is arranged in an initial position in advance and feature points of the subject are set in advance, and a start image frame of a single action image corrected by the single action shift correction means The inter-motion deviation correction means for correcting the deviation of the subject in all video frames of the single motion video based on the deviation amount.
Alternatively, the video generation device according to claim 2. 4. The motion gap correction unit warps the objects in all video frames of the single motion video based on the amount of deviation of the feature points of the object between the reference image and the start video frame. The image generation apparatus according to claim 3, wherein the deviation of the subject is corrected by performing the correction. 5. A fine frame adding, based on two video frames on which a subject is arranged, an interpolation frame for smoothly deforming the shape from the subject of one video frame to the subject of the other video frame over time. Shift correction means, adding the reference image before the start video frame and after the end video frame of the single motion video corrected by the motion shift correction means, and by the fine shift correction means, the start 4. An interpolation frame is added between a video frame and a reference image added before it, and an interpolation frame is added between the end video frame and a reference image added after that. Alternatively, the video generation device according to claim 4. 6. The image generation apparatus according to claim 5, wherein the fine deviation correction unit adds an interpolation frame by morphing between two image frames in which a subject is arranged. 7. A computer for connecting a plurality of single motion images captured for each motion type of a subject to generate a continuous motion image of the subject, and a preset feature of the shape of the subject. Based on the feature points, the single motion image is made to function as a single motion deviation correction unit that corrects the object in each video frame of the single motion video. An image generation program characterized by matching the positions of subjects. 8. The image generation program according to claim 7, wherein the subject is arranged at an initial position in advance and a feature point of the subject is set, and a single image corrected by the single motion deviation correcting unit. A video generation program, which functions as an inter-motion deviation correction unit that corrects the deviation of the subject in all the video frames of the single motion video based on the amount of deviation of the subject from the start video frame of the one motion video. . 9. The video generation program according to claim 8, wherein, based on the two video frames in which the subject is arranged, the subject of one video frame is smoothly moved to the subject of the other video frame over time. The reference image before the start video frame and after the end video frame of the single motion video corrected by the motion gap correction means. In addition, an interpolation frame is added between the start video frame and the reference image added before it by the fine shift correction means, and interpolation is performed between the end video frame and the reference image added after that. A video generation program characterized by adding frames.