JP5919926B2 - Image generation method, image generation apparatus, and program - Google Patents

Description

  The present invention relates to an image generation method, an image generation apparatus, and a program.

  There is known a technique for generating a moving image by deforming a two-dimensional original image so as to conform to motion data recorded in advance (see, for example, Patent Document 1).

  In Patent Document 1, a character in a two-dimensional original image input from an image input device is extracted in accordance with a character instruction, and the extracted character is associated with character basic structure data in accordance with an association instruction. Furthermore, it describes that the associated character is deformed according to motion data to generate and display a moving image.

JP-A-10-040407

  However, in the conventional image generation method and the like, it is possible to arrange a plurality of characters and generate a moving image by deforming the plurality of characters, but it is not possible to easily change the context between characters. There was a problem I said.

For example, consider a case where a moving image in which two characters are arranged in front of and behind each other is generated, and the front character is moved backward in order to change the front-rear relationship between the characters.
At this time, the depth information of the front character is front information, and when this character is smoothly moved backward, the depth information is sequentially recalculated based on the moving position, moving distance, etc. There is a problem that the rewriting process is required, and the processing load of the processing means increases, so that the context between characters cannot be easily changed.

  An object of the present invention is to provide an image generation method, an image generation apparatus, and a program that can easily change the context of characters when generating a moving image by deforming a plurality of arranged characters. .

In order to achieve the above object, according to one aspect of the present invention, an acquisition step of acquiring the first and second images arranged by shifting the front-rear position and a position on the back side of the first and second images are provided. A first generation step of generating a superimposed image that is superimposed on the back side image and one of the front side and the back side is substantially equal to one side of the back side image; and the first and second images A first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, and the first and second images. among them, the front side of the front side image, seen including a second generation step of generating a moving image and a second frame image is superimposed on the far side image the superimposed image as an invisible state, wherein the 1 generation step includes the front surface as the superimposed image. And the other side of the back side that is different from the one side is generated, and the second generation step uses the superimposed image as the second frame image with the other side as the near side. The present invention provides an image generation method characterized by generating a moving image including a frame image superimposed on a back side image .

  According to the present invention, the acquisition step of acquiring the first and second images arranged with the front and rear positions shifted, and the first and second images are superimposed on the back side image located on the back side, From the first generation step of generating a superimposed image in which either one of the front side and the back side is substantially equal to one side of the back image, and the near side image located on the near side among the first and second images The first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side, and the superimposed image on the near side of the near side image among the first and second images. The second generation step of generating a moving image including the second frame image superimposed on the back side image as described above makes it possible to easily change the front-rear relationship of the characters.

It is a block diagram which shows schematic structure of the moving image production | generation system of one Embodiment to which this invention is applied. It is a block diagram which shows schematic structure of the user terminal which comprises the moving image production | generation system of FIG. It is a block diagram which shows schematic structure of the server which comprises the moving image production | generation system of FIG. It is a flowchart which shows an example of the operation | movement which concerns on the back-and-front replacement setting process by the moving image generation system of FIG. It is a flowchart which shows an example of the operation | movement which concerns on the front-back replacement control process by the moving image production | generation system of FIG. It is a top view explaining operation | movement of a some character. It is a front view explaining operation | movement of a some character. It is a top view explaining operation | movement of a some character. It is a front view explaining operation | movement of a some character. It is a top view explaining operation | movement of a some character. It is a front view explaining operation | movement of a some character.

  Hereinafter, an image generation method, an image generation apparatus, and a program according to an embodiment of the present invention will be described in detail with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples.

(First embodiment)
A moving image generating system according to a first embodiment of the present invention will be described with reference to FIGS.

  As shown in FIG. 1, the moving image generation system 100 of the present embodiment includes an imaging device 1, a user terminal 2, and a server 3, and the user terminal 2 and the server 3 are variously connected via a predetermined communication network N. It is connected so that information can be sent and received.

First, the imaging device 1 will be described.
The imaging device 1 includes an imaging function for imaging a subject, a recording function for recording image data of a captured image on the recording medium C, and the like. That is, as the imaging device 1, a known device can be applied. For example, not only a digital camera having a main function as an imaging function but also a portable terminal such as a mobile phone having an imaging function that is not a main function. Including.

Next, the user terminal 2 will be described with reference to FIG.
The user terminal 2 is configured by, for example, a personal computer, and accesses a Web page (for example, a moving image generation page) opened by the server 3 and inputs various instructions on the Web page.

FIG. 2 is a block diagram illustrating a schematic configuration of the user terminal 2.
As shown in FIG. 2, specifically, the user terminal 2 includes a central control unit 201, an operation input unit 202, a display unit 203, a sound output unit 204, a recording medium control unit 205, and a communication control unit. 206 and the like.

The central control unit 201 controls each unit of the user terminal 2. Specifically, the central control unit 201 includes a CPU, a RAM, and a ROM (all not shown), and performs various control operations according to various processing programs (not shown) for the user terminal 2 stored in the ROM. At that time, the CPU stores various processing results in a storage area in the RAM, and displays the processing results on the display unit 203 as necessary.
The RAM includes, for example, a program storage area for expanding a processing program executed by the CPU, a data storage area for storing input data, a processing result generated when the processing program is executed, and the like.
The ROM stores programs stored in the form of computer-readable program codes, specifically, system programs that can be executed by the user terminal 2, various processing programs that can be executed by the system programs, and execution of these various processing programs. The data used for the storage is stored.

The operation input unit 202 is, for example, a keyboard or mouse configured with data input keys for inputting numerical values, characters, and the like, up / down / left / right movement keys for performing data selection, feeding operations, and the like, and various function keys And a key press signal and a mouse operation signal pressed by the user are output to the CPU of the central control unit 201.
Note that a touch panel (not shown) may be disposed on the display screen of the display unit 203 as the operation input unit 202 and various instructions may be input according to the touch position of the touch panel.

The display unit 203 includes a display such as an LCD or a CRT (Cathode Ray Tube), and displays various types of information on the display screen under the control of the CPU of the central control unit 201.
That is, for example, the display unit 203 displays a corresponding Web page on the display screen based on page data of a Web page (for example, a moving image generation page) transmitted from the server 3 and received by the communication control unit 206. Specifically, the display unit 203 displays various processing screens on the display screen based on image data of various processing screens related to the moving image generation processing (described later).

The sound output unit 204 includes, for example, a D / A converter, an LPF (Low Pass Filter), an amplifier, a speaker, and the like, and emits sound under the control of the CPU of the central control unit 201.
That is, for example, the sound output unit 204 converts digital data of the performance information into analog data by a D / A converter based on performance information transmitted from the server 3 and received by the communication control unit 206, and passes through an amplifier. The sound is emitted from the speaker with a predetermined tone color, pitch and pitch. Further, the sound output unit 204 may emit the sound of one sound source (for example, a musical instrument), or may emit the sounds of a plurality of sound sources simultaneously.

The recording medium control unit 205 is configured such that the recording medium C is detachable, and controls reading of data from the loaded recording medium C and writing of data to the recording medium C. In other words, the recording medium control unit 205 reads out image data (YUV data) of a subject existing image (not shown) related to a moving image generation process (described later) from the recording medium C that is detached from the imaging device 1 and attached, and performs communication control. The data is output to the unit 206.
Here, the subject presence image is an image in which a main subject exists within a predetermined background. In addition, on the recording medium C, image data of a subject existing image encoded according to a predetermined encoding format (for example, JPEG format) by an image processing unit (not shown) of the imaging apparatus 1 is recorded.
Then, the communication control unit 206 transmits the input image data of the subject existing image to the server 3 via the predetermined communication network N.

  The communication control unit 206 includes, for example, a modem (MODEM: Modulator / DEModulater), a terminal adapter (Terminal Adapter), and the like, and controls communication of information with an external device such as the server 3 via a predetermined communication network N. Is for doing.

  The communication network N is a communication network constructed using, for example, a dedicated line or an existing general public line, and various line forms such as a LAN (Local Area Network) and a WAN (Wide Area Network) are applied. Is possible. The communication network N includes, for example, various communication line networks such as a telephone line network, an ISDN line network, a dedicated line, a mobile communication network, a communication satellite line, and a CATV line network, and an Internet service provider that connects them. included.

Next, the server 3 will be described with reference to FIG.
The server 3 has a function of opening a Web page (for example, a moving image generation page) on the Internet as a Web (World Wide Web) server. The server 3 is connected to the user terminal 2 in response to an access from the user terminal 2. The page data of the web page is transmitted. Further, the server 3, as an image generation device, specifies color information of the contour portion of the subject region based on the skeleton information related to the subject skeleton of the subject image including the subject region, and the color of the contour portion of the subject region. Based on the information, a subject corresponding area of the rear image is drawn for each part of the subject, and a rear image representing the back side of the subject is generated.

FIG. 3 is a block diagram illustrating a schematic configuration of the server 3.
As shown in FIG. 3, the server 3 specifically includes a central control unit 301, a display unit 302, a communication control unit 303, a subject clipping unit 304, a storage unit 305, a moving image processing unit 306, and the like. It is prepared for.

The central control unit 301 controls each unit of the server 3. Specifically, the central control unit 301 includes a CPU, a RAM, and a ROM (all not shown), and the CPU performs various control operations according to various processing programs (not shown) for the server 3 stored in the ROM. Do. At that time, the CPU stores various processing results in a storage area in the RAM, and displays the processing results on the display unit 302 as necessary.
The RAM includes, for example, a program storage area for expanding a processing program executed by the CPU, a data storage area for storing input data, a processing result generated when the processing program is executed, and the like.
The ROM executes a program stored in the form of computer-readable program code, specifically, a system program that can be executed by the server 3, various processing programs that can be executed by the system program, and these various processing programs. The data used at the time is stored.

  The display unit 302 includes a display such as an LCD and a CRT, and displays various types of information on the display screen under the control of the CPU of the central control unit 301.

The communication control unit 303 includes, for example, a modem, a terminal adapter, and the like, and is for performing communication control of information with an external device such as the user terminal 2 via a predetermined communication network N.
Specifically, for example, the communication control unit 303 receives the image data of the subject existing image transmitted from the user terminal 2 via the predetermined communication network N in the moving image generation process (described later), and the image data Is output to the CPU of the central control unit 301.
The CPU of the central control unit 301 outputs the input image data of the subject existing image to the subject clipping unit 304.

The subject cutout unit 304 generates a subject cutout image (not shown) from the subject existing image. ("Subject image generation")
That is, the subject cutout unit 304 generates a subject cutout image in which a subject region including a subject is cut out from the subject existing image using a known subject cutout method. Specifically, the subject cutout unit 304 acquires the image data of the subject existing image output from the CPU of the central control unit 301 and, for example, the operation input unit 202 (for example, a mouse) of the user terminal 2 by the user. Based on the predetermined operation, the subject presence image is divided by a boundary line (not shown) drawn on the subject presence image displayed on the display unit 203. Subsequently, the subject cutout unit 304 estimates the background of the subject among a plurality of divided regions divided by the cutout line of the subject existing image, and performs a predetermined calculation based on the pixel value of each pixel of the background. Thus, the background color of the subject is estimated as a predetermined single color. Thereafter, the subject cutout unit 304 generates difference information (for example, a difference map or the like) of each corresponding pixel between a predetermined single color background image and the subject presence image. Then, the subject clipping unit 304 compares the pixel value of each pixel of the generated difference information with a predetermined threshold value, binarizes it, and then performs a labeling process for assigning the same number to the pixel set that constitutes the same connected component. A pixel set having the largest area is defined as a subject portion.
Thereafter, the subject clipping unit 304 applies a low-pass filter to the binarized difference information in which, for example, the pixel set having the largest area is “1”, and the other part is “0”, to the boundary part. An alpha value is generated by generating an intermediate value, and an alpha map (not shown) is generated as position information indicating the position of the subject region in the subject cutout image.
The alpha value (0 ≦ α ≦ 1) represents, for example, a weight when alpha blending an image of a subject area with respect to a predetermined background for each pixel of the subject existing image. In this case, the alpha value of the subject area is “1”, and the transparency of the subject existing image with respect to a predetermined background is 0%. On the other hand, the alpha value of the background portion of the subject is “0”, and the transparency of the subject existing image with respect to the predetermined background is 100%.

Then, based on the alpha map, the subject cutout unit 304 does not transmit the pixel having the alpha value “1” among the pixels of the subject existing image to the predetermined single color image, and the alpha value is The subject image is synthesized with a predetermined single color image so that the pixel of “0” is transmitted, and image data of the subject cut-out image is generated.
In addition, the subject cutout unit 304 sets each pixel in the subject region to a first pixel value (for example, “1”, etc.) based on the alpha map, and sets each pixel in the background region to a second value different from the first pixel value. A mask image (not shown), which is a binary image having a pixel value (for example, “0”, etc.), is generated. In other words, the subject cutout unit 304 generates a mask image as position information indicating the position of the subject region in the subject cutout image.
The image data of the subject cutout image is data associated with position information such as the generated alpha map and mask image, for example.

Note that the subject clipping method by the subject clipping unit 304 described above is an example and is not limited to this, and any known method may be used as long as it is a known method for clipping a subject area including a subject from a subject existing image. Also good.
Further, as image data of the subject cut-out image, for example, image data in RGBA format may be applied. Specifically, information on transparency (A) is added to each color defined in the RGB color space. Yes. In this case, the subject cutout unit 304 may generate position information (not shown) indicating the position of the subject region in the subject cutout image using the information on the transparency (A).

  The storage unit 305 includes, for example, a semiconductor non-volatile memory, an HDD (Hard Disc Drive), and the like. The page data of the Web page transmitted to the user terminal 2 and the image data of the subject cut-out image generated by the subject cut-out unit 304 are stored. Memorize etc.

In addition, the storage unit 305 stores a plurality of pieces of motion information 305a used for moving image generation processing.
Each motion information 305a is orthogonal to a predetermined space, that is, a two-dimensional plane space defined by, for example, two axes orthogonal to each other (for example, the x axis, the y axis, etc.) and the two axes in addition to the two axes. This is information indicating the movement of a plurality of movable points in a three-dimensional space defined by an axis (for example, the z axis). Note that the motion information 305a may be information that gives depth to the motion of a plurality of movable points by rotating a two-dimensional plane space around a predetermined rotation axis.
Here, the position of each movable point is defined in consideration of the shape of the skeleton of the moving body model (for example, a human or an animal) serving as a motion model, the position of the joint, or the like. In addition, the number of movable points can be arbitrarily set as appropriate according to the shape and size of the moving object model.
In addition, each movement information 305a is obtained by continuously arranging the coordinate information obtained by moving all or at least one of the plurality of movable points in a predetermined space at a predetermined time interval. It is expressed as Each of the coordinate information of the plurality of movable points may be, for example, information defining the amount of movement of each movable point with respect to the coordinate information of the movable point serving as a reference, or defining the absolute position coordinates of each movable point. It may be information.

The storage unit 305 stores a plurality of pieces of performance information 305b used for moving image generation processing.
The performance information 305b is information for automatically playing a song together with a moving image by a moving image reproducing unit 306e (described later) of the moving image processing unit 306. That is, a plurality of pieces of performance information 305b are defined, for example, with different tempo, time signature, pitch, scale, key, idea motto, etc., and are stored in association with the song titles.
Each piece of performance information 305b is digital data defined in accordance with, for example, the MIDI (Musical Instruments Digital Interface) standard. Specifically, the number of tracks, the resolution of quarter notes (number of Tick counts), and the like are defined. Header information, and track information including events and timings to be supplied to a sound source (for example, a musical instrument) assigned to each part. Examples of the track information event include a change in tempo or time signature, and information for instructing NoteOn / Off.

  The moving image processing unit 306 includes an image acquisition unit 306a, a back image setting unit 306b, a control point setting unit 306c, a frame generation unit 306d, a moving image reproduction unit 306e, a front / rear replacement setting unit 306f, and a front / rear replacement control unit 306g. It is equipped with.

The image acquisition unit 306a acquires a still image used for moving image generation processing. ("Subject image acquisition")
That is, the image acquisition unit 306a acquires a subject cutout image (subject image) obtained by cutting out a subject area including a subject from a subject existing image in which a background and a subject exist. Specifically, the image acquisition unit 306a acquires the image data of the subject cutout image generated by the subject cutout unit 304 as a still image to be processed.
Further, the image acquisition unit 306a acquires image data of a mask image associated with image data of a subject cutout image.

The back image generation unit 306b generates a back image that simulates the back side of the subject. ("Back image generation")
That is, the back image generation unit 306b generates the back image by drawing the subject corresponding areas of the back image for each part of the subject based on the color information of the contour portion of the specified subject area. Specifically, the rear image generation unit 306b uses the subject-corresponding region of the rear image as a part including the reference line segment of the subject, that is, representative human parts (for example, both arms, both legs, and the body). , The head, etc.) are respectively drawn to generate a back image.

The control point setting unit 306c sets a plurality of motion control points in the subject image of the subject clipped image to be processed. ("Specify movement")
That is, the control point setting unit 306c sets a plurality of motion control points at positions corresponding to a plurality of movable points in the subject image of the subject cutout image acquired by the image acquisition unit 306a. Specifically, the control point setting unit 306c reads the motion information 305a of the moving object model (for example, animal) from the storage unit 305, and the reference defined in the motion information 305a in the subject image of the subject clipped image. A motion control point corresponding to each of a plurality of movable points of a frame (for example, the first frame) is set at a desired position specified based on a predetermined operation of the operation input unit 202 of the user terminal 2 by the user. . In addition, the control point setting unit 306c sets a control point in the subject image of the subject cutout image, thereby setting a motion control point corresponding to a predetermined position in the rear image corresponding to the subject cutout image. Set automatically.
At this time, for the moving body model and the subject image, for example, the dimensions may be adjusted (for example, enlargement, reduction, deformation, etc. of the moving body model) so as to match the size of the main part such as the face. Further, for example, a position corresponding to each of a plurality of movable points in the subject image may be specified by superimposing the moving body model and the subject image.
Furthermore, the control point setting unit 306c may set corresponding motion control points for all of the plurality of movable points defined in the motion information 305a, or representative points such as the center portion and each tip portion of the subject. Only motion control points corresponding to a predetermined number of movable points may be set.

Note that the control point setting unit 306c includes a plurality of movable points of a reference frame (for example, the first frame) defined in the motion information 305a read from the storage unit 305 in the subject image of the subject clipped image. Each corresponding position may be automatically specified. For example, the control point setting unit 306c specifies a position corresponding to each of the plurality of movable points in consideration of the shape of the skeleton of the subject, the position of the joint, and the like. Then, the control point setting unit 306c sets a motion control point at a position corresponding to each of the specified plurality of movable points.
Even when the movement control point is automatically set by the control point setting unit 306c, the correction (change) of the setting position of the movement control point is accepted based on a predetermined operation of the operation input unit by the user. You may do it.

The frame generation unit 306d sequentially generates a plurality of frame images constituting the moving image. ("Generating frame images")
That is, the frame generation unit 306d sets a plurality of motion control points set in the subject image of the subject clipped image so as to follow the movements of the plurality of movable points of the motion information 305a specified by the moving image processing unit 306. A plurality of frame images are sequentially generated by moving. Specifically, for example, the frame generation unit 306d sequentially acquires coordinate information of a plurality of movable points that move at a predetermined time interval in accordance with the movement information 305a, and each movement control point corresponding to each of the movable points. Calculate the coordinates. Then, the frame generation unit 306d sequentially moves the motion control point to the calculated coordinates, and at the same time, a predetermined image area (for example, a triangle or a rectangle) set in the subject image with reference to at least one motion control point. A reference frame image (not shown) is generated by moving or deforming the mesh-shaped region.
Note that the process of moving or deforming a predetermined image region with reference to the motion control point is a known technique, and thus detailed description thereof is omitted here.

In addition, the frame generation unit 306d interpolates between two reference frame images that are adjacent to each other along a time axis generated based on a plurality of motion control points corresponding to each of the movable points after movement. (Not shown) is generated. That is, the frame generation unit 306d interpolates between two reference frame images so that a plurality of frame images are reproduced at a predetermined reproduction frame rate (for example, 30 fps) by the moving image reproduction unit 306e. Are generated in a predetermined number.
Specifically, the frame generation unit 306d, between two adjacent reference frame images,
The progress degree of the performance of the predetermined music played by the moving picture reproducing unit 306e is sequentially acquired, and an interpolated frame image reproduced between two adjacent reference frame images is sequentially generated according to the progress degree. For example, the frame generation unit 306d acquires tempo setting information and quarter note resolution (Tick count number) based on the MIDI standard performance information 305b, and plays a predetermined song played by the video playback unit 306e. Convert the elapsed time of to a Tick count. Subsequently, the frame generation unit 306d uses two Tick count numbers corresponding to the elapsed time of the performance of the predetermined song, and two adjacent references synchronized with a predetermined timing (for example, the first beat of each measure). The relative degree of progress of the performance of a predetermined song between frame images is calculated, for example, as a percentage. Then, the frame generation unit 306d generates an interpolated frame image by changing the weighting of the two adjacent reference frame images in accordance with the relative progress of the performance of the predetermined song.
In addition, since the process which produces | generates an interpolation frame image is a well-known technique, detailed description is abbreviate | omitted here.

  Further, the generation of the reference frame image and the interpolated frame image by the frame generation unit 306d is performed for both the image data of the subject cutout image and the position information such as the alpha map.

The moving image reproduction unit 306e reproduces each of the plurality of frame images generated by the frame generation unit 306d. ("Playing frame images")
That is, the moving image playback unit 306e automatically plays a predetermined song based on the performance information 305b specified based on a predetermined operation of the operation input unit 202 of the user terminal 2 by the user, and the predetermined song of the predetermined song Each of the plurality of frame images is reproduced at the timing. Specifically, the moving image playback unit 306e automatically converts the digital data of the performance information 305b of a predetermined song into analog data by a D / A converter, and automatically plays the predetermined song. Play back two reference frame images that are adjacent to each other (for example, the first beat of each measure, each beat, etc.), and the relative progress of the performance of a predetermined song between the two adjacent reference frame images Depending on the degree, each interpolated frame image corresponding to the degree of progress is reproduced.
Note that the moving image reproduction unit 306e may reproduce a plurality of frame images related to the subject image at a speed specified by the moving image processing unit 306. In this case, the moving image reproducing unit 306e changes the number of frame images to be reproduced within a predetermined unit time by changing the timing at which two adjacent reference frame images are synchronized, thereby changing the motion of the subject image. Variable speed.

  The front / rear replacement setting unit 306f arranges a plurality of characters based on an instruction from the user terminal 2 during the “motion specification” operation, and sets the transparency of the image data on the back of the character arranged in front to 100%. At the same time, a character having the same image data as that of the front character is placed at a position further behind the character placed behind and overlapping the front character.

  The front / rear replacement control unit 306g extracts the change timing of the front / rear relationship set by the front / rear replacement setting unit 306f based on an instruction from the user terminal 2 during the operation of “frame image generation”, and operates the arranged characters. (To be described later), and sequentially generate frame images with interchanged context.

  Next, a moving image generation process using the user terminal 2 and the server 3 will be described with reference to FIGS.

  However, in the operation relating to the moving image generation processing, “subject image generation”, “rear image generation”, “subject image acquisition”, “motion designation”, “frame image generation”, and “frame image reproduction” A detailed description of the basic operation such as the above will be omitted, and the operation related to the first embodiment of the present invention will be described in detail.

  Here, FIG. 4 and FIG. 5 are flowcharts showing an example of the operation related to the moving image generation process. In the following description, various data such as the image data of the subject cut image generated from the image data of the subject existing image and the back image data generated by the back image setting unit 306b are stored in the storage unit 305 of the server 3. Are stored in advance.

In the above-described “motion designation” operation, as shown in FIG. 4, the front-rear replacement setting unit 306 f of the moving image processing unit 306 determines whether to set the front-rear replacement based on an instruction from the user terminal 2. (Step S1).
Here, when the front / rear replacement setting unit 306f determines not to set the front / rear replacement (step S1: No), the process shifts to a normal “motion designation” operation and determines to set the front / rear replacement (step S1: Yes), the front / rear replacement setting unit 306f stores, in the storage unit 305 of the server 3, the first image character (hereinafter referred to as the front character), which is the first image, based on an instruction from the user terminal 2. It is acquired from the image data of the subject clipped image and placed at the front position (step S2: acquisition step).

Next, based on an instruction from the user terminal 2, the front / rear replacement setting unit 306 f stores in the storage unit 305 of the server 3 a character (hereinafter referred to as a rear character) that is the second image and is arranged behind the front character. It is acquired from the stored image data of the subject clipped image and is arranged at a position behind the front character (step S3: acquisition step).
Furthermore, the front / rear replacement setting unit 306f superimposes a subject cutout image in which either the front or the back is the same as the rear character at a position that is in front of the placed front character and overlaps the already placed rear character. An image (hereinafter referred to as a superimposed character) is generated and arranged (step S4: first generation step), and the other side of the superimposed image (the back side of the surface on which the same subject cutout image as the rear character is set) is displayed. The transparency is set to 100% (step S5). Thereafter, the front / rear replacement setting unit 306f shifts to a normal “motion designation” operation.

  Although not specifically described in the flowchart shown in FIG. 4, when the front character, the rear character, and the superimposed character are arranged by the front / rear replacement setting unit 306f, the rear image setting unit 306b generates the back side of each. Assume that the back image data is automatically set.

In this way, after the front character, the rear character, and the superimposed character are arranged by the front / rear replacement setting unit 306f, the second generation step shown in FIG. 5 is performed in the above-described “frame image generation” operation. That is, the front-rear replacement control unit 306g of the moving image processing unit 306 determines whether or not to perform the front-rear relationship replacement between the front character and the rear character based on an instruction from the user terminal 2 (step S10: second generation). Step).
Here, when it is determined that the front / rear replacement control unit 306g does not perform front / rear replacement (step S10: No), the process proceeds to a normal “frame image generation” operation, and when it is determined to perform front / rear replacement (step) S10: Yes), the front-rear replacement control unit 306g further determines whether or not to move the front character based on an instruction from the user terminal 2 (step S11: second generation step).

  Here, when it is determined that the front / rear replacement control unit 306g moves the front character (step S11: Yes), the front / rear replacement control unit 306g causes the superimposed character to rotate halfway and rotate or move the front character. Next, the frame image is sequentially generated while the image area is deformed (step S12: second generation step), and thereafter, the operation proceeds to a normal “frame image generation” operation.

  On the other hand, when the front / rear replacement control unit 306g determines that the front character is not moved (step S11: No), the front / rear replacement control unit 306g sequentially converts the frame images while deforming the image region so as to rotate the superimposed character halfway. (Step S13: second generation step) After that, the operation proceeds to a normal “frame image generation” operation.

  The content described so far with reference to FIGS. 4 and 5 will be described in more detail with reference to FIGS. However, in FIG.6 and FIG.8, the lower side of a paper surface has shown the front surface, and the upper surface of the paper surface has shown the back.

In FIG. 6, the superimposed character CH1 is set with image data (solid line portion) of the subject clipped image previously selected on the front surface, and on the back surface is the same shape as the image data and image data with 100% transparency. (The two-dot chain line portion) is set by the front / rear replacement setting unit 306f.
The front character CH2 is set with image data (solid line portion) of the subject clipped image previously selected on the front side, and on the back side, the back side of the subject (front character CH2) generated by the back image setting unit 306b is simulated. The rear image data (right oblique line portion) expressed in the figure is set by the front / rear replacement setting unit 306f.
Finally, the rear character CH3 has the same subject cut-out image data (solid line portion) as the superimposed character CH1 on the front, and the subject (rear character CH3) generated by the rear image setting unit 306b on the back. Is set by the front / rear replacement setting unit 306f.

In FIG. 6A, when the user views each character from the direction (front) indicated by DR1, the superimposed character CH1 overlaps the front of the rear character CH3, and the image data (solid line portion) of the superimposed character CH1. ) Is visible, the rear character CH3 is obscured by the superimposed character CH1 and cannot be seen.
For example, as shown in FIG. 7A, the reproduced moving image is visually recognized in a state where the superimposed character CH1 overlaps the front character CH2. (First frame image)

  Here, if frame images that cause the superimposed character CH1 to rotate half-turn are sequentially generated by the front-rear replacement control unit 306g, the positional relationship in FIG. 6A is in the order of FIG. 6B and FIG. 6C. It will be deformed sequentially.

For this reason, in FIG. 6C, when the user views each character from the direction indicated by DR2 (front), the superimposed character CH1 overlaps the front of the rear character CH3, but the superimposed character CH1 does not The image data having the same shape as the image data and the transparency of 100% (the two-dot chain line portion) can be seen. In other words, the superimposed character CH1 can be seen through and the rear character CH3 can be seen.
For example, in the reproduced video, as shown in FIG. 7B, the rear character CH3 can be seen in a state where the superimposed character CH1 is rotated by a quarter, and as shown in FIG. When the superimposed character CH1 is rotated by a quarter rotation and the half-rotated character CH1 is rotated, the superimposed character CH1 cannot be seen, and the rear character CH3 is seen behind the front character CH2. (Second frame image)
At this time, since the rear character CH3 has the same subject cut-out image data (solid line portion) as the superimposed character CH1, the superimposed character CH1 wraps behind the front character CH2 from the user side. It will be recognized as an action.

In FIG. 8A (same as FIG. 6C), when the user views each character from the direction indicated by DR3 (front), the superimposed character CH1 overlaps in front of the rear character CH3. The image data (two-dot chain line portion) having the same shape as the image data and 100% transparency can be seen on the front of the superimposed character CH1, in other words, the superimposed character CH1 can be seen through and the rear character CH3 can be seen.
For example, in the moving image to be reproduced, as shown in FIG. 9A, it appears that the rear character CH3 overlaps behind the front character CH2. (Second frame image)

  Similarly, if the frame images that further rotate the superimposed character CH1 further half-turn are sequentially generated by the front-rear replacement control unit 306g, the positional relationship in FIG. 8A (same as FIG. 6C) is obtained. 8 (b) and FIG. 8 (c) are sequentially deformed.

For this reason, in FIG. 8C, when each character is viewed from the direction (front) shown in DR4, as described above, the superimposed character CH1 overlaps the front of the rear character CH3, and the image of the superimposed character CH1. Since the data (solid line portion) is visible, the rear character CH3 is hidden by the superimposed character CH1 and cannot be seen.
For example, in the reproduced video, as shown in FIG. 9B, when the superimposed character CH1 is rotated by 1/4, the rear character CH3 is still visible, but as shown in FIG. 9C. Further, when the superimposed character CH1 is further rotated by a quarter turn, the superimposed character CH1 becomes visible and the rear character CH3 is obscured, so that the superimposed character CH1 overlaps before the front character CH2. It will be visually recognized in a state. (First frame image)
At this time, since the image data (solid line portion) of the same subject cutout image as that of the rear character CH3 is set for the superimposed character CH1, the operation in which the superimposed character CH1 is replaced before the front character CH2 from the user side. Will be recognized as.

  As described above, the front character with the image data set on the front and the back image data set on the back is placed, and the back character with the other image data set on the front and the back image data set on the back is placed behind it. In the front of the character and at the position overlapping with the back character, arrange the superimposed character with the same image data as the back character on the front and the same shape as the image data and 100% transparency image data on the back, When the context is switched, it is possible to easily replace the context of the character by sequentially generating and reproducing a frame image for half-rotating the superimposed character.

However, for example, in the state of FIG. 6 (a), when the superimposed character is simply switched to a frame image in a half-rotation state or the superimposed character itself is deleted, the superimposed character is immediately behind the front character, or , It is recognized as an operation that replaces the previous one, and it may be a slightly unnatural operation.
In this case, by sequentially generating and playing back a frame image including a process in which the superimposed character is half-rotated in time series, the superimposed character is simply half-rotated or compared with the case where the superimposed character itself is deleted. From the user side, the superimposed character appears to wrap around behind or in front of the front character while rotating in time series, so that it can be recognized as a more natural motion.

(Second Embodiment)
Moreover, the moving image generation system of 2nd Embodiment of this invention is demonstrated based on FIG.5, FIG10 and FIG.11. However, the description of the arrangement of the characters by the front / rear replacement setting unit 306f and other parts similar to those of the first embodiment of the present invention will be omitted.

  When the front / rear replacement control unit 306g determines that the front character is to be moved based on an instruction from the user terminal 2 (step S11: Yes), the front / rear replacement control unit 306g rotates the superimposed character halfway and The frame image is sequentially generated while the image region is deformed so as to be rotated or moved (step S12: second generation step), and then the operation proceeds to a normal “frame image generation” operation.

In FIG. 10A (same as FIG. 6C), when each character is viewed from the direction indicated by DR5 (front), the superimposed character CH1 overlaps the front of the rear character CH3, but the superimposed character. In front of CH1, image data (two-dot chain line portion) having the same shape as the image data and 100% transparency can be seen. In other words, the superimposed character CH1 can be seen through and the rear character CH3 can be seen.
For example, in the moving image to be reproduced, as shown in FIG. 11A, it appears that the rear character CH3 overlaps behind the front character CH2. (Second frame image)

  Similarly, if the frame images that sequentially rotate the superimposed character CH1 and further rotate the forward character CH2 by the forward / reverse switching control unit 306g are sequentially generated, the positional relationship in FIG. 10 (b) and FIG. 10 (c) are sequentially deformed.

For this reason, in FIG. 10C, when the user views each character from the direction indicated by DR6 (front), as described above, the superimposed character CH1 overlaps the front of the rear character CH3, and the superimposed character Since the image data (solid line part) of CH1 is visible, the rear character CH3 is covered with the superimposed character CH1 and cannot be seen.
For example, in the reproduced video, as shown in FIG. 11B, when the superimposed character CH1 is rotated 1/4 and the front character CH2 is rotated halfway, the rear character CH3 is still visible. As shown in FIG. 11 (c), when the superimposed character CH1 is further rotated by a quarter turn and the front character CH2 is further rotated half a turn, the superimposed character CH1 becomes visible, and the rear character CH3. Therefore, it is visually recognized in a state where the superimposed character CH1 overlaps the front character CH2. (First frame image)
At this time, since the image data (solid line portion) of the same subject cutout image as that of the rear character CH3 is set for the superimposed character CH1, from the user side, the superimposed character CH1 is placed in front of the front character CH2 that makes one rotation. It will be recognized as a wraparound action.

  As described above, the front character with the image data set on the front and the back image data set on the back is placed, and the back character with the other image data set on the front and the back image data set on the back is placed behind it. In the front of the character and at the position overlapping with the back character, arrange the superimposed character with the same image data as the back character on the front and the same shape as the image data and 100% transparency image data on the back, At the time of changing the context, it is possible to easily change the context of the character by sequentially generating and playing back the frame images that rotate the superimposed character in half and the front character.

However, for example, in the state shown in FIG. 10A, when the superimposed character is simply rotated halfway, or when the frame is switched to a frame image in which the superimposed character itself is deleted and the forward character is rotated, the superimposed character is instantaneously moved forward. It is recognized that the movement is backward or backward, and the movement is slightly unnatural.
In this case, the superimposed character is simply half-turned or superimposed by sequentially generating and playing back a frame image including the process of the superimposed character half-rotating in time series and the process of rotating the forward character. Compared with the case where the character itself is deleted, the rotation of the forward character is added, so that it is possible for the user to recognize the change of the context between the characters as a natural and more complex motion.

(Modification 1)
Modification 1 of the first and second embodiments of the present invention will be described with reference to FIG.
As shown in FIG. 7, when the superimposed character is recognized as being stationary or moving behind the moving front character by half-rotating the superimposed character, it is The character is rotated halfway and the back is turned. When the superimposed character is rotated halfway, a frame image is generated while the image area is deformed so that the rear character is further rotated halfway and turned frontward.

  In other words, a front character with image data set on the front and a back image data set on the back is placed, behind that, another image data set on the front and a back character set with back image data on the back is placed, and the front character is In addition, a superimposed character with the same image data as the rear character and the same shape as the image data and set with 100% transparency image data is placed on the front side at the position overlapping the rear character, and the context is changed. Occasionally, the back character is rotated halfway in advance and the back is directed, and when the superimposing character is rotated halfway, the back character is further rotated halfway and the frame images facing the front are sequentially generated and played back. The character is stationary while rotating, or wraps around behind the moving forward character It will be recognized as work.

  As described above, in the first modification, compared to the case where the superimposed character is simply rotated halfway, the movement of the backward character is added. It can also be recognized as a more complex action.

(Modification 2)
Modification 2 of the first embodiment and the second embodiment of the present invention will be described with reference to FIG.
As shown in FIG. 9, when the superimposed character is recognized as being stationary or moving in front of the moving front character by half-rotating the superimposed character, the rear character is controlled by the front / rear replacement control unit 306g. The frame image is generated while the image area is deformed so that the superimposed character turns halfway and turns to the front.

  In other words, a front character with image data set on the front and a back image data set on the back is placed, behind that, another image data set on the front and a back character set with back image data on the back is placed, and the front character is In addition, a superimposed character with the same image data as the rear character and the same shape as the image data and set with 100% transparency image data is placed on the front side at the position overlapping the rear character, and the context is changed. Occasionally, the superimposed character rotates by rotating the backward character half-turned in advance with the superimposed character facing in advance, and then sequentially generating and playing frame images that face the front by half-rotating the superimposed character. While still, or recognized as moving around the front of the moving character Is will be.

  As described above, in the second modification, the movement of the backward character is added as compared with the case where the superimposed character is simply rotated halfway. It can also be recognized as a more complex action.

  In the description of the first embodiment and the second embodiment, each character (front character, rear character, and superimposed character) is rotated clockwise about the vertical axis with respect to the plan view. Of course, the rotation axis and the rotation direction are not limited, and any rotation axis can be used as long as the rotation axis can be rotated in any rotation direction.

  In the description of FIG. 10 and the like, the rotation operation is illustrated as the operation of the forward character, but of course, it is not limited to the rotation operation, and any operation such as movement or dance in the frame image is performed. Anything you want to do.

  In the description of the first embodiment and the second embodiment, it is described that the transparency of the other surface of the superimposed character is set to 100% in advance. The same effect can be achieved by processing in an invisible state.

  As described in “Generating a subject image”, the front character and the rear character in the first embodiment and the second embodiment are images in which a region including a subject is cut out from an image including a background and a subject. Needless to say.

  In the description of the first embodiment and the second embodiment, it is explained that the character context is replaced by the half rotation of the superimposed character. However, by removing the superimposed character, the character context is removed. May be replaced.

In addition, in the above-described embodiment, the operations as the acquisition step, the first generation step, and the second generation step are performed under the control of the central control unit 301, the image acquisition unit 306a, the back image generation unit 306b, The video processing unit 306 includes a control point setting unit 306c, a frame generation unit 306d, a video playback unit 306e, a front / rear replacement setting unit 306f, and a front / rear replacement control unit 306g. However, the present invention is not limited to this, and may be realized as an image generation apparatus including acquisition means, first generation means, and second generation means.
That is, the front character that is the first image is acquired from the image data of the cut-out subject image stored in the storage unit 305 of the server 3 and is placed at the front position, and is placed behind the front character that is the second image. An acquisition means for acquiring the rear character to be acquired from the image data of the subject cutout image stored in the storage unit 305 of the server 3 and disposing the character at a position behind the first front character; In addition, a first generation means for generating and arranging a superimposed character that is a cut-out image of the same subject as either the front character or the rear character at a position that overlaps the rear character that has already been arranged, and a half rotation of the superimposed character And transform the image area to rotate or move the character in front. Sequentially generating a frame image without leaving, or may be configured to function as a second generating means for sequentially generating a frame image while the deformation of the image area so as to half turn superimposed characters.

In addition, in the above-described embodiment, the operations as the acquisition step, the first generation step, and the second generation step are performed under the control of the central control unit 301, the image acquisition unit 306a, the back image generation unit 306b, The video processing unit 306 includes a control point setting unit 306c, a frame generation unit 306d, a video playback unit 306e, a front / rear replacement setting unit 306f, and a front / rear replacement control unit 306g. However, the present invention is not limited to this, and a configuration realized by executing a predetermined program or the like by the central control unit 301 may be adopted.
That is, a program including an acquisition process routine, a first generation process routine, and a second generation process routine is stored in a program memory (not shown) that stores the program.
Then, the CPU of the central control unit 301 acquires the front character as the first image from the image data of the subject cutout image stored in the storage unit 305 of the server 3 by the acquisition processing routine, and arranges it at the front position. Acquisition means for acquiring a rear character to be arranged behind the front character, which is the second image, from the image data of the subject cutout image stored in the storage unit 305 of the server 3 and arranging it at a position behind the first front character. You may make it function as.
Further, the CPU of the central control unit 301 is further forward of the front character placed by the first generation processing routine and at a position overlapping with the rear character already placed, either one of the front and back faces is the rear character. You may make it function as the 1st production | generation means which produces | generates and arrange | positions the superimposition character which is the same subject clipping image.
Further, the CPU of the central control unit 301 sequentially generates frame images while deforming the image area so that the superimposed character is rotated halfway and rotated or moved with respect to the front character by the second generation processing routine. Or you may make it function as a 2nd production | generation means which produces | generates a frame image sequentially, deform | transforming an image area | region so that a superimposition character may be rotated in half.

Although several embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and equivalents thereof.
The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
An acquisition step of acquiring the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. A first generation step to:
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, A second generation step of generating a moving image including a second frame image obtained by superimposing the superimposed image on the back side image in an invisible state on the near side of the near side image among the first and second images. An image generation method comprising:
<Claim 2>
The first generation step includes
As the superimposed image, generate an image that transmits the other side different from the one side among the front side and the back side,
The second generation step includes
2. The image generation method according to claim 1, wherein a moving image including a frame image in which the superimposed image is superimposed on the back side image with the other surface side in front is generated as the second frame image. .
<Claim 3>
The second generation step includes
The image generation method according to claim 1, wherein a moving image in which the first frame image and the second frame image are switched is generated.
<Claim 4>
The second generation step includes
4. The image generation method according to claim 3, wherein a moving image is generated in which the first frame image and the second frame image are switched when the superimposed image is half-rotated.
<Claim 5>
The second generation step includes
A moving image is generated in which the first frame image and the second frame image are switched when the back-side image and the superimposed image that are substantially synchronized with each other in a substantially synchronized manner are rotated halfway. The image generation method according to claim 3.
<Claim 6>
The second generation step includes
6. The moving image in which the first frame image and the second frame image are switched when the near-side image is half-rotated is generated. 6. Image generation method.
<Claim 7>
The first and second images are
The image generation method according to claim 1, wherein an image including a subject is cut out from an image including a background and the subject.
<Claim 8>
Obtaining means for obtaining the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. First generating means for
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, Of the first and second images, second generation means for generating a moving image including a second frame image in which the superimposed image is superimposed on the back side image in an invisible state on the near side of the near side image. An image generation apparatus comprising:
<Claim 9>
Computer
Obtaining means for obtaining the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. First generating means for
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, Of the first and second images, second generation means for generating a moving image including a second frame image in which the superimposed image is superimposed on the back side image in an invisible state on the near side of the near side image. A program characterized by functioning as

DESCRIPTION OF SYMBOLS 100 Movie generation system 1 Imaging device 2 User terminal 3 Server 201, 301 Central control part 202 Operation input part 203, 302 Display part 204 Sound output part 205 Recording medium control part 206, 303 Communication control part 304 Subject clipping part 305 Storage part 306 Movie processing unit 306f Front / rear replacement installation unit 306g Front / rear replacement control unit

Claims (8)

An acquisition step of acquiring the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. A first generation step to:
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, A second generation step of generating a moving image including a second frame image obtained by superimposing the superimposed image on the back side image in an invisible state on the near side of the near side image among the first and second images. viewing including the door,
The first generation step includes
As the superimposed image, generate an image that transmits the other side different from the one side among the front side and the back side,
The second generation step includes
An image generation method comprising: generating a moving image including a frame image obtained by superimposing the superimposed image on the back side image with the other surface side on the near side as the second frame image . The second generation step includes
The image generation method according to claim 1, wherein a moving image in which the first frame image and the second frame image are switched is generated. An acquisition step of acquiring the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. A first generation step to:
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, A second generation step of generating a moving image including a second frame image obtained by superimposing the superimposed image on the back side image in an invisible state on the near side of the near side image among the first and second images. Including
The second generation step includes
A method for generating an image, comprising: generating a moving image in which the first frame image and the second frame image are switched when the superimposed image is half-rotated. An acquisition step of acquiring the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. A first generation step to:
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, A second generation step of generating a moving image including a second frame image obtained by superimposing the superimposed image on the back side image in an invisible state on the near side of the near side image among the first and second images. Including
The second generation step includes
A moving image is generated in which the first frame image and the second frame image are switched when the back-side image and the superimposed image that are substantially synchronized with each other in a substantially synchronized manner are rotated halfway. Image generation method. An acquisition step of acquiring the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. A first generation step to:
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, A second generation step of generating a moving image including a second frame image obtained by superimposing the superimposed image on the back side image in an invisible state on the near side of the near side image among the first and second images. Including
The second generation step includes
A method for generating an image, comprising: generating a moving image in which the first frame image and the second frame image are switched when the near-side image is rotated halfway. The first and second images are
The image generation method according to claim 1, wherein an image including a subject is cut out from an image including a background and the subject. Obtaining means for obtaining the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. First generating means for
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, Of the first and second images, second generation means for generating a moving image including a second frame image in which the superimposed image is superimposed on the back side image in an invisible state on the near side of the near side image. And
The first generation means includes
As the superimposed image, generate an image that transmits the other side different from the one side among the front side and the back side,
The second generation means includes
An image generating apparatus that generates a moving image including a frame image obtained by superimposing the superimposed image on the back side image with the other surface side on the near side as the second frame image . Computer
Obtaining means for obtaining the first and second images arranged by shifting the front-rear position;
Of the first and second images, a superimposed image is generated that is superimposed on the back side image located on the back side, and one of the front side and the back side is substantially equal to one side of the back side image. First generating means for
Of the first and second images, a first frame image in which the superimposed image is superimposed on the back side image with the one surface side on the near side of the near side image located on the near side, Of the first and second images, second generation means for generating a moving image including a second frame image in which the superimposed image is superimposed on the back side image in an invisible state on the near side of the near side image. Function as
The first generation means includes
As the superimposed image, generate an image that transmits the other side different from the one side among the front side and the back side,
The second generation means includes
A program for generating a moving image including a frame image obtained by superimposing the superimposed image on the back side image with the other surface side on the near side as the second frame image .

Images