JP5252068B2 - Composite image output apparatus and composite image output processing program - Google Patents

Description

  The present invention relates to a composite image output apparatus and a composite image output processing program for generating and outputting a composite image obtained by replacing a person image captured with a moving image with another character image.

  In recent years, with the advancement of special photography technology, it has become possible to create scenes that do not exist on the screen by miniature photography, composition of a person and a background image, composition of a computer graphic and a photographed image, and the like. Recently, an arbitrary character image corresponding to the motion of a model (target person) captured in advance as a moving image can be displayed as a moving image by the motion capture technology and the 3D image processing technology. In addition, image composition for adding comments to moving images has been generally performed using a net service.

  Conventionally, in the same applicant, a person image to be replaced is extracted from the photographed image according to distance information to each image portion included in the photographed image photographed by the digital camera, and the extracted target person An image output apparatus is conceivable in which an image is replaced with a mimic image made up of another character image corresponding to the pose (posture) and displayed after being synthesized (see, for example, Patent Document 1).

  In addition, the same applicant receives pose information indicating the posture of a person image to be replaced extracted from a captured image sent from a terminal device on a communication network, and a character image having a posture corresponding to the pose information. An image distribution server that reads out from a database and provides it to the terminal device is also considered (see, for example, Patent Document 2).

  According to such a conventional image output device or image distribution server, it is possible to replace an arbitrary target person image in a photographed image with another character image, and display or print it. You can enjoy using images by replacing them with other character images, or you can use the captured images more effectively in various situations by impersonating people who should not be seen by others and replacing them with other character images. .

Japanese Patent No. 3901015 Japanese Patent No. 3918632

  In the conventional technique for replacing a photographed person image with another character image, the target person image is replaced and synthesized as it is in either case of a still image or a moving image. As for special effects and animations, it is difficult to generate a composite image that forms a series of scenes, such as transforming the target person with the will of the characters.

  In recent years, especially in the middle of live-action and animation images, scenes that transform other characters by the magic of a given character have been frequently used. To generate such a transformation image, A great deal of labor and expertise was required, such as preparation of costumes, props, makeup, and editing of filmed images.

  The present invention has been made in view of such problems, and it is possible to easily generate a transformed composite image obtained by transforming another person in accordance with the will of a predetermined person with respect to a person image that has been shot. It is an object to provide a composite image output apparatus and a composite image output processing program.

The composite image output apparatus according to claim 1, a transformation character data storage unit that stores a pre-set image data of a transformation character in association with a person image registered as a user, and a series of captured image data obtained by moving image shooting A captured image acquisition means for acquiring image information, and a registered person gesture for detecting whether or not a predetermined transformation by the user-registered person image exists in the series of captured image data acquired by the captured image acquisition means. When a predetermined transformation by the user-registered person image is detected by the detection unit and the registered person gesture detection unit, the detected person image is stored in association with the transformation character data storage unit. Based on the image data of the transformed character, the face part is left as it is in the image data of the transformed character with the same posture as the person image. A synthetic image generating means for generating e was synthesized image data, and the synthesized image output means for outputting the synthesized image data generated by the composite image generation means, a personal image which is the user registered a preset transformation gesture data The registered person gesture detection means stores whether or not there is a corresponding transformation gesture based on the person image stored in the transformation gesture data storage means. It is characterized by detecting .

  According to the present invention, when a user-registered person image is shot as a moving image, a transformed composite image in which the person image being shot is transformed into a transformation character can be easily generated according to the will of the person. A composite image output apparatus and a composite image output processing program can be provided.

It is a conceptual diagram which shows the imaging | photography state of a moving image using the digital camera 10 which concerns on 1st Embodiment of the synthesized image output device of this invention, and the production | generation state of a synthesized image. The block diagram which shows the structure of the electronic circuit of the said digital camera 10 (communication terminal A). The figure which shows the content of the transformation data previously memorize | stored in the transformation data memory 14 of 1st Embodiment of the said digital camera. The figure which shows the content of the special effect data previously stored in the special effect data memory 23A of 1st Embodiment of the said digital camera. The figure which shows the content of the defense effect data previously memorize | stored in the defense effect data memory 23B of 1st Embodiment of the said digital camera. The figure which shows the content of the transformation character data memorize | stored in the transformation character data memory 24A of 1st Embodiment of the said digital camera. The block diagram which shows the structure of the electronic circuit of the server apparatus 30 which concerns on the synthesized image output apparatus of this invention. 3 is a flowchart showing an overall process of composite image output processing by the composite image output apparatus of the first embodiment. The flowchart which shows the target image extraction process which extracts the target image of the person (P, J, K) which exists in the picked-up image (G) with the synthetic | combination image output process of the said 1st Embodiment. The flowchart which shows the role transformation mark process which adds a transformation mark (24Mw) to the person image (P) by which the role registration was carried out with the synthetic | combination image output process of the said 1st Embodiment. Motion transformation mark processing for adding a transformation mark (14Mu) to a person image (J, K) in which transformation motion (14P) is detected in accordance with the composite image output processing of the first embodiment and outputting corresponding sound (14Su). The flowchart which shows. A special effect mark (23Mu) is added to a person image (P, J, K) in which a special effect motion (23P) is detected in accordance with the composite image output process of the first embodiment, and a corresponding sound (23Su) is output. The flowchart which shows a special effect mark process. The person image (P) (J, K) to which the transformation mark (24Mw) (14Mu) is added in accordance with the synthesized image output process of the first embodiment is replaced with the transformation image (24Gw) (14Gu) and synthesized. The flowchart which shows the transformation mark synthetic | combination process which sets the output of (24Sw) (14Su). A special effect image 23Gu is additionally synthesized at a predetermined position of the person image (P, J, K) to which the special effect mark (23Mu) is added in accordance with the synthesized image output process of the first embodiment, and the corresponding sound (23Su) is also added. The flowchart which shows the special effect mark synthetic | combination process which sets an output. FIG. 6 is a diagram illustrating image processing states based on a moving image captured image (G) associated with the composite image output processing of the first embodiment in order of typical frame advance (part 1), where FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. FIG. 4 is a diagram illustrating an image processing state based on a moving image captured image (G) associated with a composite image output process of the first embodiment in order of representative frame advance (part 2), in which the captured image is illustrated in FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. FIG. 6 is a diagram illustrating an image processing state based on a moving image captured image (G) associated with the composite image output process of the first embodiment in order of representative frame advance (part 3), in which the captured image is illustrated in FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. FIG. 6 is a diagram illustrating an image processing state based on a moving image photographed image (G) associated with the composite image output processing of the first embodiment in order as a representative frame advance (part 4). FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. FIG. 6 is a diagram illustrating an image processing state based on a moving image photographed image (G) associated with the composite image output process of the first embodiment in order of representative frame advance (No. 5). FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. FIG. 6 is a diagram illustrating an image processing state based on a moving image captured image (G) associated with the composite image output process of the first embodiment in order of representative frame advance (No. 6). FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. FIG. 6 is a diagram illustrating image processing states based on a moving image captured image (G) associated with the composite image output processing of the first embodiment in order of representative frame advance (part 7), in which FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. FIG. 8 is a diagram illustrating image processing states based on a moving image captured image (G) associated with the composite image output process of the first embodiment in order of representative frame advance (No. 8). FIG. The figure which shows G, The figure (B) is a figure which shows the picked-up image Gm with a mark, The figure (C) is a figure which shows the synthesized image GG. The conceptual diagram which shows the imaging | photography state of a moving image using the digital camera 10 which concerns on 2nd Embodiment of the synthesized image output apparatus of this invention, and the production | generation state of a synthesized image. The figure which shows the content of the transformation character data memorize | stored in the transformation character data memory 24A of 2nd Embodiment of the said digital camera. The figure which shows the content of the transformation data previously memorize | stored in the transformation data memory 14 of 2nd Embodiment of the said digital camera. The figure which shows the content of the special effect data previously memorize | stored in the special effect data memory 23A of 2nd Embodiment of the said digital camera. The figure which shows the content of the defense effect data previously memorize | stored in the defense effect data memory 23B of 2nd Embodiment of the said digital camera. A motion transformation mark that adds a transformation mark (24M ′) to a person image (J, K) in which transformation motion (14P) is detected in accordance with the composite image output process of the second embodiment and outputs a corresponding voice (24S ′). The flowchart which shows a process. A special / defense effect mark (23Mu / 23'Mu) is added to the person image (P, J, K) in which the special / defense effect motion (23P / 23'P) is detected in accordance with the composite image output process of the second embodiment. The flowchart which shows the special effect mark process which adds and outputs corresponding audio | voice (23Su / 23'Su). It is a figure which shows the image processing state based on the moving image picked-up image (G) accompanying the synthesized image output process of 2nd Embodiment as typical frame advance (the 1) sequentially, The figure (A) is the picked-up image G. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG. It is a figure which shows the image processing state based on the moving image picked-up image (G) accompanying the synthetic | combination image output process of 2nd Embodiment as the typical frame advance (the 2) sequentially, The figure (A) is the picked-up image G. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG. It is a figure which shows the image processing state based on the moving image picked-up image (G) accompanying the synthetic | combination image output process of 2nd Embodiment as typical frame advance (the 3) sequentially, The figure (A) is the picked-up image G. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG. FIG. 10 is a diagram showing the image processing state based on the moving image captured image (G) associated with the composite image output processing of the second embodiment as a representative frame advance (part 4) sequentially. FIG. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG. FIG. 10 is a diagram showing the image processing state based on the moving image captured image (G) associated with the composite image output processing of the second embodiment as a representative frame advance (No. 5) sequentially. FIG. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG. FIG. 10 is a diagram illustrating an image processing state based on a moving image captured image (G) associated with a composite image output process according to the second embodiment in order of typical frame advance (No. 6). FIG. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG. It is a figure which shows the image processing state based on the moving image picked-up image (G) accompanying the composite image output process of 2nd Embodiment as typical frame advance (the 7) sequentially, The figure (A) is the picked-up image G. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG. FIG. 10 is a diagram showing the image processing state based on the moving image captured image (G) associated with the composite image output processing of the second embodiment in order as a representative frame advance (No. 8). FIG. FIG. 8B is a diagram showing a marked photographed image Gm, and FIG. 10C is a diagram showing a composite image GG.

  Embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
FIG. 1 is a conceptual diagram showing a moving image shooting state and a composite image generation state using the digital camera 10 according to the first embodiment of the composite image output apparatus of the present invention.

  The digital camera 10 includes an LCD display unit 11, an input unit 12, and an audio output unit (speaker) 13 on the back surface thereof, and a moving image (captured image) G photographed according to the operation of the input unit 12 is real-time. It is displayed on the display unit 11.

  The digital camera 10 has, as main functions, a function of extracting a person image (P, J, K) included in the moving image G being shot, and a person image (P) in which the extracted person image is registered in advance. A function for recognizing whether a face image is recognized or not, and when the extracted person image is a pre-registered person image (P), it is set in advance in association with the registered person image (P) A function of adding a mark (posture transformation mark 24Mw) of a role image (24Gw), a function of replacing the person image (P) to which the role transformation mark 24Mw is added with the preset role image (24Gw), and the like extracted A function for detecting a preset gesture (motion) of a person image, and a mark (transformation) of a preset transform image (14Gu2) for a person image (K) for which a preset gesture has been detected. 14Mu2), a function for extracting a face image (Kf) from a person image (K) to which the transformation mark 14Mu is added, and a face image (Kf ′) as it is (F), and other parts than the face are the same. A function for replacing with a pre-set transformation image (14Gu2), a function for changing a person image (J) of a partner when a pre-set gesture of a pre-registered person image (P) is detected into a transformation image (Gk) A motion capture processing function for capturing the movement of the extracted person image (P, J, K) and converting it into a 3D image of the role image (24Gw) or a transformation image (14Gu), a 3D image processing function Etc.

  The face image recognition process, the motion capture process, and the 3D image process are well-known image processing techniques, and thus detailed description thereof is omitted.

  The digital camera 10 having such main functions is used by a family, for example, the boy P as a god of death, a sister K as a girl warrior B (main character), and an sister J as a girl warrior A (side role). When the mother plays and plays the series of scenes, the face image of the boy P is registered in advance in the digital camera 10 in association with the reaper image (the role image 24Gw).

  Then, boy P approached by waving a stick B like a sickle's sickle, and sister J approached in a setting that was transformed into a masked girl by the motion of god of death. The mother shoots a moving image with the digital camera 10 of a series of scenes in which the pen D is shaken to perform a motion for self-transformation (gesture) and transform into the sailor B.

  As shown in FIG. 1A, the moving image G being shot is displayed on the display unit 11 in real time and analyzed by the various functions of the digital camera 10. The character transformation mark 24Mw indicating the corresponding reaper image (the role image 24Gw) is additionally displayed, and the corresponding image of the beautiful girl warrior B from the time point when the preset gesture (motion) is detected in the personal image of the sister K. A body movement transformation mark 14Mu2 indicating (transformation image 14Gu2) is additionally displayed. In addition, a pre-set gesture (motion) has already been detected in the person image of the sister J, and a body movement transformation mark 14Mu1 indicating the beautiful girl warrior A (transformation image 14Gu1) is additionally displayed. Are additionally displayed as a mask mark Mk indicating a mask image (Gk) transformed into a masked girl in accordance with a preset gesture (motion) of the boy P registered in advance.

  As a result, the mother being photographed can see that the images of the characters of boy P, sister K, and sister J have been transformed according to their roles, and her sister J has been further transformed into a masquerade warrior A from the transformation process. It turns out that the transformation process is done.

  At this time, the voice data “cute” stored in advance in association with the sailor B (transformation image 14Gu2), the sailor A with mask (transformation image 14Gu1), and the reaper image (role image 24Gw), respectively. ♪ Cute ♪ ”,“ Cook Cook ”,“ Zaku, Zaku ”are read out and output from the audio output unit 13, so even if you are a sister K, boy P, or sister J who is playing each role, Can be easily recognized as being transformed.

  A moving image in which the role transformation mark 24Mw and the body movement transformation mark 14Mu are added to the photographed moving image (photographed image) G is referred to as a marked photographed image Gm.

  Thereafter, when the shooting of the moving image G of the series of scenes is finished and the captured image Gm with the mark is generated, the captured image Gm with the mark is analyzed by the various functions, and is shown in FIG. As shown in the figure, the reaper role image 24Gw1 corresponding to the person image of the boy P to which the role change mark 24Mw is added, and the original face image corresponding to the person image of the sister K to which the gesture change mark 14Mu2 is added. A transformation image 14Gu1 of a beautiful girl warrior B with a mask image Gk corresponding to a person image of an older sister J to which the transformation mark 14Mu1 and the mask mark Mk are added, A synthesized moving image GG synthesized with the preset background image BG is generated. In addition, since the audio data corresponding to each transformation mark is stored in association with each other, a composite image with an appropriate sound effect can be easily created.

  As a result, the user family can easily create a special-effects-like synthesized moving image GG. The generated synthesized moving image GG is displayed on the display unit 11 for viewing or output to an external monitor for viewing. By doing so, you can get unprecedented enjoyment. At this time, only a person of a person image serving as a role whose face image is registered in advance can detect and perform a pre-set gesture (motion), thereby creating a transformed image in which another person image is transformed. . Moreover, since a transformation image with the original face attached and the costume changed is generated, it is possible to easily create a transformation image with a facial expression as it is.

  FIG. 2 is a block diagram showing a configuration of an electronic circuit of the digital camera 10 (communication terminal A).

  The digital camera 10 includes a CPU 16 that is a computer.

  The CPU 16 is a system program stored in the memory 17 in advance, or a camera control program read from the external recording medium 18 such as a memory card into the memory 17 via a recording medium reading unit 19 such as a card slot, or a communication network ( N) The operation of each part of the circuit is controlled according to the camera control program read into the memory 17 from the server device 30 (in this case, the program server) via the power transmission control unit 20.

  The CPU 16 is connected to the LCD display unit 11, the input unit 12, the audio output unit 13, the memory 17, the recording medium reading unit 19, and the power transmission control unit 20, as well as a solid-state imaging device (CCD), an imaging optical system, and the like. An image photographing unit 21 having a distance sensor and an illuminance sensor is connected.

  The program memory 22 of the memory 17 includes a system program that controls the entire operation of the digital camera 10, a camera control program that controls the photographing operation, a server device 30 on the communication network N, and an external communication terminal B (PC: personal computer). ) A communication control program for controlling the communication operation with 40 is stored, and based on the captured image G captured by the image capturing unit 21, the above-described marked captured image Gm and composite image GG are generated and output. An image processing program that manages the various functions for storing the image is stored in advance.

  The various programs stored in the program memory 22 are a key input signal from the input unit 12, a shooting input signal from the image shooting unit 21, and an external device (30, 40) via the transmission control unit 20. It is activated in response to the received input signal.

  The memory 17 includes a transformation data memory 14, a special effect data memory 23A, a defense effect data memory 23B, a transformation role data memory 24A, a role-specific transformation status memory 24B, a photographic image data memory 25, and a marked photographic image data memory. 26, a composite image data memory 27, and other work data memory.

  FIG. 3 is a diagram showing the contents of the transformation data stored in advance in the transformation data memory 14 of the first embodiment of the digital camera 10.

  The transformation data memory 14 includes a storage area for transformation motion model data 14P, transformation image data 14Gu, and transformation mark data 14Mu. The storage area for the transformation image data 14Gu includes a plurality of types of transformation image data (transformation characters), The voice data and face synthesis position data are stored. For example, transformation image data 14Gu1 of Sailor A, its voice data “Lemon ♪ Lemon ♪” 14Su1, and face synthesis position data 14F1, transformation image data 14Gu2 of Sailor B, its voice data “Cute ♪ Cute ♪” 14Su2, And the face synthesis position data 14F2.

  Further, in the storage area of the transformation mark data 14Mu, transformation mark data 14Mu1 and 14Mu2 that respectively express the plurality of types of transformation image data 14Gu1 and 14Gu2 in a simplified manner are stored.

  Then, in the storage area of the transformation motion model data 14P, transformation motion model data 14P1a to 14P1c consisting of a series of motions (gestures) for transformation into the plurality of types of transformation image data 14Gu1 and 14Gu2, respectively (with both hands while rotating) 14P2a to 14P2c (motion that raises the right hand holding the pen while rotating) are stored. Note that the motion model data 14P2a-14P2c of the sailor warrior B includes motion model data portions 14P2aD-14P2cD of shaking the pen, and only a specific motion performed with the pen article is detected as a transformation motion. . For this reason, only the person with the pen of the beautiful girl warrior B can transform into the beautiful girl warrior B.

  The transformed image data 14Gu1 and 14Gu2 are stored as image data itself representing various movements, or a combination of movement data of limb skeletons and facial feature points and 3D image generation data. The transformation motion model data 14P1a to 14P1c and 14P2a to 14P2c are stored as image data representing the transformation motion itself, or movement data of the limb skeleton and facial feature points.

  FIG. 4 is a view showing the contents of special effect data stored in advance in the special effect data memory 23A of the first embodiment of the digital camera 10. As shown in FIG.

  The special effects of the present embodiment include a cure (healing) effect 23K1, 23K2 that returns the opponent from a disturbed state to a normal state, and a death (death reaper) effect 23K3 that changes the opponent from a normal state to a disturbed state.

  The special effect data memory 23A has a storage area for special effect motion model data 23P, special effect image data 23Gu, content of effects 23W, and special effect mark data 23Mu. For special effects A, B, and C respectively corresponding to a plurality of types of transformation image data 14Gu1, 14Gu2, and 24Gw1, special effect image data and audio data thereof are stored. For example, special effect image data 23Gu1 for Pretty Warrior A and its audio data “Cure Heart” 23Su1, special effect image data 23Bu2 for Pretty Girl Warrior B and its audio data “Cure Star” 23Su2, special effect image data for Shinigami 23Gu3 and its audio data “Death Moon” 23Su3.

  The storage area of the effect content 23W has the storage area of the effect type 23K and the target 23T of the effect, and the storage area of the effect type 23K corresponds to the sailors A and B (the target of the effect). A cure (healing) effect 23K1, 23K2 that restores the state from the disturbed state to the normal state and a death (death reaper) effect 23K3 that changes the opponent (effect target) from the normal state to the disturbed state corresponding to the reaper are stored.

  In the storage area of the effect target 23T in the storage area of the effect type 23W, for the special effects A, B, and C, for example, “partners to which special effect motion is directed” 23T1 as “parties to which special effects occur” , “All neighboring enemies” 23T2 and “all neighboring enemies” 23T3 are stored. Here, the surrounding enemies refer to the Shinigami, Masked (disturbed) Pretty Warrior for Pretty Soldier A and Pretty Warrior B. On the other hand, for a god of death, a masked (disturbed) girl warrior, it means normal girl warrior A and girl warrior B.

  Further, special effect mark data 23Mu1, 23Mu2, and 23Mu3 that express the plurality of types of special effect image data 23Gu1, 23Gu2, and 23Gu3 in a simplified manner are stored in the storage area of the special effect mark data 23Mu. Then, the plurality of types of special effect image data 23Gu1, 23Gu2, and 23Gu3 are generated and added from the predetermined positions of the corresponding transformation image data 14Gu1, 14Gu2, and 23Gw to the storage area of the special effect motion model data 23P. Special effect motion model data 23P1a to 23P1c consisting of a series of motions (gestures) (motions that project both hands toward each other to give a special effect from a state where both arms are separated from each other), 23P2a to 23P2c ( 23P3a to 23P3c (motion of swinging the stick forward with the right hand) and 23P3a to 23P3c (motion of turning the left hand on the waist and holding the pen with the right hand) Note that the special effect motion model data 23P2a to 23P2c of the sailor warrior B includes motion model data portions 23P2aD to 23P2cD for shaking the pen. For this reason, since only the specific motion performed with the pen article is detected as the special effect motion, only the person with the pen of the sailor B can exert the special effect of the sailor B. .

  The special effect image data 23Gu1, 23Gu2, and 23Gu3 are stored as image data itself or 3D image generation data. The special effect motion model data 23P1a to 23P1c, 23P2a to 23P2c, and 23P3a to 23P3c are stored as image data representing the special effect motion, or movement data of the limb skeleton and facial feature points.

  FIG. 5 is a diagram showing the contents of the defense effect data stored in advance in the defense effect data memory 23 </ b> B of the first embodiment of the digital camera 10.

  The defense effect is an effect that invalidates the special effect received from the opponent. For example, a person who has made a “cure effect” 23K1, 23K2, or a death effect 23K3, etc., by the opponent's special effect motion, defends and invalidates the opponent's special effect by performing this defense effect motion. is there.

  This defense effect data memory 23B has storage areas for defense effect motion model data 23′P, defense effect image data 23′Gu, and defense effect mark data 23′Mu, and the storage area for the defense effect image data 23′Gu. For the defense effects A and B, defense effect image data and sound data thereof are stored. For example, the defensive effect image data 23′Gu1 for invalidating the death effect 23K3 for the reaper P and the sound data “Bakin” 23′Su1, and the defensive effect image data 23′Gu2 for invalidating the cure effect 23K1 for the sailor A And the voice data “Parn” 23′Su2. Further, in the storage area of the defense effect mark data 23′Mu, the defense effect mark data 23′Mu1, 23′Mu2 each representing the plurality of types of defense effect image data 23′Gu1, 23′Gu2 is stored. . The special effect image data 23Gu1, 23Gu3 is stored in the storage area of the defense effect motion model data 23′P from a series of motions (gestures) for protecting the special effect image data 23Gu1, 23Gu3 at predetermined positions of the corresponding transformation image data 24Gw1, 14Gu2. Defense effect motion model data 23'P2a-23'P2c (motion that sticks out or tilts overhead), 23'P1a-23'P1c (motion that rotates the pen with the right hand in front of the face) is stored The

  Note that the defense effect motion model data 23′P1a to 23P1c of the bishoujo warrior B includes motion model data portions 23′P1aD to 23′P1cD for shaking the pen. Then, only the specific motion performed with the pen article is detected as the defense effect motion, and only the person having the pen of the beautiful girl warrior B can exert the defense effect of the defense of the beautiful girl warrior B. The reaper (P) defense effect motion model data 23'P2a-23'P2c includes motion model data portions 23'P2aB-23'P2cB for shaking a stick. Only the specific motion performed with the stick article is detected as the defense effect motion, and only the person with the god of death (P) can exert the defense effect of the defense of the god of death (P). .

  By using the special effect and the defense effect, for example, the beautiful girl warrior A (J) performs the cure effect motions 23P1a to 23P1c toward the reaper (P) to produce the cure effect 23K1 (FIG. 17) or the reaper (P ) Performs a defensive effect motion 23'P2a-23'P2c on the cure effect 23K1 received from the beautiful girl warrior A (J) to produce a defensive effect and invalidate the cure effect (FIG. 17), Shinigami (P ) Performs Death Effect Motions 23P3a-23P3c on Pretty Warrior A (J) to give Death Effect 23K3, transforming Pretty Soldier A (J) into a girl with a mask (Gk) in distress ( FIG. 18), Cure Effect Motion 23P2a-23P2c is performed on Pretty Warrior A (J) and Shinigami (P), who are in troubled state. A special effect motion is used for a scenario in which K2 is issued (FIG. 21), the sailor A (J) is returned to a normal state without a mask, and the god of death (P) is transformed into a prince (P) (FIG. 22). It can be realized just by acting with a, and it is possible to create a moving image of a special effects-like transformation composite image without preparation in advance.

  FIG. 6 is a view showing the contents of the transformation character data stored in the transformation character data memory 24A of the first embodiment of the digital camera 10. As shown in FIG.

  This transformation role data memory 24A includes transformation object image data 24T registered as a user, transformation (1) image data 24Gw and transformation (1) mark data 24Mw, transformation (2) image data 24Gw ′ and transformation (2) mark data 24Mw. 'Has a storage area for transformation condition data 24H, and the storage area for transformation (1) image data 24Gw stores multiple types of first stage transformation image data (transformation characters) and their voice data.

  In the storage area of the transformation target (face) image data 24T, a facial image of a person to be transformed into a plurality of types of transformation image data 24Gwn for each role and replaced is, for example, registered A; a facial image 24T1 of a boy P User registration is performed and stored in advance.

  As a result, in the storage area of the transformation (1) image data 24Gw corresponding to the image of the boy P of registration A, the role-death transformation (1) image data 24Gw1 and its audio data “Zaku, Zaku” 24Sw1 Remembered. Further, in the storage area of the transformation (1) mark data 24Mw, transformation (1) mark data 24Mw1 that simply represents the transformation (1) image data 24Gw1 is stored.

  Further, in the storage area of the transformation (2) image data 24Gw ′, the second stage transformation (2) image data 24Gw1 ′ and its audio data 24Sw1 ′ corresponding to the first stage transformation (1) image data 24Gw1 are stored. Remembered. Further, in the storage area of the transformation (2) mark data 24Mw ′, transformation (2) mark data 24Mw1 ′ that simply represents the transformation (2) image data 24Gw1 ′ is stored. Further, the storage area of the transformation condition data 24H stores the condition data 24H1 for transformation from the first stage transformation (1) image data 24Gw1 to the second stage transformation (2) image data 24Gw1 ′.

  For example, as the second stage transformation (2) image data 24Gw ′ corresponding to the image of the boy P, the role-prince transformation image data 24Gw1 ′ and its voice data “Kirari, Kirari,” 24Sw1 ′ are set. Then, “Beauty Warrior B Cure Effect” is set as the transformation condition data 24H1.

  Further, in the role-specific transformation state memory 24B in FIG. 2, the current transformation state (transformation (1) or transformation (2) of the registrants (registration A to registration D) registered in the transformation role data memory 24A is stored in the transformation state memory 24B. )) Is stored, and in the initial state, transformation (1) is stored. At the time when the transformation condition 24H for each role is satisfied (for example, in registration A, “after the cure effect 23K2 of Pretty Warrior B is detected”), the transformation state by this role is changed to “transformation (2)”. Change and memorize. In the first embodiment, the roles (registration B to registration D) are unregistered.

  The transformation (1) and (2) image data 24Gw and 24Gw ′ are image data representing various movements, or a combination of movement data of limb skeletons and facial feature points and 3D image generation data. Remembered.

  The photographed image data memory 25 stores photographed image data G input by taking a still image or a moving image by the image photographing unit 21.

  In the marked-captured image data memory 26, the marked-captured image data Gm (generated in real time for shooting according to the image processing program based on the captured image data G captured and stored in the captured image data memory 25). 1A) are sequentially stored.

  In the composite image data memory 27, composite image data GG generated according to the image processing program based on the marked captured image data Gm generated and stored in the marked captured image data memory 26 (see FIG. 1B). Is memorized.

  The digital camera 10 having the above configuration generates and displays the marked photographic image data Gm (see FIG. 1A) in real time based on the photographic image data G captured by the digital camera 10 itself. Thereafter, the composite image data GG (see FIG. 1B) is generated from the photographed image data Gm with the mark and displayed or output to the outside.

  On the other hand, photographic image data Gm with a mark (see FIG. 1 (A)) is generated from the photographic image data G taken with the moving image other than the photographic function of the digital camera 10, and from the photographic image data Gm with the mark Various functions for generating the composite image data GG (see FIG. 1B) are provided in the server device 30 on the communication network N, and a digital camera (communication terminal A) 10 ′ or PC (with communication function) is installed. Based on the moving image shooting data G transferred from the communication terminal B) 40, the server device 30 generates the marked shooting image data Gm and the composite image data GG, and the communication terminals (A) 10 'and (B). It is good also as a structure which responds to 40 and provides a service.

  FIG. 7 is a block diagram showing a configuration of an electronic circuit of the server apparatus 30 according to the composite image output apparatus of the present invention.

  The server device 30 includes a CPU 31 that is a computer.

  The CPU 31 operates in accordance with a system program stored in advance in the memory 32 or a server control program read into the memory 32 from an external recording medium 33 such as a CD-ROM via a recording medium reading unit 34 such as an optical disk drive. Control the behavior.

  In addition to the memory 32 and the recording medium reading unit 34 being connected to the CPU 31, data transmission control between the communication terminal A (digital camera) 10 ′ and the communication terminal B (PC) 40 on the communication network N is performed. Are connected to an electric transmission control unit 35, an input unit 36 such as a keyboard / mouse, and an LCD display unit 37.

  The program memory 32A of the memory 32 includes a system program that controls the overall operation of the server device 30, and communication operations with each communication terminal A (digital camera) 10 'and communication terminal B (PC) 40 on the communication network N. The communication control program for managing the image is stored, and based on the moving image photographic image data G transferred from each of the external communication terminals A10 ′ and B40, the marked photographic image Gm and the composite image are obtained as in the digital camera 10 described above. An image processing program for managing various functions described with reference to FIG. 1 for generating or outputting (distributing) GG is stored in advance.

  Various programs stored in the program memory 32A are activated in response to an input signal from the input unit 36 and a received input signal from each of the external communication terminals A10 'and B40 via the transmission control unit 35. .

  The memory 32 includes a transformation data memory 32B, a special effect data memory 32CA, a special effect data memory 32CB, which are substantially the same data contents as those in the digital camera 10 described with reference to FIGS. A transformation character data memory 32D, a character-specific transformation state memory 32E, a photographed image data memory 32F, a marked photographed image data memory 32G, a composite image data memory 32H, and other work data memories are provided.

  Thereby, in the server apparatus 30, based on the moving image photographic image data G transferred from the communication terminal A (digital camera) 10 'or the communication terminal B (PC) 40 connected to the communication network N, the same marked shooting as described above is performed. Image data Gm (see FIG. 1 (A)) and composite image data GG (see FIG. 1 (B)) are generated and distributed to the communication terminals A10 ′ and B40 that are the transfer source of the photographed image data G. Service.

  Next, the operation of the composite image output apparatus by the digital camera 10 or the server apparatus 30 according to the first embodiment having the above-described configuration will be described.

  FIG. 8 is a flowchart showing the overall process of the composite image output process by the composite image output apparatus of the first embodiment.

  FIG. 9 is a flowchart showing a target image extraction process for extracting a target image of a person (P, J, K) existing in the captured image (G) in accordance with the composite image output process of the first embodiment.

  FIG. 10 is a flowchart showing a role transformation mark process for adding a transformation mark (24 Mw) to a person image (P) whose role has been registered in accordance with the composite image output process of the first embodiment.

  FIG. 11 adds a transformation mark (14Mu) to a person image (J, K) in which a transformation motion (14P) is detected in accordance with the composite image output process of the first embodiment and outputs a corresponding voice (14Su). It is a flowchart which shows a motion transformation mark process.

  FIG. 12 shows a case where a special effect mark (23Mu) is added to a person image (P, J, K) in which a special effect motion (23P) is detected in accordance with the composite image output process of the first embodiment, and the corresponding sound (23Su) is added. Is a flowchart showing a special effect mark process for outputting ().

  In FIG. 13, the person image (P) (J, K) to which the transformation mark (24Mw) (14Mu) is added in accordance with the synthesized image output process of the first embodiment is replaced with the transformation image (24Gw) (14Gu). It is a flowchart which shows the transformation mark synthetic | combination process which sets the output of corresponding | compatible audio | voice (24Sw) (14Su).

  FIG. 14 shows that the special effect image 23Gu is additionally synthesized at a predetermined position of the person image (P, J, K) to which the special effect mark (23Mu) is added in accordance with the synthesized image output process of the first embodiment and the corresponding audio. It is a flowchart which shows the special effect mark synthetic | combination process which sets the output of (23Su).

  15 to 22 sequentially show image processing states based on the moving image captured image (G) accompanying the composite image output processing of the first embodiment as representative frame advance (No. 1) to (No. 8). FIG. 4A is a diagram illustrating a captured image G, FIG. 4B is a diagram illustrating a marked captured image Gm, and FIG. 3C is a diagram illustrating a composite image GG.

  In the combined image output processing steps S1 to S3 in FIG. 8, as shown in FIG. 15A to FIG. As a result, the captured image (Gm) with a mark as shown in FIGS. 15B to 22B is generated and output.

  Then, the processes of steps S4 to S7 of the composite image output process in FIG. 8 are executed based on a series of marked captured images (Gm) generated as shown in FIGS. 15 (B) to 22 (B). Then, a composite image (GG) as shown in FIGS. 15C to 22C is generated and output.

  Here, the case where the composite image output process in FIG. 8 is executed in the digital camera 10 (see FIG. 2) will be mainly described, and the composite image output process is executed in the server device 30 (see FIG. 7). Since the processing contents are substantially the same, the description thereof is omitted.

  When a series of scenes played by a family is photographed by the image photographing unit 21 of the digital camera 10, for example, as shown in FIGS. 15A to 22A, the photographed images G1 to G8 are sequentially captured image data. It is stored and stored in the memory 25 (step S1), and the process proceeds to the target image extraction process in FIG. 9 in real time (step SA).

  This target image extraction processing has two methods. In the target image extraction processing (method 1) shown in FIG. 9A, the captured image data Gn sequentially stored in the captured image data memory 25 is stored in the work memory. Are temporarily transferred and stored (step A11), and the changed portion of the image between the previous captured image data Gn-1 and the current captured image data Gn is blocked and extracted (step A12).

  Then, the position information blocked on the photographed image data Gn becomes the position information of the target image that is a person or an object, and is stored in the photographed image data memory 25 as additional information of the photographed image data Gn (step). A13).

  On the other hand, in the target image extraction process (method 2) shown in FIG. 9B, it is acquired by the distance sensor of the image photographing unit 21, which is additional information of the photographed image data Gn sequentially stored in the photographed image data memory 25. The distance information to the subject is read (step A21), and the image portion existing at a short distance is extracted as a block (step A22).

  Then, the position information blocked on the photographed image data Gn becomes the position information of the target image that is a person or an object, and is stored in the photographed image data memory 25 as additional information of the photographed image data Gn (step). A23).

  That is, the target image extraction process (method 2) shown in FIG. 9B requires distance information to the subject as additional information of the captured image data Gn, and is therefore suitable for processing by the digital camera 10, The target image extraction processing (method 1) shown in FIG. 9A is suitable for processing in the server device 30 because it does not require distance information to the subject in the captured image data Gn.

  As described above, when the person image (J, P, K) existing in the captured image data Gn is extracted by the target image extraction process (step SA), the process proceeds to the role transformation mark process in FIG. (Step SB).

  In this role transformation mark processing, the person image (J, P, K) in the photographed image data Gn extracted by the target image extraction processing is registered in the transformation role data memory 24A (see FIG. 6). Whether or not a person image (J) as transformation object (face) image data 24T is included is identified using the face image recognition process (step B1).

  Here, if it is identified that the person image having the face image 24T1 of the boy P as the transformation target (face) image data 24T is not included (step B1 (No)), the present process is terminated and the motion transformation is completed. Proceed to mark processing (step SD). Further, as shown in the photographed image data G1 to G8 in FIGS. 15A to 22A, a person image having a face image 24T1 of the boy P as the transformation target (face) image data 24T is included. If it is identified (step B1 (Yes)), the transformation state flag in the transformation state memory 24B classified by role is detected, and the role state of the registration A corresponding to the current person image P of the boy is detected ( Step B2).

If it is determined that the state is not the state of transformation (2) (the state of transformation (1)) (step B2 (No)), the special effect motion 23P is included in a series of captured images including the transformation target image. It is determined whether or not it has been detected (step B6). (This special effect will be described in detail in the special effect mark process (step SE) described later.)
For example, in the photographed image G1 shown in FIG. 15A, the special effect motion 23P is not detected (step B6 (No)), and the transformation stored in association with the transformation role data memory 24A (see FIG. 6). (1) Shinigami metamorphosis (1) Mark data 24Mw1 is read out and added corresponding to the recognized position of the face image of the identified boy's person image P, as shown in FIGS. Marked photographic image data Gm1 to Gm7 as shown in (B) are temporarily stored in the work data memory and displayed on the display unit 11 (step B8).

  Whether the transformation (1) mark data 24Mw1 added corresponding to the position of the face image of the registered person image P is added to the face image or in the vicinity of the face image. Any of these may be used.

  If it is determined that the digital camera 10 is currently shooting (step B4 (Yes)), the voice data “Zakutsu” stored in association with the transformation character data memory 24A (see FIG. 6) is stored. , Zakutsu "24Sw1 is read out and output from the audio output unit 13 as shown in FIGS. 15B to 21B (step B5).

  Similarly, when the transformation state of registration A corresponding to the current image P of the boy's boy P is detected, it is determined that the state is not the state of transformation (2) (the state of transformation (1)). (Step B2 (No)), when it is detected that a special effect motion 23P2 is performed by the younger sister K as shown in FIGS. 21 and 22, for example, in a series of captured images including the transformation target image. (Step B6 (Yes)), the role status flag in the role-specific transformation status memory 24B is changed to the status of transformation (2) (step B7) and transformed to the position corresponding to the transformation target image (P). (2) The mark 24Mw1 ′ is added, and is temporarily stored in the work data memory as marked photographed image data Gm8 as shown in FIG. 22B and displayed on the display unit 11 (step B3).

  If it is determined that the digital camera 10 is currently photographing (step B4 (Yes)), the transformation (2) stored in association with the transformation role data memory 24A (see FIG. 6). Voice data “Sparkle, Sparkle” 24Sw1 ′ is read out and output from the voice output unit 13 as shown in FIG. 22B (step B5).

  On the other hand, if the transformation state of registration A corresponding to the current boy's person image P is detected and it is determined that the state is transformation (2) (step B2 (Yes)), the transformation target image is handled. The transformation (2) mark 24Mw1 ′ is added to the position, and is temporarily stored in the work data memory as marked image data Gm1 to Gm8 as shown in FIG. B3).

  If it is determined that the digital camera 10 is currently photographing (step B4 (Yes)), the voice data “shiningly” stored in association with the transformation character data memory 24A (see FIG. 6) is stored. , “Sparkle” 24Sw1 ′ is read out and output from the audio output unit 13 as shown in FIG. 22B (step B5).

  Then, the process proceeds to the motion transformation mark process in FIG. 11 (step SD).

  In this motion transformation mark process, the person images (J, P, K) in the series of photographed image data Gn extracted by the target image extraction process are stored in advance in the transformation data memory 14 (see FIG. 3). It is determined whether or not an image portion matching the series of transformation motion model data 14P1a to 14P1c of the sailor warrior A and the series of transformation motion model data 14P2a to 14P2c of the sailor warrior B is detected. At this time, since the transformation model data 14P2a-14P2c for the sailor warrior B includes motion model data portions 14P2aD-14P2cD for shaking the pen, only the specific motion performed with the pen article is transformed. Detected as motion. For this reason, only the person (K) having the pen of the beautiful girl warrior B can be transformed into the beautiful girl warrior B. (Step D1).

  That is, as shown in the photographed image data G2 in FIG. 16A, a series of older sister person images J extracted by the target image extraction process are stored in advance in the transformation data memory 14 (see FIG. 3). When it is detected that the image portion matches with the series of transformation motion model data 14P1a to 14P1c of the beautiful girl warrior A (step D1 (Yes)), it is associated with the transformation data memory 14 (see FIG. 3). The transformation mark data 14Mu1 of the beautiful girl warrior A stored is read out and added corresponding to the position of the face image of the sister image J of the motion detected as shown in FIG. 16B. Such marked image data Gm2 is temporarily stored in the work data memory and displayed on the display unit 11 (step D2). On the other hand, when an image portion whose gesture matches the transformation motion model data 14P is not detected (step D1 (No)), this process is terminated and the process proceeds to the special effect mark process (step SE) in FIG.

  It should be noted that the transformation mark data 14Mu1 added corresponding to the position of the face image of the person image J detected in motion is either added over the face image or added in the vicinity of the face image. It may be.

  If it is determined that the digital camera 10 is currently shooting (step D3 (Yes)), the beautiful girl warrior A stored in association with the transformation data memory 14 (see FIG. 3) is stored. Audio data “Lemon ♪ Lemon ♪” 14Su1 is read and output from the audio output unit 13 as shown in FIG. 16B (step D4).

  Further, as shown in the photographed image data G6 in FIG. 20A, a series of younger person images K extracted by the target image extraction process are stored in advance in the transformation data memory 14 (see FIG. 3). When it is determined that the image motion is the same as the transformation motion model data 14P2a to 14P2c including the motion model data portions 14P2aD to 14P2cD for waving the series of the sailor B (step D1 (Yes)), Only a person with the pen of the role of the beautiful girl warrior B can be transformed into the beautiful girl warrior B. As a result, the transformation mark data 14Mu2 of the beautiful girl warrior B stored in association with the transformation data memory 14 (see FIG. 3) is read out, and the facial image of the sister's person image K whose motion is detected is recognized. 20B and 22B, the image data Gm6 to Gm8 with marks as shown in FIGS. 20B to 22B are temporarily stored in the work data memory and displayed on the display unit 11 (step S11). D2).

  Note that the transformation mark data 14Mu2 added corresponding to the position of the face image of the person image K whose motion has been detected is either added over the face image or added in the vicinity of the face image. It may be.

  If it is determined that the digital camera 10 is currently shooting (step D3 (Yes)), the beautiful girl warrior B stored in association with the transformation data memory 14 (see FIG. 3) is stored. The audio data “Cute ♪ Cute ♪” 14Su2 is read and output from the audio output unit 13 as shown in FIGS. 20B to 22B (step D4).

  Then, the process proceeds to the special effect mark process in FIG. 12 (step SE).

  The special effect (see FIG. 4) in the embodiment of the composite image output apparatus includes a cure (healing) effect 23K1, 23K2 that transforms the opponent from an evil state into a normal state, and an opponent in an evil state from the normal state. There is a death (reaper) effect 23K3 that transforms into

  In this special effect mark processing, a person image (P, J, K) in a series of photographed image data Gn extracted by the target image extraction processing is stored in advance in the special effect data memory 23A (see FIG. 4). Detected image portions corresponding to the series of transformation motion model data 23P1a to 23P1c of special effect A, the series of transformation motion model data 23P2a to 23P2c of special effect B, and the series of transformation motion model data 23P3a to 23P3c of special effect C are detected. It is determined whether or not it has been done. At this time, when a pen or a stick is included in the special effect motion model data 23P2 or 23P3 like the special effect B or the special effect C, only the matching motion by the person having the pen or the stick is detected as the special effect motion ( Step E1).

  That is, as shown in the photographed image data G3 in FIG. 17A, a series of older sister person images J extracted by the target image extraction process are stored in advance in the special effect data memory 23A (see FIG. 4). If it is detected that the gesture part matches the series of transformation motion model data 23P1a to 23P1c of the special effect A (step E1 (Yes)), the special effect data memory 23A (see FIG. 4) is associated. The special effect mark data 23Mu1 of the special effect A stored and read out is read out, and corresponding to a predetermined position (for example, a third position from the upper part of the image) of the elder person image J whose motion is detected, A person to whom the transformation mark data 24Mw1 of the reaper P, which is the target 23T1 of the effect (the other party to which the special effect motion is directed), is added Is added toward the direction of the image (Boy) P, is displayed on the display unit 11 is temporarily stored in a working data memory as marked photographic image data Gm3 as shown in FIG. 17 (B) (step E2).

  If it is determined that the digital camera 10 is currently photographing (step E3 (Yes)), the special effect A stored in association with the special effect data memory 23A (see FIG. 4) is stored. The audio data “Cure Heart” 23Su1 is read out and output from the audio output unit 13 as shown in FIG. 17B (step E4). In this way, the audio data “Cure Heart” 23Su1 of the special effect A is output from the digital camera 10 so that the special effect processing is appropriately executed by the computer (actor (sister J, boy P)). It can be easily recognized and can perform with a realistic feeling.

  Whether the special effect mark data 23Mu1 added corresponding to a predetermined position of the person image J whose motion is detected is added to the person image J or added in the vicinity of the person image J. Either may be sufficient. As a result, it is easy for the mother who is taking a picture to have the sister (J) perform the special effect motion, so that the special effect processing is appropriately executed by the computer and a composite image to which the special effect image is added is created. Can be recognized.

  Further, as shown in the photographed image data G4 in FIG. 18A, the special effect data memory 23A (see FIG. 4) for a series of person images P of the boy registered in the face image extracted by the target image extraction process. ), A series of transformation motion model data 23P3a to 23P3c of special effect C stored in advance and a portion having a stick (23P3aB to 23P3cB) are detected to be image portions that match gestures (step E1). (Yes)), the special effect mark data 23Mu3 of the special effect C stored in association with the special effect data memory 23A is read out, and a predetermined position (for example, an image) of the person image P of the boy whose motion is detected The beautiful girl corresponding to the effect target 23T3 (all neighboring enemies) corresponding to the position of one third from the top) Is added in the direction of the person image (sister) J to which the transformation mark data 14Mu1 of the person A has been added, and is temporarily stored in the work data memory as marked image data Gm4 as shown in FIG. It is displayed on the display unit 11 (step E2).

  If it is determined that the digital camera 10 is currently photographing (step E3 (Yes)), the special effect C stored in association with the special effect data memory 23A (see FIG. 4) is stored. The audio data “Death Moon” 23Su3 is read and output from the audio output unit 13 as shown in FIG. 18B (step E4). As described above, the actor (sister J, boy P) indicates that the special effect processing is appropriately executed by the computer by outputting the audio data “Death Moon” 23Su3 of the special effect C from the digital camera 10 in this way. Can be easily recognized.

  Further, as shown in the photographed image data G7 in FIG. 21A, the special effect B stored in advance in the special effect data memory 23A for a series of younger person images K extracted by the target image extraction process. If it is detected that the image portion includes a series of special effect motion model data 23P2a to 23P2c and a portion having a pen (23P2aD to 23P2cD) that the gestures match (step E1 (Yes)), the special effect data The special effect mark data 23Mu2 of the special effect B stored in association with the memory 23A (see FIG. 4) is read out, and a predetermined position (for example, 3 minutes from the upper part of the image) of the younger sister's person image K whose motion is detected. With the mask mark data Mk corresponding to the effect target 23T2 (all nearby enemies). State) A character image (sister) J of the beautiful girl warrior A and a character image (boy) P to which the transformation mark data 24Mw1 of the god of death is added are added and marked as shown in FIG. The captured image data Gm7 is temporarily stored in the work data memory and displayed on the display unit 11 (step E2).

  If it is determined that the digital camera 10 is currently photographing (step E3 (Yes)), the special effect B stored in association with the special effect data memory 23A (see FIG. 4) is stored. The audio data “Cure Star” 23Su2 is read out and output from the audio output unit 13 as shown in FIG. 21B (step E4). In this way, the audio data “CURE STAR” 23Su2 of the special effect B is output from the digital camera 10, so that the special effect processing is appropriately executed by the computer (sister K, sister J, It can be easily recognized by boy P).

  Here, as a special effect (see FIG. 4), there are a cure effect 23K1, 23K2 for returning the opponent who is the target 23T of the effect from the disturbed state to the normal state, and a death (Reaper) that changes the normal state to the disturbed state. ) Effect 23K3, while the defense effect (see FIG. 5) invalidates the special effect received from the opponent. The transformation process (steps E5 to E23) associated with the special effect and the defense effect will be described below.

  That is, as shown by the marked image data Gm3 in FIG. 17B, the special effect A (cure effect 23K1) is changed from the position of the sister (J) to which the transformation mark data 14Mu1 of the sailor warrior A is added. When the accompanying special effect mark data 23Mu1 is displayed and output (steps E1 and E2), the target 23T of the special effect A stored in advance in the special effect data memory 23A (see FIG. 4) is all the surrounding enemies. If it is determined that the opponent 23T1 is directed to the special effect motion that is not (step E5 (No)), the opponent (the person image P that is the boy of the god of death) who directed the special effect motion 23P1 in the screen (Gm3). ) Is detected and set as the subject of the effect (step E12). In this way, the person image of the other party indicated by the gesture (motion) of the person image is detected and set as the subject of the special effect, so that the special effect is appropriately applied to the intended partner of the performer who performs the special effect. And can easily create a composite image in which the other party is transformed.

  If it is determined that the effect type 23K of the special effect A is the cure effect 23K1 (step E7 (Yes)), the target of the detected and set effect is a portrait image of a sailor with a mask mark Mk. It is determined that it is not (step E13 (No)), and is determined to be a person image P with the reaper mark 24Mw1 (step E15 (Yes)).

  And about the person image P of the boy who is the subject of the effect (the role of the reaper), the defense effect motion model data 23′P2a to 23′P2c in FIG. 5 and the part having the stick (23′P2aB to 23′P2cB) are also included. Then, when it is detected that the image portion is the same as the gesture (step E16 (Yes)), a heart indicating that the special effect has been invalidated is adjacent to the corresponding heart-shaped special effect mark data 23Mu1. The defensive effect mark data 23′Mu2 in a broken shape is read and added (step E17). In this case, since the protective effect of the god of death (P) is exerted, the transformation state of the god of death is continued.

  Then, the audio data “Parn” 23′Su2 corresponding to the defense effect mark data 23′Mu2 is read and output from the audio output unit 13 (step E22).

  As described above, the defense effect mark 23′Mu2 indicating that the special effect is invalidated is displayed as the marked photographed image Gm3, so that the mother who is photographing while viewing the camera screen (11) It is possible to visually confirm that the effect is properly produced and the special effect is invalidated. In addition, the voice data “Pan” 23′Su2 corresponding to the defense effect mark 23′Mu2 is output, so that not only the mother but also all the performers can properly achieve the defense effect and invalidate the special effect. Can be confirmed audibly.

  Further, as shown by the marked photographed image data Gm4 in FIG. 18B, the special effect C (death effect 23K3) is determined from the position of the boy (P) registered with the face image to which the death mark transformation mark data 24Mw1 is added. ) Associated with special effect mark data 23Mu3 is displayed and output (steps E1, E2), the special effect C effect target 23T stored in advance in the special effect data memory 23A (see FIG. 4) If it is determined that all the enemies are 23T3 (step E5 (Yes)), the opponent's opponent (person image J who is the sister of the role of the beautiful girl warrior A) is detected in the screen (Gm4), and the subject of the effect (Step E6).

  Then, when it is determined that the effect type 23K of the special effect C is the death effect 23K3 (step E7 (No)), the target of the detected and set effect is attached with the transformation mark data 14Mu1 of the sailor warrior. It is determined that there is (step E8 (Yes)).

  Here, when the defense effect motion (see FIG. 5) of the defense effect A is not detected for the person image J of the sister (the role of the beautiful girl warrior A) who is the target of the effect (step E9 (No)), the beauty Next to the transformation mark data 14Mu1 corresponding to the girl warrior A, mask mark data Mk indicating that the girl warrior A has been transformed into a disturbing state is added (step E10).

  In other words, in this case, the image J of the sister of the role of the beautiful girl warrior A is a death (death reaper) corresponding to the motion 23P3a to 23P3c of the special effect C by the character image P of the boy of the reaper registered in advance in the transformation role data memory 24A. ) By the effect, it has been transformed into a distressed masked girl warrior.

  Then, audio data “Cook Cook” Sk corresponding to the mask mark data Mk is output from the audio output unit 13 (step E19).

  In step E9, the motion model data 23′P1a to 23′P1c (see FIG. 5) of the defense effect A for the person image J of the sister (role of the girl warrior A) who is the target 23T3 of the death effect 23K. , If it is detected that the image includes a portion having a pen (23′P1aD to 23′P1cD) (step E9 (Yes)), the corresponding moon-shaped special effect mark data 23Mu3 Next, the defense effect mark data 23′Mu1 in the form of a broken moon indicating that the special effect has been invalidated is read and added (step E11). In this case, the transformation effect of the sailor warrior A will be continued by providing the defense effect from the older sister J of the sailor warrior A role.

  Then, the sound data “backing” 23′Su1 corresponding to the defense effect mark data 23′Mu1 is read and output from the sound output unit 13 (step E20). However, as in the present embodiment, when it is determined that only the girl warrior B has a pen on the role, since the sister (J) of the girl warrior A role does not have a pen, the defense effect motion The gesture corresponding to 23′P1a to 23′P1c is not detected, and the defense effect A cannot be produced.

  Further, as shown by the marked photographed image data Gm7 in FIG. 21B, the special effect B (cure effect 23K2) is changed from the position of the sister (K) to which the transformation mark data 14Mu2 of the sailor warrior B is added. When the accompanying special effect mark data 23Mu2 is displayed and output (steps E1 and E2), the target 23T of the special effect B stored in advance in the special effect data memory 23A (see FIG. 4) is all the enemies in the vicinity. If it is determined that it is 23T2 (step E5 (Yes)), the image of the sister who has been transformed into a troubled girl warrior A with mask Mk as an opponent in the screen (Gm7) J and the person image P of the boy to which the transformation mark data 24Mw1 of the Shinigami role is added are detected and set as the subject of the effect (step E6).

  Then, when it is determined that the effect type 23K2 of the special effect B is a cure effect (step E7 (Yes)), in the case of the sister's person image J that is the target of the detected and set effect, the mask mark Mk It is judged that it is a beautiful girl warrior A (step E13 (Yes)).

  Then, as shown by the marked image data Gm8 in FIG. 22B, the mask mark data Mk of the sailor A with the mask mark Mk is deleted (step E14).

  In other words, in this case, the sister image of the role of the beautiful girl warrior A is masked by the cure effect corresponding to the motions 23P2a to 23P2c of the special effect B by the character image K of the sister of the role of the beautiful girl warrior B. The troubled sailor A has returned to the original normal sailor A.

  Then, the audio data 14Su1 “Lemon ♪ Lemon ♪” corresponding to the transformation mark data 14Mu1 of the original Sailor A is read and output from the audio output unit 13 (step E21).

  On the other hand, when it is determined that the effect type 23K2 of the special effect B is a cure effect (step E7 (Yes)), in the case of the person image P of the boy of the reaper who is the object of the detected and set effect Is determined to have the reaper mark 24Mw1 (step E15 (Yes)).

  Here, when the defense effect motion (see FIG. 5) of the defense effect B is not detected for the person image P of the boy (the god of death) who is the subject of the effect (step E16 (No)), FIG. 22 (B ) Marked photographic image data Gm8, the god of death metamorphosis mark data 24Mw1 is deleted from the person image P of the boy (rage of god of death), and the transformation of the star prince (2) mark data 24Mw1 ' It is added (step E18).

  That is, in this case, the character image P of the boy who plays the role of Shinigami is converted from the Shinigami to the original star by the cure effect corresponding to the motions 23P2a to 23P2c of the special effect B by the character image K of the sister of the role of the beautiful girl warrior B. It was transformed into a prince and returned.

  Then, the transformation of the prince of the stars (2) The voice data 24Sw1 ′ “brilliantly” corresponding to the mark data 24Mw1 ′ is read and output from the voice output unit 13 (step E23).

  As described above, when the special effect mark processing (steps E1 to E23) based on the detection of the special effect motion 23P and the defense effect motion 23′P from the person image (P, J, K) in the captured image G is completed, As shown in FIGS. 15B to 22B, the continuous photographed image data Gm1 to Gm8 with marks generated on the working data memory in accordance with the processing of steps SA to SE are marked photographed images. It is stored and saved in the data memory 26 (step S2).

  When it is determined that the shooting input of the series of shot image data G1 to G8 from the image shooting unit 21 has been completed (step S3 (Yes)), the series of the shot images generated and stored in the marked shot image data memory 26. It is determined whether or not to proceed to the composite image generation process based on the marked image data Gm1 to Gm8 (step S4).

  Here, when it is determined that the process proceeds to a composite image generation process in response to a user operation for generating a composite image on the input unit 12 or automatically upon completion of acquisition of the series of captured image data (step S4). (Yes)), the series of marked photographed image data Gm1 to Gm8 is read from the marked photographed image data memory 26 to the working memory (step S5), and the process proceeds to the transformation mark composition process in FIG. Step SF).

  In this transformation mark composition process, first, the marked photographed image data Gm1 to Gm8 read from the marked photographed image data memory 26 to the working memory is stored in advance in the transformation character data memory 24A (see FIG. 6). It is determined whether or not an image to which the transformation mark data 14Mu stored in advance is included in the transformation mark data 24Mw being stored or the transformation data memory 14 (see FIG. 3) (step F1).

  Here, the transformation mark data 24Mw stored in advance in the transformation role data memory 24A (see FIG. 6) and the transformation mark data 14Mu stored in advance in the transformation data memory 14 (see FIG. 3) are added. Is determined not to be included (step F1 (No)), the process is terminated and the process proceeds to a special effect mark composition process (step SH).

  Further, as shown in FIGS. 15B to 21B, the image of the Shinigami transformation (1) with the mark data 24Mw1 added, and the transformation back to the star prince as shown in FIG. 22B (2 ) An image to which the mark data 24Mw1 ′ is added, an image to which the transformation mark data 14Mu1 of the sailor warrior A is added as shown in FIGS. 16B to 22B, and FIG. 20B. When it is determined that there is an image to which the transformation mark data 14Mu2 of the sailor warrior B is added as shown in FIG. 22 (B) (step F1 (Yes)), the transformation mark data 24Mw1, 24Mw1 ′ , 14Mu1, 14Mu2 corresponding to the positions of the person images P, J, K in the vicinity of the mark are obtained as additional information of the photographed image data Gn by the target image extraction process (step SA). It is specified from the position information of the existing block (step F2).

  Then, the identified person images P, J, and K are subjected to motion capture processing, and the Shinigami transformation mark data 24Mw1 and the star prince transformation mark data 24Mw1 stored in the transformation role data memory 24A (see FIG. 6). Transformation corresponding to ′ (1) image data 24Gw1 and transformation (2) image data 24Gw1 ′, transformation mark of the sailor A and sailor B stored in the transformation data memory 14 (see FIG. 3) Based on the transformation image data 14Gu1 and 14Gu2 corresponding to the data 14Mu1 and 14Mu2, as shown in FIGS. 15 (C) to 21 (C), the transformation of the reaper in the same pose (posture) as the person image (boy) P (1) Image data 24Gw1, the king of stars in the same pose (posture) as the person image (boy) P as shown in FIG. Transformation (2) image data 24Gw1 ′, transformation image data 14Gu1 of a sailor A in the same pose (posture) as the person image (sister) J, as shown in FIGS. 16C to 22C, FIG. As shown in FIG. 20 (C) to FIG. 22 (C), the transformation image data 14Gu2 of the sailor B in the same pose (posture) as the person image (sister) K is replaced on the preset background image BG. Are synthesized (step F3). At this time, composite image data GG1 to GG8 are generated (step F3).

  Then, it is determined whether or not there is face synthesis position data in the transformed image data replaced corresponding to each mark data (step F4).

  In this case, the face synthesis position data corresponding to the Shinigami transformation (1) image data 24Gw1 is not stored, and the process proceeds to Step F8. Further, it is determined that there is face synthesis position data 24F ′ (see FIG. 6) corresponding to the transformation of the prince of the stars (2) image data 24Gw1 ′, and face synthesis corresponding to the transformation image data 14Gu1 of the sailor warrior A. It is determined that there is position data 14F1 (see FIG. 3), and it is determined that there is face synthesis position data 14F2 (see FIG. 3) corresponding to the transformation image data 14Gu2 of the sailor B (step F4 (Yes)).

  Then, the face synthesis position data 24F ′, 14F1, and 14F2 are read (step F5), and the face image portion (Pf, Jf, Kf) in the target image in each corresponding captured image Gn is extracted (step F6). ), The extracted face image portions (Pf, Jf, Kf) are resized in accordance with the face synthesis positions (24F ′, 14F1, 14F2) of the corresponding transformation image data 24Gw1 ′, 14Gu1, 14Gu2 (Pf). ', Jf', Kf '), as shown in FIGS. 16C to 22C, the image data (24Gw1', 14Gu1, 14Gu2) of the transformation character is synthesized (step F7).

  In this manner, the face image portion (Pf, Jf, Kf) extracted from the captured image data Gn is resized according to the position corresponding to the face synthesis position data, and the image data (24Gw1) of each corresponding transformation character ', 14Gu1, 14Gu2), the face part of the live-action image is appropriately combined according to the face size of the make-up character, and the make-up character can be synthesized and displayed with the face of the live-action image.

  For this reason, it is possible to easily display an image of a transforming character as small as a fairy, or as large as a giant.

  As described above, each corresponding face image portion (Pf, Jf, Kf) is resized according to the face synthesis position (24F ′, 14F1, 14F2) of each transformation image data (24Gw1 ′, 14Gu1, 14Gu2). The transformation image data (24Gw1 ′, 14Gu1, 14Gu2) is used as transformation image data of only the costume portion without the face portion, and the corresponding person image (boy P, sister J, sister K) is not synthesized. ) May be simply combined.

  Next, in this transformation mark compositing process, it is determined whether or not the masked mark data Mk is included in the marked imaged image data Gmn to be processed (step F8).

  That is, the mask mark data Mk is added to the person image J of the sister of the role of the beautiful girl warrior A as in the marked imaged image data Gm4 to Gm7 shown in FIGS. 18B to 21B. If it is determined (step F8 (Yes)), as shown in FIGS. 18 (C) to 21 (C), it is matched with the face synthesis position 14F1 of the face image portion in the person image J of the sister. The combined image GG is generated by combining the mask image Gk (step F9).

  Then, the reaper voice data “Zaku, Zaku” 24Sw1, the prince of the star is stored in the transformation character data memory 24A (see FIG. 6) in association with the respective transformation image data (24Gw1 ′, 14Gu1, 14Gu2). Voice data “Silver ♪ Lemon ♪” 14Su1, “Cute ♪ Cute ♪” voice data “Silver ♪ Lemon ♪” 14 Su1, “Sound ♪ Cute ♪” stored in the transformation data memory 14 (see FIG. 3) 14Su2 is read and stored in association with the synthesized images GG1 to GG8 generated as shown in FIGS. 15C to 22C (step F10). Here, when the mask mark data Mk is added to the figure image J of the sister of the role of the beautiful girl warrior A and the figure image K of the sister of the role of the beautiful girl warrior B, the beautiful girl warrior A, beauty The voice data “Cook Cook” Sk corresponding to the mask mark data Mk is associated with the synthesized images GG1 to GG8 instead of the voice data “Lemon ♪ Lemon ♪” 14Su1, “Cute ♪ Cute ♪” 14Su2 of the girl warrior B. Saved. (Step F10).

  If it is determined that the masked image data Gmn to be processed does not include the masked mark data Mk (step F8 (No)), the masked image Gk combining process and voice setting process (step S8). F9, F10) are skipped.

  Then, the process proceeds to the special effect mark composition process in FIG. 14 (step SH).

  In this special effect mark composition process, first, the marked image data Gm1 to Gm8 read from the marked image data memory 26 to the working memory is stored in advance in the special effect data memory 23A (see FIG. 4). It is determined whether or not the stored special effect mark data 23Mu or the defense effect data memory 23B (see FIG. 5) includes image data Gmn to which the previously stored defense effect mark data 23′Mu is added. (Step H1).

  Here, as shown in FIG. 15B, FIG. 16B, FIG. 19B, and FIG. 20B, image data to which special effect mark data 23Mu and defense effect mark data 23'Mu are added. If it is determined that Gm does not exist (step H1 (No)), the process ends, and the process proceeds to step S6.

  On the other hand, as shown in FIGS. 17B, 18B, 21B, and 22B, the special effect mark data 23Mu1 corresponding to the special effect A and the defense effect corresponding to the defense effect B If it is determined that the mark data 23′Mu2, the special effect mark data 23Mu3 corresponding to the special effect C, and the special effect mark data 23Mu2 corresponding to the special effect B are present (step H1 (Yes)), the special effect mark The position of the person image (sister, sister, boy) J, K, P in the vicinity of the mark to which the data 23Mu1, 23Mu2, 23Mu3 or the defense effect mark data 23'Mu2 is added correspondingly is the target image extraction process (step It is specified from the position information of the block obtained as additional information of the photographed image data Gn by (SA) (step H2).

  As a result, the special effect mark data 23Mu1, 23Mu2, 23Mu3 in the special effect data memory 23A (see FIG. 4) or the defense effect mark data 23′Mu2 in the defense effect data memory 23B (see FIG. 5) is stored. Special effect image data 23Gu1, 23Gu2, 23Gu3 or defense effect image data 23'Gu2 being read out, and a predetermined position (for example, 3 minutes from the upper part of the image) of each of the specified person images (J, K, P) Corresponding to the position (1)), the composite image data GG is generated by further compositing in the direction of the person image to which the transformation mark data of the enemy role (the other party) is added (step H3).

  For example, as shown in FIGS. 17B and 17C, the special effect image data 23Gu1 is used as a god of death from the position of the arm of the character image J of the beautiful girl warrior A role to which the special effect mark data 23Mu1 is added. Are additionally synthesized in the direction of the person image P. On the other hand, from the position of the arm of the person image P of the reaper role to which the defense effect mark data 23'Mu2 is added, the defense effect image data 23'Gu2 is directed in the direction of the person image J of the beautiful girl warrior A role. Is added and synthesized.

  As shown in FIGS. 18B and 18C, the special effect image data 23Gu3 is obtained from the position of the arm of the person image P of the reaper role to which the special effect mark data 23Mu3 is added. Additional synthesis is performed in the direction of the person image J of the role.

  Further, as shown in FIGS. 21B and 21C, the special effect image data 23Gu2 is disturbed from the position of the arm of the person image K of the role of the beautiful girl warrior B to which the special effect mark data 23Mu2 is added. The mask is added and synthesized in the direction of the person image J of the role of the sailor A (step H3).

  Further, depending on the special effect, the tip of the special effect image data 23Gu may be used as an explosion effect image and extended to a position where it reaches the transformation image data of the person image of the opponent who is the enemy character (additional composition). Step H4).

  The special effect A audio data “Cure Heart” 23Su1, the special effect B audio data “Cure Star” 23Su2, and the special effect C stored in the special effect data memory 23A (see FIG. 4) are stored. The voice data “Death Moon” 23Su3 is read out, and the voice data “Pan” 23′Su2 of the defense effect B stored in association with the defense effect data memory 23B (see FIG. 5) is read out. 17C, FIG. 18C, FIG. 21C, and FIG. 22C are stored in association with the composite images GG3, GG4, GG7, and GG8 generated (steps). H5).

  Then, as shown in FIGS. 15C to 22C, a series of synthesized image data GG1 to GG8 generated on the working data memory in accordance with the processing of the steps SF to SH is obtained as a synthesized image data memory. 27 and stored (step S6).

  A series of synthesized image data (including associated audio data) GG1 to GG8 stored in the synthesized image data memory 27 is displayed as shown in FIGS. 15 (C) to 22 (C). Reproduced and displayed by the unit 11 and reproduced and output by the audio output unit 13 (step S7). When the composite image output device is configured as the server device 30 (see FIG. 7), the communication image on the communication network N that is the transmission source of the series of captured image data G1 to G8 received in step S1. For the communication terminal A (digital camera) 10 ′ and the communication terminal B (PC) 40, a series of synthesized image data (including associated audio data) GG1 to GG8 stored in the synthesized image data memory 27. The read response is transmitted (step S7).

  Therefore, according to the composite image output device of the first embodiment using the digital camera 10 or the server device 30 configured as described above, when a series of captured image data Gn captured by the image capturing unit 21 is input, the captured image A person image (P, J, K) included in the data Gn is extracted, and for the person image P of a god of death role registered in advance as a face image in the transformation role data memory 24A, the transformation image (1) of the god of death concerned. A composite image GG1 replaced with the data 24Gw1 is generated, and whether or not the special effect motion (gesture) 23P3a to 23P3c of the death effect 23K3 preset in the special effect data memory 23A is detected and determined. When it is detected and determined that the special image motion (gesture) 23P3a to 23P3c of the death effect 23K3 set in advance for the person image P of the god of death included in the photographed image data G4, The special effect image data 23Gu3 corresponding to the death effect 23K3 is synthesized in the direction of the person image J of the role of the beautiful girl warrior A set as the target 23T3 of the effect, and the person image of the role of the beautiful girl warrior A A mask image Gk is added and synthesized to J, and is generated and output as a composite image GG4 transformed into a sailor A with a mask.

  That is, according to the composite image output apparatus of the first embodiment as described above, the special effect motion (gesture) 23P3a to the death effect 23K3 set in advance for the person image P of the reaper role registered in advance by the face image. If it is detected that the player is performing 23P3c, the mask image Gk is added to the person image J of the role of the girl warrior A set as the subject 23T3 of the death effect, and transformed into the girl warrior A with a mask. The generated composite image GG4 is generated and output.

  As a result, among the person images (P, J, K) extracted from the captured image Gn captured by the moving image, only the predetermined special effect motion 23P3 based on the person image P of the boy registered in advance is used. It is possible to generate a composite image in which the person image J of the other party set as the special effect target 23T3 is replaced with a transformation image, and a character as determined in advance has a predetermined gesture ( By performing (motion), any amateur can easily generate a transformation composite image in a special effects movie or animation that transforms a specific opponent. Then, the generated synthesized moving image GGn can be output and displayed on the display unit 11 or can be output and displayed on an external monitor to enjoy unprecedented enjoyment.

  Further, according to the composite image output device of the first embodiment using the digital camera 10 or the server device 30 having the above-described configuration, a person image P of a boy registered in advance as a god of death is detected from the photographed image Gn, and its gesture (motion (1) When the synthesized image GGn is generated by replacing the person image P with the image data 24Gw1, and the combined image GGn is generated, the transformation (1) image data stored in the transformation character data memory 24A is generated. 24Gw1 audio data “Zaku, Zaku,...” 24Sw1 is stored and output in association with the synthesized image GGn, so that it is possible to easily and more realistically create a special effect-like synthesized moving image GGn.

  Further, according to the composite image output device of the first embodiment using the digital camera 10 having the above-described configuration, the captured image data Gn displayed in real time on the display unit 11 during the moving image shooting is registered in advance as the Shinigami role. When a boy's person image P is detected from the photographed image Gn, the person image P is transformed into a god of death (1) transformation (1) mark data 24Mw1 indicating that the person is transformed into image data 24Gw1. Is displayed on the display unit 11 as mark-captured photographic image data Gmn added, and the audio data “Zaku, Zaku,...” 24Sw1 of the transformation (1) image data 24Gw1 is also output from the audio output unit 13. The family being operated is transformed by replacing the subject person image (boy) P according to the registered role. Can be easily recognized, and even in the subject (boy P) who is playing this, it is easy to hear by voice that he has been transformed according to the registered role. Can be recognized.

  In the composite image output apparatus according to the first embodiment, only the transformation object (face) image data 24T1 of the boy who becomes the reaper is registered in advance in the transformation role data memory 24A, and the transformation image data memory 24A detects the transformation from the captured image Gn. For the person image P of the boy, a composite image GGn is generated by replacing the transformed image of the god of death (1) image data 24Gw1 stored in the transformation character data memory 24A. For the other characters, such as the beautiful girl warriors A and B, the gesture (motion) of the person image extracted from the photographed image Gn is stored in the transformation data memory 14 without previously registering the person to be transformed. When the motion model data 14P1a to 14P1c or 14P2a (+ D) to 14P2c (+ D) of the sailor warrior A or B match, the transformation image data 14Gu1 or 14Gu2 of the sailor warrior A or B is replaced. An image GGn was generated.

  On the other hand, as described in the composite image output apparatus of the second embodiment, the transformation role data memory 24A (see FIG. 24) includes the Shinigami role, the beautiful girl A role, and the beautiful girl B role, The transformation target (face) image data 24T1, 24T2, and 24T3 are registered in advance, and only the registered person images (P, J, and K) are replaced with the corresponding transformation image data 24Gw1, 24Gu1, and 24Gu2, respectively. Then, special effect (gesture) motions 23P3a to 23P3c, 23P1a to 23P1c, and 23P2a to 23P2c (see FIG. 26) are detected by the registered person images (P, J, K) and set as the effect targets. The person image (P, J, K) of the other party is replaced with the transformation image data 24Gu1 ′ of the girl warrior A with the mask Gk (see FIG. 33), or from the girl warrior A with the mask Gk It is replaced with the transformation image data 24Gu1 ′ of the sailor A (see FIG. 37), or it is replaced with the transformation image data 24Gw1 ′ from the god of death to the transformation image data 24Gw1 ′ (see FIG. 37). It is also possible to adopt a configuration in which only each person can transform the opponent by the predetermined gesture (motion).

(Second Embodiment)
FIG. 23 is a conceptual diagram illustrating a moving image shooting state and a combined image generation state using the digital camera 10 according to the second embodiment of the combined image output apparatus of the present invention.

  Also in the digital camera 10 of the second embodiment, when a person (P, J, K) who plays each role is photographed as in the first embodiment, as shown in FIG. The person images P, J, and K included in the image G are extracted, and the characters (transformation image data 24Gw1, 24Gu1 ′ (+ Gk), 24Gu2 ′) to which the extracted person images P, J, and K are to be transformed are extracted. ) Is added to the transformation mark data (24Mw1, 24Mu1 ′ (+ Mk), 24Mu2 ′) indicating that it is to be replaced, and is displayed on the display unit 11 in real time.

  Then, according to the transformation mark data (24Mw1, 24Mu1 ′ (+ Mk), 24Mu2 ′) added to the person images P, J, K of the marked photographed image Gm, as shown in FIG. A composite image GG in which the person images P, J, and K are replaced with the corresponding transformation image data (24Gw1, 24Gu1 ′ (+ Gk), 24Gu2 ′) is generated.

  Along with this, for example, the motion 23P2 of the special effect (cure effect) coincides with the gesture of the person image K replaced with the transformation image data 24Gu2 ′ of the sailor B, as shown in FIGS. Then, a marked photographic image Gm to which the special effect (cure effect) mark data 23Mu2 is added is generated and displayed, and the special effect (cure effect) image data 23Gu2 is displayed corresponding to the mark data 23Mu2. The replaced composite image GG is generated.

  Then, as shown in FIGS. 5B and 5C, the mask image Gk of the masked girl warrior A set as the effect target of the special effect (cure effect) is deleted and the original girl warrior A is deleted. The transformed image data 24Gu1 ′ is replaced with the transformed image GG, in which the transformed image of the god of death (1) is replaced with the transformed image data 24Gw1 ′ of the prince of stars (2).

  Here, in the composite image output apparatus according to the second embodiment, the person images P, J, and K that are transformation targets that are the role of Shinigami, the role of beautiful girl (Pretty Warrior) A, and the role of beautiful girl (Pretty Warrior) B. Since each face image is registered in advance and extracted from the photographed image G, only each person registered in advance can perform image composition in which the other party is transformed by the predetermined gesture (motion). it can.

  The configuration of the electronic circuit of the digital camera 10 and the server device 30 of the second embodiment is the same as that of the first embodiment described with reference to FIGS. 2 and 7, and will be described here. Is omitted.

  FIG. 24 is a view showing the contents of the transformation character data stored in the transformation character data memory 24A of the second embodiment of the digital camera 10. As shown in FIG.

  The transformation role data memory 24A of the second embodiment includes transformation object image data 24T registered as a user, transformation (1) image data 24G and transformation (1) mark data 24M, transformation (2) image data 24G ′ and transformation ( 2) It has a storage area for mark data 24M ′ and transformation condition data 24H. (1) In the storage area for image data 24G, there are a plurality of types of first stage transformation image data (transformation characters) and their voice data. Face synthesis position data is stored.

  In the storage area of the transformation target (face) image data 24T, a facial image of a person to be transformed into a plurality of types of transformation image data 24G for each role and replaced is, for example, registered A; a facial image 24T1 of a boy P , Registration B; older sister J face image 24T2, registration C; younger sister K face image 24T3, registered in advance and stored.

  Accordingly, in the storage area of the transformation (1) image data 24G corresponding to the image of the boy P of registration A, the role-reaper transformation (1) image data 24Gw1 and its voice data “Zaku, Zaku” 24Sw1 In the storage area of the transformation (1) mark data 24M, transformation (1) mark data 24Mw1 that simply represents the transformation (1) image data 24Gw1 is stored.

  Further, in the storage area of the transformation (1) image data 24G corresponding to the image of the sister J of registration B, the transformation of the role-beautiful girl A (1) image data 24Gu1 and its audio data “remo, remo,. 24Su1 and face synthesis position data 24F1 are stored, and in the storage area of the transformation (1) mark data 24M, transformation (1) mark data 24Mu1 that simply represents the transformation (1) image data 24Gu1 is stored.

  Further, in the storage area of the transformation (1) image data 24G corresponding to the image of the sister C of the registration C, the transformation of the role-beautiful girl B (1) the image data 24Gu2 and its audio data “kyu, kyu,. 24Su2 and face synthesis position data 24F2 are stored, and in the storage area of the transformation (1) mark data 24M, transformation (1) mark data 24Mu2 that simply represents the transformation (1) image data 24Gu2 is stored.

  Further, in the storage area for the transformation (2) image data 24G ′, the second transformation (2) image data 24G ′ and its audio data 24S ′ corresponding to the first transformation (1) image data 24G. Face synthesis position data is stored. Further, in the storage area for the transformation (2) mark data 24M ′, transformation (2) mark data 24M ′ for simply expressing the transformation (2) image data 24G ′ is stored.

  Thus, in the storage area of the transformation (2) image data 24G ′ corresponding to the image of the boy P of registration A, the transformation (2) image data 24Gw1 ′ in which the role of the god of death has returned to the original prince of stars The voice data “shiny, shimmery” 24Sw1 ′ and face synthesis position data 24F ′ are stored, and in the storage area of the transformation (2) mark data 24M ′, the transformation (2) transformation that simply represents the image data 24Gw1 ′ ( 2) Mark data 24Mw1 ′ is stored.

  Further, in the storage area of the transformation (2) image data 24G ′ corresponding to the image of the sister J of registration B, the transformation of the beautiful girl A further transformed into the beautiful girl warrior (2) image data 24Gu1 ′ and The voice data “lemon, lemon,...” 24Su1 ′ and face synthesis position data 24F1 ′ are stored, and the transformation (2) image data 24Gu1 ′ is simply expressed in the storage area of the transformation (2) mark data 24M ′. Transformation (2) Mark data 24Mu1 'is stored.

  Further, in the storage area of the transformation (2) image data 24G ′ corresponding to the image of the sister C of registration C, the transformation that the beautiful girl B is further transformed into the beautiful girl warrior B (2) the image data 24Gu2 ′ and The voice data “cute, cute,...” 24Su2 ′ and face synthesis position data 24F2 ′ are stored, and the transformation (2) image data 24Gu2 ′ is simply expressed in the storage area of the transformation (2) mark data 24M ′. Transformation (2) Mark data 24Mu2 'is stored.

  In the storage area for the transformation condition data 24H, condition data 24H for transformation from the first stage transformation (1) image data 24G to the second stage transformation (2) image data 24G ′ is stored.

  Specifically, the first stage (Grim Reaper) transformation (1) image data 24Gw1 corresponding to the image of boy P in Registration A (2) Transformation of the second stage (Star Prince) (2) Transformation from image data 24Gw1 ′ As the condition data 24H1, “Cure effect of the beautiful girl warrior B” is set, and the transformation from the first stage (beautiful girl A) corresponding to the image of the older sister J of registration B (1) The second stage (beautiful girl from the image data 24Gu1 Transformation of Warrior A) (2) “Person's transformation motion” is set as transformation condition data 24H2 to image data 24Gu1 ′, and transformation of the first stage (Beautiful girl B) corresponding to the image of sister C of registration C ( 1) Transformation from the image data 24Gu2 to the second stage (Pretty Warrior B) (2) “Transformation motion of the person” is set as transformation condition data 24H3 from the image data 24Gu2 ′.

  FIG. 25 is a diagram showing the contents of transformation data stored in advance in the transformation data memory 14 of the second embodiment of the digital camera 10.

  The transformation data memory 14 has storage areas for transformation target image data 24T, transformation motion model data 14P, transformation image data 24G ′, and transformation mark data 24M ′ registered as a user.

  In the storage area of the transformation image data 24G ′ and the transformation mark data 24M ′, the second stage in which the transformation condition 24H in the transformation role data memory 24A (see FIG. 24) is “personal transformation motion” 24H2 and 24H3. Transformation (2) Image data 24G 'and transformation (2) Image data 24Gu1' of Sailor A corresponding to mark data 24M ', voice data "Lemon ♪ Lemon ♪" 24Su1', Face synthesis position data 24F1 ', transformation mark data 24Mu1', transformation image data 24Gu2 'of the beautiful girl warrior B, voice data "Cute ♪ Cute ♪" 24Su2', face synthesis position data 24F2 ', transformation mark data 24Mu2' are stored.

  In the storage area of the transformation motion model data 14P, transformation motion model data 14P1a to 14P1c comprising a series of motions (gestures) for transformation into the two types of second-stage transformation image data 24Gu1 ′ and 24Gu2 ′, respectively. (Motion that raises both hands while rotating), 14P2a to 14P2c (motion that raises the right hand while rotating) are stored.

  Then, in the storage area of the transformation target image data 24T, the face image data 24T2 of the sister J who is a beautiful girl A role registered in advance in the transformation role data memory 24A remains as it is, and the second stage transformation image data ( Pretty Warrior A) Stored in association with transformation motion model data 14P1a to 14P1c for transforming into 24Gu1 ′, and the face image data 24T3 of the sister K who is a pre-registered beautiful girl B role is stored as is. It is stored in association with the transformation motion model data 14P2a to 14P2c for transformation into the second stage transformation image data (Pretty Warrior B) 24Gu2 ′.

  FIG. 26 is a diagram showing the contents of special effect data stored in advance in the special effect data memory 23A of the second embodiment of the digital camera 10. As shown in FIG.

  Similarly to the first embodiment, the special effect data memory 23A of the second embodiment corresponds to each of the three types of special effect motion model data 23P1a to 23P1c, 23P2a to 23P2c, and 23P3a to 23P3c. Special effect image data 23Gu1, 23Gu2, 23Gu3, audio data “Cure Heart” 23Su1, “Cure Star” 23Su2, “Death Moon” 23Su3, effect type “Cure Effect” 23K1, “Cure Effect” 23K2, “Death Effect” 23K3, The target of effect “partner to which special effect motion is directed” 23T1, “all neighboring enemies” 23T2, “all neighboring enemies” 23T3, and special effect mark data 23Mu1, 23Mu2, 23Mu3 are stored.

  In the storage area of the transformation target (face) image data 24T in the special effect data memory 23A, the beautiful girl warrior A stored as the second stage transformation (2) image data 24Gu1 ′ in the transformation role data memory 24A. Is associated with the special effect motion model data 23P1a to 23P1c, and the image data 24T2 of the older sister J is registered, and the second stage transformation (2) the beautiful girl warrior B stored as the image data 24Gu2 ′ can be performed. Special effect motion model data 23P3a capable of performing a reaper stored as face image data 24T3 of the first stage, and being stored as the first stage transformation (1) image data 24Gw1 in association with the special effect motion model data 23P2a to 23P2c ˜23P3c in association with the face image data 24 of boy P 1 is registered.

  FIG. 27 is a diagram showing the contents of the defense effect data stored in advance in the defense effect data memory 23B of the second embodiment of the digital camera 10. As shown in FIG.

  The defense effect data memory 23B of the second embodiment corresponds to each of two types of defense effect motion model data 23'P1a to 23'P1c and 23'P2a to 23'P2c, as in the first embodiment. Then, the defense effect image data 23'Gu1, 23'Gu2, voice data "backing" 23'Su1, "Parn" 23'Su2, and defense effect mark data 23'Mu1, 23'Mu2 are stored.

  In the storage area of the transformation object (face) image data 24T in the defense effect data memory 23B, the beautiful girl warrior B stored as the second stage transformation (2) image data 24Gu2 'in the transformation role data memory 24A. Sister K's face image data 24T3 is registered in association with the defense effect motion model data 23'P1a to 23'P1c, and the first stage transformation (1) Reaper stored as image data 24Gw1 can be performed The face image data 24T1 of the boy P is registered in association with the defense effect motion model data 23'P2a to 23'P2c.

  In the composite image output apparatus of the second embodiment, the transformation motion model data 14P shown in FIG. 25, the special effect motion model data 23P shown in FIG. 26, and the defense effect motion model data shown in FIG. In 23′P, for each individual motion model data, a specific person (face) image 24T that is a detection target of the motion (gesture) is registered in advance, so that the same in the composite image output apparatus of the first embodiment. Each motion model data (transformation; 14P2aD to 14P2cD) including a pen D and a rod B for identifying and detecting a certain person's motion (gesture) required to the motion model data to some extent (special effects; 23P2aD-23P2cD, 23P3aB-23P3cB) (protective effect; 23 P1aD~23'P1cD, 23'P2aB~23'P2cB) do not require.

  Therefore, according to the composite image output device of the second embodiment, no props are required for setting each role to a specific person, such as the Shinigami role, the Pretty Warrior A role, the Pretty Warrior B role, etc. .

  Next, the operation of the composite image output apparatus by the digital camera 10 or the server apparatus 30 according to the second embodiment having the above-described configuration will be described.

  In the composite image output apparatus according to the second embodiment, a flowchart (see FIG. 8) showing the overall process of the composite image output process, and a person (P, J, K) A flowchart of the target image extraction process for extracting the target image (see FIG. 9), and the transformation mark (24M / 24M ′) is added to the person image (P, J, K) whose role is registered in the synthesized image output process. A flowchart (see FIG. 10) showing a role transformation mark process to be added, and a person image (P, J, K) to which a transformation mark (24M / 24M ′) has been added in accordance with the composite image output process is transformed into a transformed image (24G / 24G). ') And a flowchart (refer to FIG. 13) showing a transformation mark synthesis process for setting the output of the corresponding voice (24S / 24S'), and accompanying the synthesized image output process. A special / defense effect image (23Gu / 23'Gu) is additionally synthesized at a predetermined position of the person image (P, J, K) to which the special / defense effect mark (23Mu / 23'Mu) is added and the corresponding voice (23Su The flowchart (see FIG. 14) showing the special effect mark composition processing for setting the output of / 23′Su) is omitted because it is the same as the composite image output processing of the first embodiment. To do.

  In FIG. 28, a transformation mark (24M ′) is added to the person image (J, K) in which the transformation motion (14P) is detected in accordance with the composite image output processing of the second embodiment, and the corresponding voice (24S ′) is added. It is a flowchart which shows the motion transformation mark process to output.

  FIG. 29 shows a special / defense effect mark (23Mu /) on a person image (P, J, K) in which a special / defense effect motion (23P / 23'P) has been detected in the composite image output process of the second embodiment. 23 'is a flowchart showing a special effect mark process for adding 23'Mu) and outputting corresponding sound (23Su / 23'Su).

  30 to 37 sequentially show image processing states based on the moving image captured image (G) accompanying the composite image output processing of the second embodiment as representative frame advance (No. 1) to (No. 8). FIG. 4A is a diagram illustrating a captured image G, FIG. 4B is a diagram illustrating a marked captured image Gm, and FIG. 3C is a diagram illustrating a composite image GG.

  The process of steps S1 to S3 of the composite image output process in FIG. 8 is similar to that in the first embodiment, as shown in FIGS. 30 (A) to 37 (A). It is repeatedly executed as the captured image (G) is acquired, and a marked captured image (Gm) as shown in FIGS. 30B to 37B is generated and output.

  Then, the process of steps S4 to S7 of the composite image output process in FIG. 8 is a series of marks generated as shown in FIGS. 30B to 37B, as in the first embodiment. This is executed based on the attached photographed image (Gm), and a composite image (GG) as shown in FIGS. 30C to 37C is generated and output.

  In the second embodiment, the case where the composite image output process in FIG. 8 is executed in the digital camera 10 (see FIG. 2) will be mainly described, and the same composite image in the server device 30 (see FIG. 7). When the output process is executed, the process contents are substantially the same, and the description thereof is omitted.

  When a series of scenes played by a family is photographed by the image photographing unit 21 of the digital camera 10 as shown in FIGS. 30A to 37A, for example, the photographed images G1 to G8 are sequentially captured image data. It is stored and stored in the memory 25 (step S1), and the process proceeds to the target image extraction process in FIG. 9 in real time (step SA).

  When the person image (P, J, K) existing in the captured image data Gn is extracted by the target image extraction process (step SA), the process proceeds to the role transformation mark process in FIG. 10 (step SB). .

  In this role transformation mark processing, the person image (P, J, K) in the photographed image data Gn extracted by the target image extraction processing is registered as a user in the transformation role data memory 24A (see FIG. 24). Whether or not the person image (P) (J) (K) as the transformation target (face) image data 24T is included is identified using the face image recognition process (step B1).

  Here, as shown in the photographed image data G1 to G8 of FIGS. 30A to 37A, respectively, a person image having a face image 24T1 of a boy P as a transformation target (face) image data 24T, or an older sister. When it is identified that a person image having the face image 24T2 of J or a person image having the face image 24T3 of the sister K is included (step B1 (Yes)), each correspondence in the transformation state memory 24B for each role The transformation status flag is detected, and the role status of registration A corresponding to the current boy image P of the boy, the status status of registration B corresponding to the current portrait image J of the older sister, or the current status image K of the younger sister The role status of the corresponding registered C is detected (step B2).

When it is determined that the state is not the state of transformation (2) (the state of transformation (1)) (step B2 (No)), in a series of captured images including the corresponding transformation target images, Special effect motion 23P (death effect motions 23P3a to 23P3c in the case of a boy's portrait image P, cure effects motions 23P1a to 23P1c in the case of a sister's portrait image J, and a cure effect in the case of a sister's portrait image K It is determined whether or not motions 23P2a to 23P2c) have been detected (step B6). (This special effect will be described in detail in the special effect mark process (step SE) described later.)
For example, in the case of a boy's person image P included in the photographed image G1 shown in FIG. 30A, the special effect motion 23P is not detected (step B6 (No)), so the transformation character data memory 24A (see FIG. 24). ) And the metamorphosis (1) metamorphosis (1) mark data 24Mw1 of the metamorphosis (1) stored in association with () is read out, and the face image of the identified boy person image P is recognized. In addition, the recorded image data Gm1 to Gm7 with marks as shown in FIGS. 30B to 36B are temporarily stored in the work data memory and displayed on the display unit 11 (step B8).

  Further, for example, in the case of the sister's person image J included in the photographed image G1 shown in FIG. 30A, the special effect motion 23P is not detected (step B6 (No)), so the transformation character data memory 24A (FIG. 24) and the transformation (1) transformation data (1) mark data 24Mu1 for the beautiful girl A stored in association with the information is read out and added in correspondence with the position of the identified older person image J. Then, it is temporarily stored in the work data memory as marked image data Gm1 as shown in FIG. 30B and displayed on the display unit 11 (step B8).

  Also, for example, in the case of the sister's person image K included in the photographed image G5 shown in FIG. 34A, the special effect motion 23P is not detected (step B6 (No)), so the transformation character data memory 24A (FIG. 24), the transformation (1) mark data 24Mu2 for the beautiful girl B of the transformation (1) stored in association with the information is read out and added in correspondence with the position of the identified younger person image K. Then, the image data Gm5 with a mark as shown in FIG. 34B is temporarily stored in the work data memory and displayed on the display unit 11 (step B8).

  The transformation (1) mark data 24Mw1, 24Mu1, and 24Mu2 added corresponding to the positions of the registered person images P, J, and K are added to the person images P, J, and K, respectively. Alternatively, it may be added in the vicinity of the person images P, J, and K.

  If it is determined that the digital camera 10 is currently shooting (step B4 (Yes)), for example, as shown in FIGS. 30 (B) to 36 (B), a Shinigami transformation (1) mark When the data 24Mw1 is added, the audio data “Zaku, Zaku” 24Sw1 stored in association with the transformation character data memory 24A (see FIG. 24) is read and output from the audio output unit 13. (Step B5).

  Similarly, for example, as shown in FIG. 30B, when the transformation (1) mark data 24Mu1 for the beautiful girl A is added, it is associated with the transformation role data memory 24A (see FIG. 24). The stored audio data “Remo, Remo” 24Su1 is read out and output from the audio output unit 13 (step B5).

  Similarly, when the transformation (1) mark data 24Mu2 for the beautiful girl B is added as shown in FIG. 34 (B), it is stored in association with the transformation role data memory 24A (see FIG. 24). The voice data “Kyu, Kyu” 24Su2 being read is read and output from the voice output unit 13 (step B5).

  Similarly, when the transformation state of registration A corresponding to the current image P of the boy's boy P is detected, it is determined that the state is not the state of transformation (2) (the state of transformation (1)). (Step B2 (No)), as shown in FIGS. 36 and 37, for example, in the series of captured images including the transformation target image, the special effect motions 23P2a to 23P2c for the sister image K registered in the face image. Is detected (step B6 (Yes)), the role status flag of the boy P in the role-specific change status memory 24B is changed to the status of change (2) (step B7). A transformation (2) mark 24Mw1 ′ is added to the position corresponding to the target image (P), and is temporarily stored in the work data memory as marked image data Gm8 as shown in FIG. Both are displayed on the display unit 11 (step B3).

  If it is determined that the digital camera 10 is currently photographing (step B4 (Yes)), the transformation (2) stored in association with the transformation role data memory 24A (see FIG. 24). Voice data “shiny, shining” 24Sw1 ′ is read out and output from the voice output unit 13 as shown in FIG. 37B (step B5).

  On the other hand, if the transformation state of registration A corresponding to the current boy's person image P is detected and it is determined that the state is transformation (2) (step B2 (Yes)), the transformation target image is handled. A transformation (2) mark 24Mw1 'is added to the position, and is temporarily stored in the working data memory as marked photographed image data Gm8 as shown in FIG. 37B and displayed on the display unit 11 (step B3). .

  If it is determined that the digital camera 10 is currently shooting (step B4 (Yes)), the voice data “Kiriari” stored in association with the transformation character data memory 24A (see FIG. 24) is stored. , “Sparkle” 24Sw1 ′ is read out and output from the audio output unit 13 as shown in FIG. 37B (step B5).

  If the transformation state of registration B corresponding to the current sister's person image J is detected and it is determined that the state is transformation (2) (step B2 (Yes)), it corresponds to the transformation target image. The transformation (2) mark 24Mu1 ′ is added to the position, and is temporarily stored in the work data memory as marked image data Gm2 to Gm8 as shown in FIGS. 31 (B) to 37 (B) and the display unit 11. Is displayed (step B3).

  If it is determined that the digital camera 10 is currently photographing (step B4 (Yes)), the voice data “Lemon” stored in association with the transformation character data memory 24A (see FIG. 24) is stored. , Lemon ”24Su1 ′ is read out and output from the audio output unit 13 as shown in FIGS. 31B to 37B (step B5).

  Further, when the transformation state of registration C corresponding to the current younger sister's person image K is detected and it is determined that it is in the state of transformation (2) (step B2 (Yes)), it corresponds to the transformation target image. The transformation (2) mark 24Mu2 'is added to the position, and is temporarily stored in the work data memory as marked image data Gm6 to Gm8 as shown in FIGS. Is displayed (step B3).

  If it is determined that the digital camera 10 is currently photographing (step B4 (Yes)), the voice data “cute” stored in association with the transformation character data memory 24A (see FIG. 24) is stored. , “Cute” 24Su2 ′ is read out and output from the audio output unit 13 as shown in FIGS. 35 (B) to 37 (B) (step B5).

  Then, the process proceeds to the motion transformation mark process in FIG. 28 (step SD ′).

  In this motion transformation mark processing, a person image (P, J, K) in a series of photographed image data Gn extracted by the target image extraction processing is registered in advance in the transformation data memory 14 (see FIG. 25). An image that matches the series of transformation motion model data 14P1a to 14P1c of the beautiful girl warrior A by the portrait image J of the older sister's face and the series of transformation model data 14P2a to 14P2c of the beautiful girl warrior B by the portrait image K of the sister's face It is determined whether or not a portion has been detected (step D1 ′).

  That is, as shown in the photographed image data G2 of FIG. 31 (A), the transformation data memory 14 (see FIG. 25) for a series of person images J of the sister registered in the face image extracted by the target image extraction process. Is detected as an image portion whose gesture matches the series of transformation motion model data 14P1a to 14P1c of the beautiful girl warrior A (step D1 ′ (Yes)), the transformation data memory 14 (see FIG. 25). ) Is stored in correspondence with the position of the face image of the sister image J detected by the motion detection, and is added in correspondence with the position detected by the motion image. The photographed image data Gm2 with a mark as shown in FIG. 31 (B) is temporarily stored in the work data memory and displayed on the display unit 11 (step D2). On the other hand, if an image portion whose gesture matches the transformation motion model data 14P is not detected (step D1 (No)), this process is terminated, and the process proceeds to the special effect mark process (step SE ′) in FIG.

  If it is determined that the digital camera 10 is currently photographing (step D3 (Yes)), the beautiful girl warrior A stored in association with the transformation data memory 14 (see FIG. 25) is stored. The audio data “Lemon ♪ Lemon ♪” 24Su1 ′ is read and output from the audio output unit 13 as shown in FIG. 31 (B) (step D4).

  Further, as shown in the photographed image data G6 in FIG. 35A, the transformation data memory 14 (see FIG. 25) for a series of younger sister images K registered as face images extracted by the target image extraction process. When it is determined that the image portion is identical in gesture with the series of transformation motion model data 14P2a to 14P2c of the beautiful girl warrior B stored in (step D1 (Yes)), the transformation data memory 14 (see FIG. 25). ) Is stored in correspondence with the position of the face image of the younger sister's person image K whose motion has been detected, and is added to the figure. The photographed image data Gm6 to Gm8 with marks as shown in FIG. 35 (B) to FIG. Shown (step D2).

  If it is determined that the digital camera 10 is currently photographing (step D3 (Yes)), the beautiful girl warrior B stored in association with the transformation data memory 14 (see FIG. 25) is stored. The audio data “Cute ♪ Cute ♪” 24Su2 ′ is read and output from the audio output unit 13 as shown in FIGS. 35B to 37B (step D4).

  Then, the process proceeds to the special effect mark process in FIG. 29 (step SE ′).

  In this special effect mark process, a face image (P, J, K) in a series of photographed image data Gn extracted by the target image extraction process is previously stored in the special effect data memory 23A (see FIG. 26). Match with a series of transformation motion model data 23P1a to 23P1c of special effect A and a series of transformation motion model data 23P2a to 23P2c of special effect B and a series of transformation motion model data 23P3a to 23P3c of special effect C based on the registered person image It is determined whether an image portion to be detected has been detected (step E1 ').

  That is, as shown in the photographed image data G3 in FIG. 32A, the special effect data memory 23A (see FIG. 26) for a series of person images J of the sister registered in the face image extracted by the target image extraction process. ) Is stored in advance as a series of transformation motion model data 23P1a to 23P1c of special effect A (step E1 ′ (Yes)), the special effect data memory 23A (FIG. 26), the special effect mark data 23Mu1 of the special effect A stored in association with the image of the special effect A is read, and the effect target 23T1 (special Of the person image (boy) P to which the transformation mark data 24Mw1 of the reaper is the opponent Is added toward the direction, is displayed on the display unit 11 is temporarily stored in a working data memory as marked photographic image data Gm3 as shown in FIG. 32 (B) (step E2).

  If it is determined that the digital camera 10 is currently photographing (step E3 (Yes)), the special effect A stored in association with the special effect data memory 23A (see FIG. 26) is stored. The audio data “Cure Heart” 23Su1 is read out and output from the audio output unit 13 as shown in FIG. 32B (step E4). In this way, the audio data “Cure Heart” 23Su1 of the special effect A is output from the digital camera 10 so that the special effect processing is appropriately executed by the computer (actor (sister J, boy P)). It can be easily recognized and can perform with a realistic feeling.

  Whether the special effect mark data 23Mu1 added corresponding to a predetermined position of the person image J whose motion is detected is added to the person image J or added in the vicinity of the person image J. Either may be sufficient. As a result, the mother (J), who has registered the face image in advance, has performed special effect motion, so that the special effect processing is appropriately executed by the computer, and a composite image with the special effect image added is created. Can be easily recognized.

  Further, as shown in the photographed image data G4 of FIG. 33A, the special effect data memory 23A (see FIG. 26) is used for a series of person images P of the boy registered in the face image extracted by the target image extraction process. ) Is stored in advance as a series of transformation motion model data 23P3a to 23P3c of special effect C (step E1 ′ (Yes)), it corresponds to the special effect data memory 23A. The special effect mark data 23Mu3 of the special effect C that is attached and stored is read out, and the effect target 23T3 (all surrounding enemies) corresponding to a predetermined position of the boy image P of the motion detected. Added to the direction of the person image (sister) J to which the transformation mark data 24Mu1 ′ of the certain sailor A is added Which is displayed on the display unit 11 is temporarily stored in a working data memory as marked photographic image data Gm4 as shown in FIG. 33 (B) (step E2).

  If it is determined that the digital camera 10 is currently photographing (step E3 (Yes)), the special effect C stored in association with the special effect data memory 23A (see FIG. 26) is stored. The audio data “Death Moon” 23Su3 is read out and output from the audio output unit 13 as shown in FIG. 33B (step E4). As described above, the actor (sister J, boy P) indicates that the special effect processing is appropriately executed by the computer by outputting the audio data “Death Moon” 23Su3 of the special effect C from the digital camera 10 in this way. Can be easily recognized.

  Further, as shown in the photographed image data G7 in FIG. 36A, a series of person images K of the sister registered in the face image extracted by the target image extraction process are stored in advance in the special effect data memory 23A. When it is detected that the image portion matches the gesture with the special effect motion model data 23P2a to 23P2c of the special effect B (step E1 ′ (Yes)), the special effect data memory 23A (see FIG. 26) The special effect mark data 23Mu2 of the special effect B stored in association with each other is read out, and corresponds to a predetermined position (for example, a third position from the upper part of the image) of the sister's person image K whose motion is detected. With the mask mark data Mk that is the target 23T2 of the effect (all nearby enemies) (disturbed state) Is added in the direction of the person image (boy) P to which the transformation mark data 24Mw1 of the god of death is added, and is temporarily stored in the work data memory as marked image data Gm7 as shown in FIG. It is displayed on the display unit 11 (step E2).

  If it is determined that the digital camera 10 is currently photographing (step E3 (Yes)), the special effect B stored in association with the special effect data memory 23A (see FIG. 26) is stored. The audio data “CURE STAR” 23Su2 is read out and output from the audio output unit 13 as shown in FIG. 36B (step E4). In this way, the audio data “CURE STAR” 23Su2 of the special effect B is output from the digital camera 10, so that the special effect processing is appropriately executed by the computer (sister K, sister J, It can be easily recognized by boy P).

  Here, in the special effect (see FIG. 26), as in the first embodiment, a cure (healing) effect 23K1, 23K2 for returning the partner who is the target 23T of the effect from the disturbance state to the normal state, and the normal state There is a death (death reaper) effect 23K3 that changes the state to a disturbing state, while the defense effect (see FIG. 27) also invalidates the special effect received from the opponent as in the first embodiment. The transformation process (steps E5 to E23) associated with the special effect and the defense effect will be described below.

  That is, as shown by the marked photographed image data Gm3 in FIG. 32B, the special effect A is determined from the position of the sister (J) registered with the face image to which the transformation mark data 24Mu1 ′ of the sailor warrior A is added. When special effect mark data 23Mu1 associated with (cure effect 23K1) is displayed and output (steps E1 ', E2), the target of the effect of special effect A stored in advance in the special effect data memory 23A (see FIG. 26) If it is determined that 23T is the opponent 23T1 to which the special effect motion that is not all of the surrounding enemies is directed (step E5 (No)), the opponent to which the special effect motion 23P1 is directed in the screen (Gm3) A person image P) is detected and set as an effect target (step E12). Since the person image of the other party indicated by the gesture (motion) of the person image registered in the face image is detected and set as the subject of the special effect in this way, the registered actor who performs the special effect as desired In addition, the special effect can be produced as appropriate, and a composite image in which the other party is transformed can be easily created.

  If it is determined that the effect type 23K of the special effect A is the cure effect 23K1 (step E7 (Yes)), the target of the detected and set effect is a portrait image of a sailor with a mask mark Mk. It is determined that it is not (step E13 (No)), and is determined to be a person image P with the reaper mark 24Mw1 (step E15 (Yes)).

  Then, it is detected that the person image P of the boy (reaper role) registered as the face image to be the effect is an image portion whose gesture coincides with the defense effect motion model data 23′P2a to 23′P2c in FIG. Then (step E16 ′ (Yes)), next to the corresponding heart-shaped special effect mark data 23Mu1, the defensive effect mark data 23′Mu2 having a broken heart indicating that the special effect has been invalidated is displayed. It is read and added (step E17). In this case, the transformation effect of the god of death is continued because the defense effect of the boy (P) of the god of death registered in the face image is exerted.

  Then, the audio data “Parn” 23′Su2 corresponding to the defense effect mark data 23′Mu2 is read and output from the audio output unit 13 (step E22).

  As described above, the defense effect mark 23′Mu2 indicating that the special effect is invalidated is displayed as the marked photographed image Gm3, so that the mother who is photographing while viewing the camera screen (11) It is possible to visually confirm that the effect is properly produced and the special effect is invalidated. In addition, the voice data “Pan” 23′Su2 corresponding to the defense effect mark 23′Mu2 is output, so that not only the mother but also all the performers can properly achieve the defense effect and invalidate the special effect. Can be confirmed audibly.

  Further, as shown by the marked photographed image data Gm4 in FIG. 33 (B), the special effect C (death effect 23K3) starts from the position of the boy (P) whose face image is registered to which the death mark transformation mark data 24Mw1 is added. ) Special effect mark data 23Mu3 is displayed and output (steps E1 ′, E2), the special effect C effect target 23T stored in advance in the special effect data memory 23A (see FIG. 26) is If it is determined that all the enemies are 23T3 (step E5 (Yes)), the enemy opponent (person image J who is the sister of the role of the beautiful girl warrior A) is detected in the screen (Gm4), and the target of the effect (Step E6).

  Then, when it is determined that the effect type 23K of the special effect C is the death effect 23K3 (step E7 (No)), the target of the detected and set effect is attached with the transformation mark data 24Mu1 'of the sailor warrior. (Step E8 (Yes)).

  Here, when the defense effect motion (see FIG. 27) of the defense effect A is not detected for the person image J of the sister (role of the girl warrior A) whose face image is registered as the target of the effect (step E9 ′ ( No)), next to the transformation mark data 24Mu1 'corresponding to the beautiful girl warrior A, mask mark data Mk indicating that the beautiful girl warrior A has been transformed into a disturbing state is added (step E10).

  In other words, in this case, the image J of the sister of the role of the beautiful girl warrior A is a death (death reaper) corresponding to the motion 23P3a to 23P3c of the special effect C by the character image P of the boy of the reaper registered in advance in the transformation role data memory 24A. ) By the effect, it has been transformed into a distressed masked girl warrior.

  Then, audio data “Cook Cook” Sk corresponding to the mask mark data Mk is output from the audio output unit 13 (step E19).

  In step E9 ′, the motion model data 23′P1a to 23′P1c of the defensive effect A is applied to the person image J of the sister (role of the girl warrior A) whose face image is the target 23T3 of the death effect 23K. When it is detected that the image part matches the gesture (see FIG. 27) (step E9 ′ (Yes)), the special effect is invalidated next to the corresponding moon-shaped special effect mark data 23Mu3. The defensive effect mark data 23′Mu1 in the form of a cracked moon is read and added (step E11). In this case, the transformation effect of the sailor warrior A will be continued because the defense effect is produced from the sister J registered as the face image of the sailor warrior A.

  Then, the sound data “backing” 23′Su1 corresponding to the defense effect mark data 23′Mu1 is read and output from the sound output unit 13 (step E20).

  Also, as shown by the marked image data Gm7 in FIG. 36 (B), the special effect B is determined from the position of the sister (K) registered with the face image to which the transformation mark data 24Mu2 'of the sailor warrior B is added. When special effect mark data 23Mu2 associated with (cure effect 23K2) is displayed and output (steps E1 ', E2), the target of the effect of special effect B stored in advance in the special effect data memory 23A (see FIG. 26) If it is determined that 23T is all surrounding enemies 23T2 (step E5 (Yes)), transform into a warrior girl warrior A with masked Mk as an opponent in the screen (Gm7). The image of the older sister's character J and the character image P of the boy to whom the transformation mark data 24Mw1 of the reaper is added are detected and set as the target of the effect. Step E6).

  Then, when it is determined that the effect type 23K2 of the special effect B is a cure effect (step E7 (Yes)), in the case of the sister's person image J that is the target of the detected and set effect, the mask mark Mk It is judged that it is a beautiful girl warrior A (step E13 (Yes)).

  Then, the mask mark data Mk of the sailor A with the mask mark Mk is deleted, as shown by the photographed image data Gm8 with a mark in FIG. 37B (step E14).

  That is, in this case, the image J of the sister of the role of the beautiful girl warrior A is a cure (healing) corresponding to the motion 23P2a to 23P2c of the special effect B by the image of the sister image K registered in advance as the role of the beautiful girl warrior B. By the effect, the troubled girl warrior A with a mask is transformed into the original normal girl warrior A and returned.

  Then, the audio data 24Su1 ′ “lemon ♪ lemon ♪” corresponding to the transformation mark data 24Mu1 ′ of the original sailor A is read and output from the audio output unit 13 (step E21).

  On the other hand, when it is determined that the effect type 23K2 of the special effect B is a cure effect (step E7 (Yes)), in the case of the person image P of the boy of the reaper who is the object of the detected and set effect Is determined to have the reaper mark 24Mw1 (step E15 (Yes)).

  Here, when the defensive effect motion (see FIG. 27) of the defensive effect B is not detected for the person image P of the boy (reaper role) registered as the face image as the target of the effect (step E16 ′ (No)) 37B, as shown by the marked photographed image data Gm8 in FIG. 37B, the god of death transformation mark data 24Mw1 is deleted from the person image P of the boy (rage of reaper), and the transformation of the star prince (2 ) Mark data 24Mw1 'is added (step E18).

  That is, in this case, the character image P of the boy who plays the role of Shinigami is caused by the cure effect corresponding to the motions 23P2a to 23P2c of the special effect B by the sister's character image K registered in advance as the role of the beautiful girl warrior B. It was transformed from Shinigami into the original star prince and returned.

  Then, the transformation of the prince of the stars (2) The voice data 24Sw1 ′ “brilliantly” corresponding to the mark data 24Mw1 ′ is read and output from the voice output unit 13 (step E23).

  In this way, special effect mark processing (steps E1 to E23) based on detection of the special effect motion 23P and the defense effect motion 23′P from the person image (P, J, K) registered in the face image in the captured image G in advance. ) Is completed, as shown in FIGS. 30B to 37B, continuous photographed image data Gm1 with marks generated on the work data memory in accordance with the processing of steps SA to SE ′. ˜Gm8 are stored and saved in the marked image data memory 26 (step S2).

  When it is determined that the shooting input of the series of shot image data G1 to G8 from the image shooting unit 21 has been completed (step S3 (Yes)), the series of the shot images generated and stored in the marked shot image data memory 26. It is determined whether or not to proceed to the composite image generation process based on the marked image data Gm1 to Gm8 (step S4).

  Here, when it is determined that the process proceeds to a composite image generation process in response to a user operation for generating a composite image on the input unit 12 or automatically upon completion of acquisition of the series of captured image data (step S4). (Yes)), the series of marked photographed image data Gm1 to Gm8 is read from the marked photographed image data memory 26 to the working memory (step S5), and the process proceeds to the transformation mark composition process in FIG. Step SF).

  In this transformation mark composition processing, first, the marked photographed image data Gm1 to Gm8 read from the marked photographed image data memory 26 to the working memory is stored in advance in the transformation character data memory 24A (see FIG. 24). The transformation (1), (2) mark data 24M, 24M ′ and the transformation data memory 14 (see FIG. 25) to which the transformation mark data 24M ′ previously stored are added are included. Determination is made (step F1).

  Here, the transformation mark data 24M, 24M ′ stored in advance in the transformation role data memory 24A (see FIG. 24) and the transformation mark data 24M ′ stored in advance in the transformation data memory 14 (see FIG. 25) are stored. If it is determined that the added image is not included (step F1 (No)), the process is terminated and the process proceeds to the special effect mark composition process (step SH).

  Further, as shown in FIGS. 30B to 37B, the image of the Shinigami transformation (1) with the mark data 24Mw1 added, and the transformation back to the star prince as shown in FIG. 37B (2 ) An image to which mark data 24Mw1 ′ is added, and an image to which transformation mark data 24Mu1 of beautiful girl A is added as shown in FIG. 30 (B), as shown in FIGS. 31 (B) to 37 (B). An image with the transformation mark data 24Mu1 'of the beautiful girl warrior A added thereto, and an image with the transformation mark data 24Mu2 of the beautiful girl B added thereto as shown in FIG. 34B, FIGS. When it is determined that there is an image to which the transformation mark data 24Mu2 ′ of the sailor warrior B is added as shown in (B) (step F1 (Yes)), the transformation mark data 24Mw1, 24Mw1 ′, 24Mu. The positions of the person images P, J, K near the mark to which 1,24Mu1 ′, 24Mu2, 24Mu2 ′ are added correspondingly are obtained as additional information of the photographed image data Gn by the target image extraction process (step SA). It is specified from the position information of the current block (step F2).

  Then, the identified person images P, J, and K are subjected to motion capture processing, and the Shinigami transformation mark data 24Mw1 and the star prince transformation mark data 24Mw1 stored in the transformation role data memory 24A (see FIG. 24). The transformation (1) image data 24Gw1 and transformation (2) image data 24Gw1 'corresponding to', the transformation role data memory 24A (see FIG. 24) or the transformation data memory 14 (see FIG. 25). Transformation mark data 24Mu1, 24Mu2 of beautiful girl A, B and transformation corresponding to transformation mark data 24Mu1 ', 24Mu2' of beautiful girl warrior A, B (1) Image data 24Gu1, 24Gu2 and transformation (2) Image data 24Gu1 ′, 24Gu2 ′, as shown in FIGS. 30 (C) to 36 (C). The transformation of the god of death in the same pose (posture) as the object image (boy) P (1) Image data 24Gw1, the prince of the star in the same pose (posture) as the person image (boy) P as shown in FIG. Transformation (2) Image data 24Gw1 ′, transformation (1) Image data 24Gu1 of beautiful girl A in the same pose (posture) as the person image (sister) J as shown in FIGS. 30 (C) to 37 (C) Transformation of Sailor A (2) Image data 24Gu1 ′, transformation of Pretty B in the same pose (posture) as the person image (sister) K as shown in FIGS. 19C to 37C (1) ) The image data 24Gu2 and the transformation of the beautiful girl warrior B (2) The image data 24Gu2 'is replaced with the preset background image BG and synthesized (step F3). At this time, composite image data GG1 to GG8 are generated (step F3).

  Then, it is determined whether or not there is face synthesis position data in the transformed image data replaced corresponding to each mark data (step F4).

  In this case, the face synthesis position data corresponding to the Shinigami transformation (1) image data 24Gw1 is not stored, and the process proceeds to Step F8. Also, it is determined that there is face synthesis position data 24F ′ (see FIG. 24) corresponding to the transformation of the prince of the stars (2) image data 24Gw1 ′, and the transformation of the beautiful girl A (1) corresponding to the image data 24Gu1. Face synthesis position data 24F1 and transformation of Pretty Soldier A (2) Face synthesis position data 24F1 'corresponding to image data 24Gu1', Transformation of Pretty Girl B (1) Face synthesis position data 24F2 corresponding to image data 24Gu2 and beauty It is determined that there is also face synthesis position data 24F2 ′ (see FIGS. 24 and 25) corresponding to the transformation of the girl warrior B (2) image data 24Gu2 ′ (step F4 (Yes)).

  Then, the face synthesis position data 24F ', 24F1, 24F1', 24F2, 24F2 'are read (step F5), and the face image portions (Pf, Jf, Kf) in the target image in each corresponding photographed image Gn. Are extracted (step F6), and the extracted face image portions (Pf, Jf, Kf) are converted into face synthesis positions (24F ′, 24Gu1, 24Gu1, 24Gu1, 24Gu1, 24Gu1, 24Gu1, 24Gu1, 24Gu1 ′, 24Gu1 ′). 24F1, 24F1 ′, 24F2, 24F2) (Pf ′, Jf ′, Kf ′), and as shown in FIGS. 30C to 37C, the image data (24Gw1) of the transformation character is changed. ', 24Gu1, 24Gu1', 24Gu2, 24Gu2 ') (step F7).

  In this manner, the face image portion (Pf, Jf, Kf) extracted from the captured image data Gn is resized according to the position corresponding to the face synthesis position data, and the image data (24Gw1) of each corresponding transformation character ', 24Gu1, 24Gu1', 24Gu2, 24Gu2 '), the face part of the live action is appropriately synthesized according to the face size of the makeover character, and the makeover character can be synthesized and displayed with the face of the live action. .

  Next, in this transformation mark compositing process, it is determined whether or not the masked mark data Mk is included in the marked imaged image data Gmn to be processed (step F8).

  That is, as shown in the photographed image data Gm4 to Gm7 with marks shown in FIGS. 32 (B) to 36 (B), a mask mark is applied to the sister's person image J registered as the face image of the role of the beautiful girl warrior A. When it is determined that the data Mk is added (step F8 (Yes)), as shown in FIGS. 32 (C) to 36 (C), the face of the face image portion in the sister image person J is shown. A composite image GG is generated by combining the mask image Gk in accordance with the composite position 24F1 (step F9).

  Then, the reaper audio data “Zaku, Zaku” stored in the transformation character data memory 24A (see FIG. 24) in association with the respective transformation image data (24Gw1 ′, 24Gu1, 24Gu1 ′, 24Gu2, 24Gu2 ′). 24Sw1, the prince's voice data “Kirari, Kirari” 24Sw1 ′, the transformation character data memory 24A (see FIG. 24) or the transformation data memory 14 (see FIG. 25), the beautiful girl A, the beautiful girl warrior A Voice data “Remo, Remo” 24Su1, “Lemon ♪ Lemon ♪” 24 Su1 ′, Pretty B, Pretty Warrior B ’s voice data “Cu, Kyu” 24Su2, “Cute ♪ Cute ♪” 24Su2 ′, Stored in association with the composite images GG1 to GG8 generated as shown in FIGS. 30 (C) to 37 (C). That (step F10). Here, when the mask image data Mk is added to the figure image J of the sister of the role of the beautiful girl A or the role of the beautiful girl warrior A or the character image K of the sister of the role of the beautiful girl B or the beautiful girl warrior B The voice data “Remo, Remo” 24Su1, “Lemon ♪ Lemon ♪” 24Su1 ′, “Kyu, Kyu” 24Su2, “Cute” of the beautiful girl A, beautiful girl warrior A, beautiful girl B, beautiful girl warrior B The voice data “Cook Cook” Sk corresponding to the mask mark data Mk is stored in association with the composite images GG1 to GG8 instead of “♪ Cute ♪” 24Su2 ′. (Step F10).

  If it is determined that the masked image data Gmn to be processed does not include the masked mark data Mk (step F8 (No)), the masked image Gk combining process and voice setting process (step S8). F9, F10) are skipped.

  Then, the process proceeds to the special effect mark composition process in FIG. 14 (step SH).

  In this special effect mark composition process, first, the marked image data Gm1 to Gm8 read from the marked image data memory 26 to the working memory is stored in advance in the special effect data memory 23A (see FIG. 26). It is determined whether or not the stored special effect mark data 23Mu or the defense effect data memory 23B (see FIG. 27) includes image data Gmn to which the pre-stored defense effect mark data 23′Mu is added. (Step H1).

  Here, as shown in FIGS. 30B, 31B, 34B, and 35B, image data to which special effect mark data 23Mu and defensive effect mark data 23'Mu are added. If it is determined that Gm does not exist (step H1 (No)), the process ends, and the process proceeds to step S6.

  On the other hand, as shown in FIGS. 32B, 33B, 36B, and 37B, the special effect mark data 23Mu1 corresponding to the special effect A and the defense effect corresponding to the defense effect B If it is determined that the mark data 23′Mu2, the special effect mark data 23Mu3 corresponding to the special effect C, and the special effect mark data 23Mu2 corresponding to the special effect B are present (step H1 (Yes)), the special effect mark The position of the person image (sister, sister, boy) J, K, P in the vicinity of the mark to which the data 23Mu1, 23Mu2, 23Mu3 or the defense effect mark data 23'Mu2 is added correspondingly is the target image extraction process (step It is specified from the position information of the block obtained as additional information of the photographed image data Gn by (SA) (step H2).

  Then, the special effect mark data 23Mu1, 23Mu2, 23Mu3 in the special effect data memory 23A (see FIG. 26) or the defense effect mark data 23′Mu2 in the defense effect data memory 23B (see FIG. 27) are stored in association with each other. Special effect image data 23Gu1, 23Gu2, 23Gu3 or defense effect image data 23'Gu2 being read out, and a predetermined position (for example, 3 minutes from the upper part of the image) of each of the specified person images (J, K, P) Corresponding to the position (1)), the composite image data GG is generated by further compositing in the direction of the person image to which the transformation mark data of the enemy role (the other party) is added (step H3).

  For example, as shown in FIGS. 32 (B) and 32 (C), the special effect image is obtained from the position of the arm of the person image J registered as the face image of the girl warrior A role to which the special effect mark data 23Mu1 is added. Data 23Gu1 is additionally synthesized in the direction of the person image P of the reaper role. On the other hand, from the position of the arm of the person image P registered as the face image of the reaper role to which the defense effect mark data 23′Mu2 is added, the defense effect image data 23′Gu2 is the person image of the beautiful girl warrior A role. Additional synthesis is performed in the direction of J.

  Further, as shown in FIGS. 33B and 33C, the special effect image data 23Gu3 is obtained from the position of the arm of the person image P registered as the face image of the reaper with the special effect mark data 23Mu3 added. Additional synthesis is performed in the direction of the portrait image J of the sailor warrior A.

  Further, as shown in FIGS. 36B and 36C, the special effect image is obtained from the position of the arm of the person image K registered as the face image of the girl warrior B role to which the special effect mark data 23Mu2 is added. The data 23Gu2 is additionally synthesized in the direction of the person image J of the role of the beautiful girl warrior A with a mask in which it is disturbed (step H3).

  Further, depending on the special effect, the tip of the special effect image data 23Gu may be used as an explosion effect image and extended to a position where it reaches the transformation image data of the person image of the opponent who is the enemy character (additional composition). Step H4).

  The special effect A audio data “Cure Heart” 23Su1, the special effect B audio data “Cure Star” 23Su2, and the special effect C stored in the special effect data memory 23A (see FIG. 26) are stored. The voice data “Death Moon” 23Su3 is read out, and the voice data “Pern” 23′Su2 of the defense effect B stored in association with the defense effect data memory 23B (see FIG. 27) is read out. 32C, FIG. 33C, FIG. 36C, and FIG. 37C are stored in association with the synthesized images GG3, GG4, GG7, and GG8 generated (steps). H5).

  Then, as shown in FIGS. 30C to 37C, a series of synthesized image data GG1 to GG8 generated on the work data memory in accordance with the processing of steps SF to SH is obtained as a synthesized image data memory. 27 and stored (step S6).

  Then, a series of synthesized image data (including associated audio data) GG1 to GG8 stored in the synthesized image data memory 27 is displayed as shown in FIGS. 30 (C) to 37 (C). Reproduced and displayed by the unit 11 and reproduced and output by the audio output unit 13 (step S7). When the composite image output apparatus according to the second embodiment is configured as the server apparatus 30 (see FIG. 7), it is a transmission source of the series of captured image data G1 to G8 received in step S1. A series of synthesized image data (including associated audio data) stored in the synthesized image data memory 27 for the communication terminal A (digital camera) 10 ′ and the communication terminal B (PC) 40 on the communication network N. GG1 to GG8 are read and transmitted as a response (step S7).

  Therefore, according to the composite image output device of the second embodiment using the digital camera 10 or the server device 30 having the above-described configuration, when a series of captured image data Gn captured by the image capturing unit 21 is input, the captured image The person image (P, J, K) included in the data Gn is extracted, and for the person image P of the reaper role registered in advance in the transformation role data memory 24A of the face image, the transformation of the reaper (1). The composite images GG1 to GG7 replaced with the image data 24Gw1 are generated, and whether or not the special effect motion (gesture) 23P3a to 23P3c of the death effect 23K3 preset in the special effect data memory 23A is detected is determined. In addition, the role of the beautiful girl warrior A who is registered in advance in the transformation role data memory 24A as a face image For the person image J, synthetic images GG2 to GG8 are generated by replacing the face image as it is with the transformation image data 24Gu1 'of the sailor A, and the cure image preset in the special effect data memory 23A is generated. It is determined whether or not the special effect motion (gesture) 23P1a to 23P1c of the effect 23K1 is being performed, and the person image K of the role of the beautiful girl warrior B registered in advance as a face image in the transformation character data memory 24A. Generates a composite image GG6 to GG8 in which the face image is left as is and is replaced with the transformation image data 24Gu2 ′ of the sailor B, and a special effect of the cure effect 23K2 preset in the special effect data memory 23A. Detection judgment whether or not effect motion (gesture) 23P2a-23P2c is being performed It is.

  When it is detected and determined that the special image motion (gesture) 23P3a to 23P3c of the death effect 23K3 set in advance for the person image P of the god of death included in the photographed image data G4, The special effect image data 23Gu3 corresponding to the death effect 23K3 is synthesized in the direction of the person image J of the role of the beautiful girl warrior A set as the target 23T3 of the effect, and the person image of the role of the beautiful girl warrior A A mask image Gk is added and synthesized to J, and is generated and output as a composite image GG4 transformed into a sailor A with a mask. In addition, for example, when it is detected and determined that the special effect motion (gesture) 23P2a to 23P2c of the preset cure effect 23K2 is performed for the person image J of the role of the beautiful girl warrior B included in the photographed image data G7 In the image, the special effect image data 23Gu2 corresponding to the cure effect 23K2 is directed in the direction of the portrait image J of the girl warrior A with mask Gk and the portrait image P of the god of death set as the target 23T2 of the effect. At the same time, the mask image Gk is deleted from the character image J of the role of the beautiful girl warrior A with the mask Gk and transformed into the original beautiful girl warrior A, and the transformation of the god of death (1) image data 24Gw1 Is transformed and output as a composite image GG8 transformed into image data 24Gw1 ′.

  That is, according to the composite image output apparatus of the second embodiment, the special effect motion (gesture) 23P3a to the death effect 23K3 set in advance for the person image P of the reaper role registered in advance by the face image. If it is detected that the player is performing 23P3c, the mask image Gk is added to the person image J of the role of the girl warrior A set as the subject 23T3 of the death effect, and transformed into the girl warrior A with a mask. The generated composite image GG4 is generated and output. In addition, it is detected that the special effect motion (gesture) 23P2a to 23P2c of the preset cure effect 23K2 is also detected for the portrait image J of the role of the beautiful girl warrior B registered in advance by the face image. When it is determined, the mask image Gk is deleted from the character image J of the girl warrior A role with the mask Gk set as the target 23T2 of the cure effect and transformed into the original girl warrior A. The person image P of the god of death set as the cure effect target 23T2 is generated and output as a composite image GG8 transformed into the prince's transformation (2) image data 24Gw1 '.

  As a result, among the person images (P, J, K) extracted from the captured image Gn obtained by moving image shooting, predetermined special effects by the character images P, J, K of boys, sisters, and sisters registered in advance are registered. In accordance with only the motions 23P3, 23P1, and 23P2, the composite image obtained by replacing the person image (J) (P) (P, J) of the opponent character set as the target 23T3, 23T1, 23T2 of the special effect with a transformed image. Can be generated, and the characters in a special-effects movie or animation that transforms a specific opponent by performing predetermined gestures (motion) with the intention of the characters as determined in advance Any amateur can easily generate an image. Then, the generated synthesized moving image GGn can be output and displayed on the display unit 11 or can be output and displayed on an external monitor to enjoy unprecedented enjoyment.

  Further, according to the composite image output device of the second embodiment using the digital camera 10 or the server device 30 having the above-described configuration, the person image P of the boy registered in advance as a god of death is detected from the photographed image Gn, and its gesture (motion (1) When the synthesized image GGn is generated by replacing the person image P with the image data 24Gw1, and the combined image GGn is generated, the transformation (1) image data stored in the transformation character data memory 24A is generated. 24Gw1 audio data “Zaku, Zaku, ...” 24Sw1 is stored and output in association with the synthesized image GGn, and a sister's portrait image J registered in advance as the role of the beautiful girl warrior A is detected from the photographed image Gn. The person image J is replaced with the transformation image data 24Gu1 ′ of the sailor A corresponding to the gesture (motion). When the synthesized image GGn is generated, the audio data “lemon, lemon,...” 24Su1 ′ of the transformation image data 24Gu1 ′ stored in the transformation role data memory 24A is associated with the synthesized image GGn. The younger sister image K registered as the role of the girl warrior B is detected from the photographed image Gn, and the transformation image data 24Gu2 ′ of the girl warrior B corresponding to the gesture (motion) When the synthesized image GGn is generated by replacing the person image K and synthesized, the voice data “cute, cute,...” 24Sw2 ′ of the transformed image data 24Gu2 ′ stored in the transformed character data memory 24A is synthesized. Since it is stored and output in association with the image GGn, it is easier and more realistic to create a special-effects-like synthesized moving image GGn It can be.

  Further, according to the composite image output device of the second embodiment by the digital camera 10 having the above-described configuration, the captured image data Gn displayed in real time on the display unit 11 during the moving image shooting is registered in advance as the god of death. When a boy's person image P is detected from the photographed image Gn, the person image P is transformed into a god of death (1) transformation (1) mark data 24Mw1 indicating that the person is transformed into image data 24Gw1. Is displayed on the display unit 11 as mark-captured photographic image data Gmn added, and the audio data “Zaku, Zaku,...” 24Sw1 of the transformation (1) image data 24Gw1 is also output from the audio output unit 13. When a sister's person image J registered in advance as the role of the beautiful girl warrior A is detected from the photographed image Gn, It is displayed on the display unit 11 as marked photographic image data Gmn to which transformation mark data 24Mu1 'indicating that it has been transformed into the transformation image data 24Gu1' of the girl warrior A, and the voice of the transformation image data 24Gu1 '. The data “lemon, lemon,...” 24Su1 ′ are also output from the audio output unit 13, and when the younger sister's person image K previously registered as the beautiful girl warrior B is detected from the photographed image Gn, The person image K is displayed on the display unit 11 as marked photographic image data Gmn to which the transformation mark data 24Mu2 'indicating that the transformation is made into the transformation image data 24Gu2' of the sailor B is added. The audio data “cute, cute,...” 24Su2 ′ of the transformation image data 24Gu2 ′ is also an audio output unit. 3, for example, the family who is in the shooting operation can easily recognize that the subject person images (boy, sister, sister) P, J, and K have been transformed and replaced according to their registered roles. Even if you are on the subject (boy P, sister J, sister K) who is playing this, it is easy to hear that you have been transformed according to the registered role. Can be recognized.

  Note that each processing method performed by the composite image output apparatus described in each of the above embodiments, that is, a composite image output process shown in the flowchart of FIG. 8, a target image extraction process associated with the composite image output process shown in the flowchart of FIG. The role transformation mark process accompanying the composite image output process shown in the flowchart of FIG. 10, the motion transformation mark process of the first embodiment accompanying the composite image output process shown in the flowchart of FIG. 11, and the composite image shown in the flowchart of FIG. The special effect mark processing according to the first embodiment accompanying the output processing, the transformation mark composition processing accompanying the composite image output processing shown in the flowchart of FIG. 13, and the special effect mark composition processing accompanying the composite image output processing shown in the flowchart of FIG. FIG. 28 is a flowchart showing the second embodiment of the composite image output process. Each of the methods such as the motion transformation mark processing of the state and the special effect mark processing of the second embodiment accompanying the composite image output processing shown in the flowchart of FIG. 29 is a memory card ( (ROM card, RAM card, etc.), magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), external storage medium 18 (33) such as semiconductor memory, etc. can be distributed. Various computer terminals storing the transformation data 14 (32B), special effect data 23A (32CA), defense effect data 23B (32CB), and transformation role data 24A (32D) in the memory 17 (32) The program stored in the medium 18 (33) is read into the program memory 22 (32A), and the operation is controlled by the read program, so that the role is registered from the moving image photographic image data Gn described in the above embodiments. It is possible to realize the function of extracting a person image and creating / outputting a composite image GGn using the transformed image, and execute the same process as described above.

  Further, program data for realizing each of the above methods can be transmitted on a communication network (Internet) N as a program code form, and a computer terminal (program server) connected to the communication network (Internet) N The above program data can be taken in, and the function of extracting the role-registered person image from the moving image photographic image data Gn and the function of creating and outputting the composite image GGn using the transformed image can be realized.

  Note that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the invention at the stage of implementation. Further, each of the embodiments includes inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in each embodiment or some constituent elements are combined in different forms, the problems described in the column of the problem to be solved by the invention If the effects described in the column “Effects of the Invention” can be obtained, a configuration in which these constituent requirements are deleted or combined can be extracted as an invention.

10 ... Digital camera (communication terminal A)
11 (37) ... Display unit 12 (36) ... Input unit 13 ... Audio output unit 14 (32B) ... Transformation data memory 14P ... Transformation motion model data 14Gu (24G ') ... Transformation image data 14Su (24S') ... Transformation voice Data 14Mu (24M ') ... Transformation mark data 14F (24F') ... Face synthesis value data 16 (31) ... CPU
17 (32) ... Memory 18 (33) ... External recording medium 19 (34) ... Recording medium reading section 20 (35) ... Transmission control section 21 ... Image photographing section 22 (32A) ... Program memory 23A (32CA) ... Special effects Data memory 23P ... Special effect motion model data 23Gu ... Special effect image data 23Su ... Special effect sound data 23K ... Type of effect 23T ... Target of effect 23Mu ... Special effect mark data 23B (32CB) ... Defective effect data memory 23'P ... Defense effect motion model data 23'Gu ... Defense effect image data 23'Su ... Defense effect voice data 23'Mu ... Defense effect mark data Mk ... Mask mark data 24A (32D) ... Transformation role data memory 24T ... Transformation target (face) Image data 24Gw (24G) ... Transformation (1) Image data 4Sw (24S) ... Transform (1) Audio data 24Mw (24M) ... Transform (1) Mark data 24Gw '(24G') ... Transform (2) Image data 24Sw '(24S') ... Transform (2) Audio data 24Mw ' (24M ') ... Transformation (2) Mark data 24H ... Transformation condition data 24B (32E) ... Transformation status by role 25 (32F) ... Photographed image data memory 26 (32G) ... Marked photoimage data memory 27 (32H) ... Composite image data memory 30 ... server device 40 ... communication terminal B (PC)
N ... Communication network (Internet)
BG ... Background image data Gn ... Shooting image data Gmn ... Marked shooting image data GGn ... Composite image data P ... Person image (boy)
J ... person image (sister)
K ... person image (sister)

Claims (5)

A transformation character data storage means for storing a human-registered person image in association with image data of a pre-set transformation character;
Captured image acquisition means for acquiring a series of captured image data taken by a movie,
A registered person gesture detection means for detecting whether or not a predetermined transformation gesture is caused by the user-registered person image in the series of photographed image data acquired by the captured image acquisition means;
When the registered person gesture detection unit detects a predetermined transformation by the person image registered by the user, the detected person image is stored in association with the transformation character data storage unit. Based on the image data, composite image generation means for generating composite image data in which the face portion is replaced with the image data of the transformation character having the same posture as the person image;
A composite image output means for outputting the composite image data generated by the composite image generation means;
A transformation data storing unit that stores preset transformation data and the user-registered person image in association with each other;
With
The registered person gesture detection unit detects whether or not there is a corresponding transformation by a person image stored in the transformation data storage unit . The synthesized image generating means, when a predetermined transformation gesture by the person image registered by the registered person gesture detection means is detected, for the subsequent captured image data, the detected person image is used as the transformation character. Based on the image data of the transformation character stored and associated by the data storage means, the image data of the transformation character having the same posture as the person image is generated and the composite image data is generated by replacing the face portion as it is. The composite image output device according to claim 1 . The synthesized image generation means, when a predetermined transformation gesture by the person image registered by the registered person gesture detection means is detected, is added to the image data of the transformation character having the same posture as the person image. Replace the face portion as it is, and generate composite image data that correlates the voice data of the transformation character.
Composite image output apparatus according to claim 1 or 2, characterized in that. The composite image generation means includes
When the registered person gesture detection means detects a predetermined transformation by the person image registered by the user, the photographed image data with the mark added with the mark data indicating the transformation character corresponding to the person image is generated. Having a photographed image generating means with a mark,
Based on the marked photographed image data generated by the marked photographed image generation means, the person image with the mark data indicating the transformation character is added to the image data of the transformation character having the same posture as the person image. Replace the face portion of the image as it is, and generate composite image data that correlates the voice data of the transformation character,
With the acquisition of a series of captured image data by the captured image acquisition means, a marked captured image display control means for displaying the marked captured image data generated by the marked captured image generation means,
Along with the display control of the marked photographed image data by the marked photographed image display control means, the voice output control means for outputting the voice data of the transformation character associated with the mark data of the marked photographed image data,
Composite image output apparatus according to any one of claims 1 to 3, further comprising a. Computer
A transformation character data storage means for storing a human-registered person image in a memory in association with image data of a predetermined transformation character,
Captured image acquisition means for acquiring a series of captured image data taken with a movie,
Registered person gesture detection means for detecting whether or not there is a predetermined transformation gesture by the user-registered person image in the series of photographed image data acquired by the captured image acquisition means,
When the registered person gesture detection unit detects a predetermined transformation by the person image registered by the user, the detected person image is stored in association with the transformation character data storage unit. Based on the image data, composite image generation means for generating composite image data in which the face portion is replaced with the image data of the transformation character having the same posture as the person image;
A composite image output means for outputting the composite image data generated by the composite image generation means;
Transformation data storage means for storing preset transformation data and the user-registered person image in association with each other;
To function as,
The registered person gesture detection means is a computer-readable composite image output process that functions to detect whether or not there is a corresponding transformation gesture based on the person image stored in the transformation image data storage means. program.

Images