JP5028225B2 - Image composition apparatus, image composition method, and program - Google Patents

Description

The present invention relates to an image synthesizing apparatus, images synthesis method, and a program and, more particularly, an image synthesizing apparatus capable of obtaining a composite image in which the object moves in the screen easily, images synthesis method, and a program.

  In recent years, photographic images have been digitized so that not only still images but also moving images can be enjoyed easily on a TV display or personal computer (PC) by outputting them on the camera display or externally. It has become. When enjoying still images and movies, you can not only simply play the recorded image as it is, but also separate the main subject and background image, and create an animation image that moves the main subject freely. Come.

For example, Patent Document 1, by sequentially selecting from the original image such as a still image or a moving image, the image processing apparatus is disclosed that can create one animation image from the thumbnail image.

  The image processing apparatus disclosed in Patent Document 1 requires many operations from the user, and can only be used by a user who is quite skilled in the technology and who does not have time and effort. In addition, it requires a lot of operations from the user, and only a user who is quite skilled in the technology and who does not spare time can use it.

The present invention, such has been made in view of the circumstances, and an object thereof is to provide an image processing apparatus capable of easily create videos enjoyed as a work in the form of animation, images processing method, and a program.

In order to achieve the above object, an image synthesizing apparatus according to a first invention includes a moving image capturing unit that continuously captures an object moving on a background image, and a plurality of images captured by the moving image capturing unit. A background image composition unit that synthesizes the images, and extracts a jumped object from the subjects included in the plurality of images. Based on the size change and the moving direction of the objects, A detecting unit that detects a change in size and a moving direction of the jumped object included in the specific frame, and the jumped object included in the specific frame in the continuously photographed image includes A continuous image that changes on the background image according to a change in the size of the object included in the specific frame detected by the detection unit and generates a continuous image that moves in the moving direction detected by the detection unit Generator and the back Comprising the above-described background image synthesized by the image synthesizing unit, and a synthesizing unit for synthesizing the image sequence generated by the continuous image generating unit.

In the image composition device according to the second invention, in the first invention, the background image composition unit does not photograph the object in each of a plurality of images photographed by the moving image capturing unit. The background image is synthesized by extracting and synthesizing the parts.

According to a third aspect of the present invention, in the first aspect , the continuous image generation unit includes an adjustment unit that adjusts the moving direction.
According to a fourth aspect of the present invention, in the first aspect , the continuous image generation unit includes an adjustment unit that adjusts the moving speed of the object.
Furthermore, in the image composition device according to the fifth invention, in the first invention, the composition unit composes a different background image instead of the background image.

In order to achieve the above object, an image composition method according to a sixth aspect of the present invention continuously shoots an object moving on a background image, synthesizes the background image from the plurality of photographed images, The jumped target is extracted from the subjects included in the image, and the size of the jumped target included in the specific frame in the plurality of images is extracted based on the change in the size of the target and the moving direction. The jumped object included in the specific frame in the continuously photographed image is detected on the background image and the size of the object included in the specific frame is detected. And a continuous image that moves in the moving direction is generated, and the background image and the continuous image are synthesized.

In order to achieve the above object, a program according to a seventh aspect of the invention selects a background image from one image among a plurality of images obtained by continuously photographing an object moving on a background image, or A step of synthesizing the background image from the image, and extracting a jumped object from the subjects included in the plurality of images, and based on the size change and the moving direction of the object, Detecting a change in size and a moving direction of the jumped object included in the specific piece, selecting the specific piece including the target object from the plurality of images, and jumping the target object of the specific piece. And a step of generating a continuous image that changes in accordance with a change in the size of the image and moves in the moving direction, and a step of synthesizing the background image and the continuous image. The

According to the present invention, it is possible to provide an image processing apparatus capable of easily create videos enjoyed as a work in the form of animation, images processing method, and a program.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. FIG. 1 is a block diagram showing a configuration of a camera 10 according to an embodiment of the present invention. The camera 10 is a digital camera, and includes a moving image capturing unit 2, a head face position determination unit 5, a display unit 8, a composite display unit 8a, an operation determination unit 12, a communication unit 13, an image processing unit 15, an image recording unit 16, and a background. A recording unit 16a, a main subject recording unit 16b, an animation creation unit 17, a background separation unit 21, a comparison unit 21a, a change determination unit 21b, and an image composition unit 22 are provided.

  The moving image pickup unit 2 includes an image pickup device that can read out an image signal at high speed, and in response to a full press operation of a release button, a moving image (continuously shot image, continuous shot image) based on a photoelectric conversion signal of a subject image is displayed. Image data can be read out. When the still image shooting mode is set, still images can be read out in the same manner as moving images.

  The image processing unit 15 performs processing such as image processing and image compression on the image data read by the moving image capturing unit 2 in cooperation with the control unit 11. The image recording unit 16 includes a recording medium that records the image data processed by the image processing unit 15, a recording control unit for recording on the recording medium, and the like. The moving image data includes both images of a moving subject and a background that does not move, and a background separation unit 21 in the image processing unit 15 separates the moving subject and the background.

  That is, the comparison unit 21a compares the images in the moving image, and the change determination unit 21b separates an image with a change as a main subject and an image without a change as a background based on the comparison result. The separated main subject image data and background image data are recorded in the background recording unit 16a and the main subject recording unit 16b in the image recording unit 16, respectively.

  The image composition unit 22 in the image processing unit 15 arranges the main subject on the background image separated by the background separation unit 21 according to a predetermined procedure, and performs image composition on the background image. The animation creating unit 17 creates moving image data in an animated manner in order to reproduce the image synthesized by the image synthesizing unit 22 in a moving image style.

  The head face position determination unit 5 determines the face pattern and the shape pattern of the person in the image and detects the position of the face and head when the photographed image includes a person as a subject. The detected face and head positions are used for focusing at the time of shooting, and this information is also recorded when the image data is recorded in the image recording unit 16. Further, it is used for determining the main subject and determining the movement.

  The display unit 8 includes a liquid crystal monitor or the like disposed on the back surface of the camera 10 or the like. The display unit 8 reproduces and displays the image data recorded in the image recording unit 16, and also performs live view display for observing the subject image instead of the optical viewfinder. In addition, images are synthesized and animation is displayed, and the user can enjoy moving image display such as animation display. Further, the composite display unit 8a in the display unit 8 displays the held image in a translucent manner on the screen when shooting an image suitable for image synthesis, and compares it with the translucent display, thereby comparing the object. The movement can be confirmed.

  The communication unit 13 performs transmission / reception with a server or an external device via the Internet or a network. Via this communication part 13, a picked-up image and a synthesized image can be transmitted outside, and an image etc. can be received from the outside. The operation determination unit 12 determines the operation state of the operation member of the camera 10 by the user. The control unit 11 is connected to each unit in the camera 10 and performs overall control of the camera 10 according to the operation of the operation member and the like determined by the operation determination unit 12.

  Next, the operation of the camera 10 according to this embodiment configured as described above will be described. In the present embodiment, the state in which the subject jumps is photographed with a moving image (continuous shooting), the background and the main subject are separated from the moving image, and the extracted main subject is swimming in the air against the extracted background. Generate an animated image as if it were.

  First, background separation and main subject change determination will be described with reference to FIGS. FIG. 2 shows a situation where the photographer 42 shoots a moving image (continuous shooting) of the main subject 41. Ideally, it is better to fix the camera 10 to a tripod 31 or the like so that the background image does not move. However, if the camera 10 has a camera shake correction circuit or a correction mechanism, handheld shooting may be used.

  The photographer 42 continuously takes a picture of the main subject 41 jumping from the starting point in FIG. 2A to the final point shown in FIG. 2C through the intermediate point in FIG. 2B. That is, at the starting point, the main subject is located at a distance L1 from the camera 10 (FIG. 2 (a)), and in FIG. 2 (b), the main subject jumps to the highest point by a jump, and at the final point, The subject is located away from the camera 10 by a distance L2.

  In the shooting state shown in FIG. 2, moving image shooting (continuous shooting) is executed, and a moving image consisting of a plurality of continuous still images can be acquired. A background image is extracted from the still image of a plurality of frames. In this embodiment, two images before and after the jump (FIG. 3 (a)) and after the jump (FIG. 3 (b)) as shown in FIG. 3 are used. Extract a background image.

  In the present embodiment, change information of the main subject is acquired using the image shown in FIG. That is, in FIG. 3, the main subject 41 has moved from the position X1 close to the left end of the screen to the position X2 close to the right end of the screen. When there is this position change X1 → X2, the head face position detection unit 5 detects a change in the size of the head (F1 → F2). This size change of the head (F1 → F2) is used when changing the size of the main subject 41 when animating, as will be described later.

  Assume that the main subject 41 is moved as shown in FIG. 4A when extracting the background using FIGS. 3A and 3B. In this case, when the images before and after FIG. 3 (a) are compared, there is no moving image in the right half, and when the images before and after FIG. 3 (b) are compared, the image moves to the left half of the screen. You can see that there is no. Therefore, a background image as shown in FIG. 4B can be obtained by synthesizing the right half image of FIG. 3A and the left half image of FIG. 3B.

  The main subject 41 can be considered as a difference between the background image shown in FIG. 4B and the actually captured image. Therefore, an image obtained by extracting only the main subject 41 can be obtained by obtaining a difference between background images from an actually captured image. When there is a sufficient memory capacity or when the main subject can move quickly, as shown in FIG. 4C, an image in which the main subject 41 has moved outside the shooting screen is used as a background image. You may make it select.

  Next, background image composition and main subject extraction based on continuous shooting will be described with reference to FIG. When moving image shooting (continuous shooting) is performed in the shooting state as shown in FIG. 2, images of a plurality of continuous frames as shown in FIGS. 5A and 5B are obtained. In this example, nine frames of images are obtained, arranged in time series in the order of upper left → right, middle left → right, lower left → right. The image shown in FIG. 3 (a) is the image 101 shown in the upper left uppermost part of FIGS. 5 (a) and 5 (b), and the image shown in FIG. 3 (b) is that shown in FIGS. 5 (a) and 5 (b). This corresponds to the image 109 shown in the lower right bottom row. Note that the continuously shot images shown in FIGS. 5A and 5B can be displayed on the display unit 8 of the camera 10.

  An image for extracting the main subject 41 is selected from these continuously photographed images. In the example shown in FIG. 5A, the main subject 41 is separated based on the image 105 at the time of the highest jump. However, when separating the main subject 41, various other states can be selected such as the moment when both hands are spread or the moment when the expression is changed. Here, an image 105 is selected, and an image for extracting the main subject 41 is shown in FIG.

  Next, the background image is synthesized using the image 101 and the image 109 shown in FIG. 5B and using the portion where the subject does not move as described above. FIG. 5D shows the background image 110 synthesized in this way. In synthesizing the background image, the background separation unit 21 of the camera 10 may automatically determine and synthesize the non-moving portion. Also, the user can view the two images for background synthesis while watching the continuously shot images. You may choose.

  Only the main subject 41 is extracted using the image to be extracted (FIG. 5C) as the main subject and the synthesized background image 110 (FIG. 5D). That is, if a portion that exists in FIG. 5C and does not exist in FIG. 5D is extracted, only the main subject 41 as shown in FIG. 5E can be extracted.

  Next, using the image to be extracted as the main subject (FIG. 5C), as shown in FIG. 6A, the main subject 41 is moved in the screen as if swimming in the air. The generation of a moving animation will be described with reference to FIG. That is, when the continuous image 111 of the main subject shown in FIG. 6A is assigned to the continuous shooting of nine frames as shown in FIGS. 5A and 5B, the result is as shown in FIG. 6B. If this is continuously played back, it is possible to create and play back images that could not be expressed in the past, such as floating in the air.

  In this case, the change in the size of the main subject 41 is reflected by being derived from the relationship between the distance X and the size F of the head obtained in FIGS. That is, using a pseudo perspective method, an animation in which the main subject 41 moves away or approaches is created. In the image divided into nine as in the present embodiment, X is obtained from the position where the main subject is displayed on each frame by dividing the screen into nine horizontally. When the screen width is X0, the main subject display position of each frame is X0 / 9, 2 × X0 / 9, 3 × X0 / 9,. In addition, a change in consideration of (F2-F1) / (X2-X1) is applied to the main subject of each frame. That is, in this example, X0 / 9 is multiplied by this to determine the change in size.

  Specifically, the person is enlarged or reduced using the size of each part as shown in FIG. 16 (a) and 16 (c) correspond to FIGS. 3 (a) and 3 (b), and assume that there is a position X3 (FIG. 16 (b)) where the person jumps high in the middle. When it divides into nine and jumps the highest at the position of n × X0 / 9, the person to be synthesized at the position of the previous frame, (n−1) × X0 / 9, has a face size change rate , R = (T3 / F3) · (F2-F1) / (X2-X1), the size of the subject is T + R × X0 / 9, and the subsequent frames are T−R × X0 / 9 The relationship of the person size T can be determined from the graph shown in Fig. 17. Although the pseudo perspective method has been described by the method shown in Fig. 17, other than this, Processing that changes the size of the main subject may be performed depending on the method.

  Further, the moving direction of the main subject 41 may be the moving direction 121 before and after the adopted frame shown in FIG. 5B, or the user may change freely. FIG. 7A shows an example in which the floating image of the main subject 41 obtained in this way is synthesized and displayed on the original background image. By displaying from the upper left to the lower right sequentially, it is possible to reproduce the state in which the main subject 41 floats and moves. Of course, you may enjoy without a background, you may affix it on another image and enjoy a surreal image | video as shown in FIG.7 (b). Such background selection is another new way to enjoy images.

  Since the background and the main subject to be extracted are selected from the continuously shot moving images in this way, animation can be created by a series of operations and can be enjoyed at the moment when shooting is finished. In addition, selecting the best image from the continuously shot moving images can be said to be an effect unique to a moving image having a large number of images.

  Next, the operation of the camera that can easily take an unreal photo as described above will be described using the flowchart shown in FIG. First, when entering a camera control routine, image data is acquired from the imaging unit prior to shooting, and live view display is performed on the display unit 8 for composition determination (S1). Subsequently, it is determined whether or not the release button has been fully pressed by the operation determination unit 12, that is, whether or not shooting is to be performed (S2).

  As a result of the determination, when shooting is performed, continuous shooting is performed (S5). The continuous shooting here is assumed to be faster than the continuous shooting speed of 5 frames per second like the continuous shooting of a general camera, for example, capable of shooting with 15 frames, 30 frames, and 60 frames like a moving image. . Subsequently, it is determined whether or not the continuous shooting is finished (S6). If the result of determination is that it has not ended, processing returns to step S5 and continuous shooting is continued.

  When the continuous shooting is finished as a result of the determination in step S6, the image data is recorded in the image recording unit 16 (S7). The end of continuous shooting may be performed by the photographer, but it may be determined that the subject is no longer off the screen and automatically ended. When the image recording is completed, the process returns and returns to the first step.

  If the result of determination in step S2 is not shooting, it is determined whether or not confirmation is before shooting (S11). As described above, in this embodiment, in order to extract the main subject from the background image, the moving image shooting (continuous shooting) as described with reference to FIGS. 2 to 5 is necessary. In particular, as shown in FIGS. 3A and 3B, it is necessary that the main subject 41 be photographed as one frame for continuous photographing, and as shown in an image 105 in FIG. 5A. In addition, it is necessary that the jumped main subject is within the screen. Therefore, it is possible to confirm whether or not the main subject 41 satisfies the above conditions before shooting.

  If the pre-shooting confirmation mode is set as a result of the determination in step S11, it is determined whether or not the holding operation has been performed (S12). The holding operation is, for example, an operation of a release switch or the like. If the holding operation is performed as a result of the determination, the still image at the time when the holding operation such as the release switch is performed is acquired from the moving image pickup unit 2 and the still image is temporarily held (S13).

  If the result of determination in step S <b> 12 is that a holding operation has not been performed, it is determined whether or not a holding screen is already present (S <b> 17). After holding the still image in step S13 or if the result of determination in step S17 is that there is a holding screen, the held still image is displayed on the display unit 8 in a translucent manner (S14). By displaying the held still image in a semi-transparent state, it becomes possible to start shooting by determining in advance the relationship between the jumping landing point and the jumping point as shown in FIG. .

  Therefore, in the pre-shooting confirmation mode, for example, the position before the jump is recorded, and how far it can fly can be practiced (S12 Yes → S14 or S17 Yes → S14). Thus, in the present embodiment, a moving main subject is assumed, and a pre-shooting confirmation mode is provided in order to grasp the movement of the main subject in advance.

  Once the semi-transparent display in step S14 is performed, it is next determined whether or not to end the pre-shooting confirmation mode (S15). When ending the pre-shooting confirmation mode, the photographer operates an end button of the pre-shooting confirmation mode, and therefore it is determined whether or not this button has been operated. If the result of determination is that the display is finished, the translucent display on the display unit 8 is finished (S16). If there is no holding screen in step S17, if it is not the end in step S15, or if the translucent display is ended, the process returns and returns to the first step 1.

  If the result of determination in step S <b> 11 is not confirmation before shooting, it is next determined whether or not playback mode is set (S <b> 21). If the result of determination is that it is not playback mode, processing returns to step S1. On the other hand, if the playback mode is selected, a playback operation is performed (S22). In the reproducing operation, image data of still images and moving images recorded in the image recording unit 16 is read and displayed on the display unit 8. In reproduction, thumbnail display for displaying a plurality of frames at a reduced size or display for each frame is performed.

  Subsequently, it is determined whether or not to animate (S23). This is a determination as to whether or not an operation member that instructs animation is operated when a moving image (continuous shooting) is selected. If the result of determination is that it is not playback mode, processing returns to step S1. On the other hand, when animation is performed, background extraction is performed (S24). In the background extraction, as described with reference to FIGS. 4 and 5, a background image is synthesized from an image of a portion where the main subject 41 does not exist. The background extraction subroutine will be described later with reference to FIG.

  When the background extraction is completed, the main subject is extracted (S25). The main subject extraction is a process for extracting only the main subject 41 as shown in FIG. The main subject extraction subroutine will be described later with reference to FIG. When the main subject is extracted, the main subject and the background image are synthesized (S26). Thereafter, the composite image is reproduced and adjusted (S27). During playback, it is possible to adjust the traveling direction, speed, background, etc. while viewing the floating image of the main subject. The processing in step S26 and step S27 will be described later with reference to FIG.

  Next, the background extraction subroutine of step S24 will be described with reference to FIG. First, the comparison unit 21a and the change determination unit 21b of the background separation unit 21 determine whether there is a moving object-free screen (S41). If there is an image in which a moving object has passed or an image before passing in a moving image obtained by shooting a series of moving objects, that image can be used as a background image. As a result of the determination, if there is a screen without moving objects, that screen is set as a background image (S48).

  On the other hand, if there is no moving object screen as a result of the determination, a frame having no motion is determined in the left half of the screen (S42). Subsequently, a frame having no movement in the right half of the screen is determined (S43). That is, in step S42 and step S43, a frame having no moving object is searched for in each of the left half and the right half of the screen.

  When the frame determination in steps S42 and S43 is completed, it is determined whether or not the background image can be synthesized using the non-motion portion (S44). If there is no image that does not move left and right in the series of moving images, the background image cannot be combined, so it is determined whether the background image can be combined. If the result of determination is that synthesis is possible, a background image is synthesized using the two detected images (S46). On the other hand, if the composition is impossible, a warning is displayed (S45). When step S45, S46 or S48 ends, the process returns to the original flow.

  Next, a sub-routine for extracting a main subject from a captured image using a background image will be described with reference to FIG. The subject extracted here is an image of the main subject used for animation, such as floating in the air, as will be described later. Which image to select from a series of videos may be determined by the user's preference, but in this embodiment, when the main subject jumps to the highest position, the image is preferentially selected and the user selects If you don't like it, you can change it manually.

  When the main subject extraction subroutine is entered, first, a frame in which the main subject has changed to the highest position is selected from a series of moving images (S51). When the selection is made, the image is displayed on the display unit 8, so that the user can see the image. The user determines whether the main subject can be extracted from the display image (S52). If the image is acceptable, the user operates an operation member such as an OK button and manually selects the image to change to another image (S53).

  If the main subject is selected automatically or manually, the comparison is then performed by the designated frame and background image comparing unit 21a (S54). Subsequently, a specified frame that is not in the background image is extracted as a main subject (S56). It is determined whether or not there is a problem that the main subject extracted here protrudes from the screen (S56). If there is a problem as a result of the determination, a redo warning is displayed (S57), and the process returns to the original routine.

  If the image of the main subject is OK, then a change in the direction and size of the main subject used for synthesizing the animation image such as floating in the air is detected. First, a frame that is in front of the designated frame but not in the background frame is extracted (S58). Subsequently, a frame that is behind the designated frame and not in the background frame is extracted (S59). In these steps S58 and S59, the main subject before and after the designated frame can be extracted. Based on the change of the main subject extracted from the preceding and following frames, the change of the moving direction and size of the main subject is determined (S60). This determination is performed using a pseudo perspective method as described in FIGS. 3, 16, and 17.

  Next, a synthesis and correction subroutine for synthesizing, reproducing, and adjusting the background image and the main subject will be described with reference to FIG. As described above, this subroutine corresponds to steps S26 and S27 of the flowchart shown in FIG.

  When the composition and correction subroutine is entered, first, replacement of the first frame for shooting is started (S61). For example, as shown in FIG. 5, when nine-frame continuous shooting is performed, the first frame is replaced with a composite image. When the replacement with the composite image is started, a copy of the main subject is created (S62). In duplication, the size of the main subject image is determined according to the change in the size of the main subject obtained in step S60, and the main subject image is enlarged or reduced.

  Next, the main subject is synthesized with the background image (S63). This is performed by combining the main subject generated in step S62 with the background image created in step S46 or step S48 of FIG. This completes the image composition for the first frame. Subsequently, it is determined whether or not the next frame is to be replaced (S64). As a result of the determination, if there is a next frame, the next frame is replaced (S65), and the process returns to step S62 to copy the main subject to the captured frame (S62). (S64).

  By repeating steps S62 to S65 up to the last frame, it is possible to obtain a composite image in which the size and position of the main subject change little by little corresponding to all the frames that have been shot. That is, an animation of the same time as the time of shooting is obtained, and the user is recognized as a camera capable of shooting the animation.

  When the replacement is completed for all the frames as a result of the determination in step S64, it is determined whether or not to confirm (S66). This confirmation is a step for reproducing and displaying the completed state of the composite image generated by repeating steps S62 to S65, and determining the operation state of the operation member for confirmation reproduction. If the user does not want to confirm playback, the process skips step S66 and returns to the original routine.

  As a result of the determination in step S66, if confirmation playback is to be performed, the composite image that has been animated is reproduced and displayed on the display unit 8 (S67). Subsequently, it is determined whether or not adjustment is necessary (S68). This step is to determine whether or not the user looks at the playback display of the composite image and adjusts the moving speed or the like of the main subject according to his / her preference. This determination is made based on whether or not the user has operated the adjustment operation member.

  If the result of determination in step S <b> 68 is that adjustment is not necessary, step S <b> 68 is skipped and the original routine is returned to. On the other hand, if adjustment is necessary, the moving speed (speed) of the main subject is switched (S69). This speed switching is performed by selecting a speed switching control menu and operating the cross key operation member 23 shown in FIG. In response to this operation, the image is thinned or increased, and the display control of the speed display bar 24 in the speed adjustment mode is performed. For example, as shown in FIGS. 7 (a) and 7 (b), when 9 frames were spent to display an animation image, if the speed is increased, the image is thinned as shown in FIG. 12 (b). Speed up and display animated images with 3 frames.

  Next, the direction of the main subject is switched (S70). In this step, the direction of movement of the main subject 41 is displayed by an arrow by selecting a direction switching control menu. In this state, fine adjustment may be performed by operating the cross operation member 23 provided on the back surface of the camera 10, for example. In the example shown in FIGS. 7A and 7B, the moving direction of the main subject 41 is the upper right, but by adjusting, the moving direction can be changed to the lower right as shown in FIG. 13B. it can.

  When the direction switching is finished, background switching is performed (S71). In this step, the background image 110 is displayed as a thumbnail by selecting the background switching control menu. In this state, the background image for the main subject 41 can be switched by operating the cross operation member 23. When the background switching is completed, the process returns to step S61, and image composition is performed again with the contents set for switching. FIG. 15 shows an example of a moving image that is animated by changing the direction and the background as described above and is rich in change.

  As described above, in the present embodiment, an object such as a person (main subject) is continuously shot (moving image shooting, continuous shooting), and a background image is selected from the continuously shot images, or a plurality of images are selected. The background image is synthesized from the images. In addition, a change in the size of the object and the moving direction are detected from a plurality of images taken continuously, and based on this information, a continuous image 111 is generated for the object shown in the specific frame, and this is used as a background image. By synthesizing, an animated image is generated. For this reason, it is possible to provide an image composition device that can create a moving image of a scene that cannot normally be photographed with a simple operation and enjoy watching it.

  In the present embodiment, both the direction of movement and the change in the size of the main subject are detected, and the extracted continuous image 111 of the main subject is generated based on both pieces of information (FIG. 11). S62). However, it goes without saying that either one of the information on the direction of movement or the change in the size is used, and for the other information, a continuous image of the main subject is generated according to a preset default value.

  In the present embodiment, when compositing images, a main subject is assigned to each frame that has been continuously shot (see S61 to S65 in FIG. 11). However, in addition to the one-to-one assignment, an arbitrary number may be assigned. Further, the background image is preferentially used as a composite of continuously shot images. However, the background image is not limited to this, and the background image as shown in FIG. Alternatively, a background image synthesized from continuously taken images may be used.

  Further, in the present embodiment, the background image synthesis, the main subject extraction, and the image synthesis are performed as they are in a continuously shot camera. However, the present invention is not limited to this, and only continuous shooting is performed in the camera. Background image synthesis, main subject extraction, image synthesis, and the like are performed by an information device such as a personal computer using continuous shooting images recorded in the camera. Of course, the processing may be performed.

  In the embodiment of the present invention, not only a person existing in the center of the screen but also, for example, a moving object in the peripheral part of the screen is taken as an object to be photographed. Of course, it is possible to extract the object from the background so that it can be reproduced with an animation effect in the same manner as the main subject 41. In this embodiment, a digital camera is used as an apparatus for continuous shooting. However, the camera may be a digital single-lens reflex camera or a compact digital camera, and may be a mobile phone or a personal digital assistant (PDA). : Of course, a camera built in Personal Digital Assist) may be used.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

It is a block diagram which shows the structure of the camera concerning one Embodiment of this invention. In one embodiment of the present invention, it is a diagram showing a situation when the photographer continuously shoots the main subject, (a) is the starting point of the main subject, (b) is the intermediate point, (c) is the final point It is a figure which shows a mode. In one Embodiment of this invention, it is a figure which shows the picked-up image before and behind the jump of the main subject, (a) is before a jump, (b) is a picked-up image after a jump. FIG. 4 is a diagram illustrating generation of a background image in one embodiment of the present invention, where (a) shows how the main subject moves, (b) shows the synthesized background image, and (c) shows the main subject. An example of being outside the screen and usable as a background image is shown. FIG. 2 is a diagram for explaining background image synthesis and main subject extraction in an embodiment of the present invention, wherein (a) and (b) are diagrams showing each frame continuously shot, and (c) is a main subject. (D) shows the synthesized background image, and (e) shows the extracted main subject. In one Embodiment of this invention, the continuous image of the main subject produced | generated is shown, (a) is a continuous image of a main subject, (b) is a figure which shows the main subject allocated to each frame continuously image | photographed. is there. FIG. 6 is a diagram showing each frame of an image that has been combined and animated in an embodiment of the present invention, and (a) is an example in which a main subject is combined with a background image generated from images obtained by continuously shooting, (b) FIG. 5 is a diagram showing an example of combining background images using different ones. It is a flowchart which shows the operation | movement of the camera control in the camera concerning one Embodiment of this invention. It is a flowchart which shows the operation | movement of background extraction in the camera concerning one Embodiment of this invention. 5 is a flowchart showing an operation of taking out a main subject in a camera according to an embodiment of the present invention. It is a flowchart which shows the operation | movement of a synthesis | combination and correction | amendment in the camera concerning one Embodiment of this invention. In the camera concerning one Embodiment of this invention, it is a figure explaining speed switching, (a) is a figure at the time of adjustment, (b) is a figure which shows the animation synthesized image after adjustment. In the camera concerning one embodiment of the present invention, it is a figure explaining direction change, (a) is a figure at the time of adjustment, and (b) is a figure showing the animation synthesized picture after adjustment. It is a figure explaining background switching in the camera concerning one embodiment of the present invention. It is a figure which shows the synthesized image which performed direction switching and background switching in the camera concerning one Embodiment of this invention. In one embodiment of the present invention, it is a diagram showing the relationship of the size of each image when obtaining a change in the size of the main subject, (a) is the starting point, (b) is when the main subject jumps, c) shows the final point. 6 is a graph showing a change in size of a main subject in an embodiment of the present invention.

Explanation of symbols

2 ... moving image capturing unit, 5 ... head face position determining unit, 8 ... display unit, 8a ... composite display unit, 10 ... camera, 12 ... operation determining unit, 13. ..Communication unit, 15 ... Image processing unit, 16 ... Image recording unit, 16a ... Background recording unit, 16b ... Main subject recording unit, 17 ... Animal creation unit, 21 ... Background separation unit, 21a ... comparison unit, 21b ... conversion determination unit, 22 ... image composition unit, 23 ... cross operation member, 24 ... speed display bar, 31 ... tripod, 41 ... Main subject, 42 ... Photographer, 101 ... Image, 105 ... Image, 109 ... Image, 110 ... Background image, 111 ... Continuous image, 121 ... Movement direction

Claims (7)

A moving image capturing unit that continuously captures an object moving on a background image;
A background image synthesizing unit that synthesizes the background image from a plurality of images captured by the moving image capturing unit;
The jumped object is extracted from the subjects included in the plurality of images, and the jumped object included in the specific frame in the plurality of images is extracted based on the size change and the moving direction of the object. A detection unit for detecting a change in size and a moving direction;
The jumped target object included in the specific frame in the continuously photographed images changes the size of the target object included in the specific frame detected by the detection unit on the background image. A continuous image generating unit that generates a continuous image that changes according to the moving direction detected by the detecting unit,
A composition unit that synthesizes the background image synthesized by the background image synthesis unit and the continuous image generated by the continuous image generation unit;
An image composition device comprising:   The background image synthesis unit is configured to synthesize the background image by extracting and synthesizing portions of the plurality of images captured by the moving image capturing unit that are not captured by the object. The image synthesizing apparatus according to claim 1, wherein the image synthesizing apparatus is provided. The image composition apparatus according to claim 1 , wherein the continuous image generation unit includes an adjustment unit configured to adjust the moving direction. The image composition apparatus according to claim 1 , wherein the continuous image generation unit includes an adjustment unit that adjusts a moving speed of the object. The image synthesizing apparatus according to claim 1 , wherein the synthesizing unit synthesizes a different background image instead of the background image. Shoot continuously the object moving on the background image,
The background image is synthesized from the plurality of captured images,
The jumped object is extracted from the subjects included in the plurality of images, and the jumped object included in the specific frame in the plurality of images is extracted based on the size change and the moving direction of the object. Detects the change in size and direction of movement,
The jumped object included in the specific piece in the continuously photographed image changes on the background image according to a change in the size of the target object included in the specific piece, Generate a continuous image moving in the moving direction,
Combining the background image and the continuous image;
An image composition method characterized by the above. Selecting a background image from one of a plurality of images obtained by continuously capturing an object moving on the background image, or synthesizing the background image from the plurality of images;
The jumped object is extracted from the subjects included in the plurality of images, and the jumped object included in the specific frame in the plurality of images is extracted based on the size change and the moving direction of the object. Detecting a change in size and direction of movement;
Selecting the specific piece including the object from the plurality of images, and generating a continuous image that changes in accordance with a change in the size of the jumped object of the specific piece and moves in the moving direction; ,
Combining the background image and the continuous image;
A program that causes a computer to execute.