JP2012151724A - Synthetic image creating apparatus, synthetic image creating program, and imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve such a problem that a complex image processing is necessary to change a size of a subject, position, lightness and darkness, shading, clearness, saturation, chromatic sum, and so on in order to generate an image having perspective and stereoscopic effect.SOLUTION: The synthetic image creating apparatus comprises: a layer creating part 112 which generates a plurality of layer images according to perspective of a subject by edging and cutting off subjects with common perspective on an image to make one layer image; and a layer synthesizing part 113 which generates a synthetic image with a stereoscopic effect by synthesizing a plurality of layer images generated by the layer creating part 112.

Description

  The present invention relates to a technique for creating a stereoscopic image from a two-dimensional image.

  By digitizing an image, it becomes easier to process the image, and various image processing / processing techniques have been developed. For example, a camera that extracts a specific person from an image (see Patent Document 1) and a camera that has a contour extraction function that specifies a range of a specific subject on a screen (see Patent Document 2) are known.

  On the other hand, recently, there has been a demand for a camera that generates a creative image by performing image processing on an image obtained by photographing. As one of them, several cameras that can generate a stereoscopic image have been proposed. For example, an image composition method for synthesizing a two-dimensional logo mark into a three-dimensional image is known (see Patent Document 3). There has also been proposed an image processing apparatus that converts an image obtained by photographing into an image that emphasizes perspective and stereoscopic effect (see Patent Document 4).

JP 2010-098404 A JP 2010-213105 A JP 2001-223877 A JP 2010-154422 A

  However, in the technique described in Patent Document 4 that converts an image obtained by shooting into an image that emphasizes perspective and stereoscopic effect, the size, position, brightness, shading, clearness, saturation, color summation of the subject, etc. It was necessary to carry out complicated image processing to change.

  An object of this invention is to provide the technique which produces | generates an image with a three-dimensional effect, without performing complicated image processing.

  A composite image creation apparatus according to an aspect of the present invention creates a single layer image by trimming a subject with a common perspective on the image, and creates a plurality of layer images according to the perspective of the subject. A layer creation unit; and a layer synthesis unit that creates a composite image having a stereoscopic effect by synthesizing a plurality of layer images created by the layer creation unit.

  An imaging apparatus according to another aspect of the present invention provides a single layer image by trimming and cropping subjects having a common perspective on an image, and creates a plurality of layer images according to the perspective of the subject A composition unit, a layer composition unit that creates a three-dimensional composite image by compositing a plurality of layer images created by the layer creation unit, a display unit that displays an image, and a live view image at the time of shooting A display control unit to be displayed on the display unit, and the display control unit causes the display unit to display a three-dimensional composite image created by the layer composition unit as the live view image.

  According to still another aspect of the present invention, there is provided a composite image creation program that creates a single layer image by trimming a subject having a common perspective on the image and cutting a plurality of layer images according to the perspective of the subject. The computer is caused to execute a step of creating and a step of creating a three-dimensional composite image by combining the plurality of generated layer images.

  According to the present invention, it is possible to easily generate a stereoscopic image without performing complicated image processing for changing the size of the subject.

It is a block diagram which shows the structure of the digital camera provided with the synthesized image production apparatus in one embodiment. It is a figure which shows an example of a live view image. It is a figure for demonstrating the production | generation method of a layer image when the number of layer images is set to five. It is a figure which shows an example of the image with a three-dimensional effect produced | generated by synthesize | combining the five layer image shown in FIG. FIG. 5A is a diagram showing an example of a composite image when the number of layer images is nine, and FIG. 5B is a case where the number of layer images is five, and a shadow is added to the layer image. It is a figure which shows an example of the synthesized image at the time of performing the process to perform. It is a figure for demonstrating the principle of expressing the display effect that the viewpoint changed. 7A is a diagram showing a state in which the composite image shown in FIG. 4 is displayed as seen from the L viewpoint, and FIG. 7B is a diagram showing the composite image shown in FIG. FIG. 6 is a diagram illustrating a state where display is performed as viewed from an R viewpoint. FIG. 8A is a diagram illustrating an example of a composite image displayed on the display unit in a state where the digital camera is not tilted, and FIG. 8B is a diagram where the digital camera is tilted to the right side. It is a figure which shows an example of the synthesized image displayed on a display part. It is a figure which shows an example of the screen for a user to specify the subject included in a layer image. It is a flowchart which shows the processing content performed with the digital camera provided with the synthesized image production apparatus in one Embodiment. It is a flowchart which shows the detailed content of a three-dimensional live view display process.

  FIG. 1 is a block diagram illustrating a configuration of a digital camera 1 according to an embodiment. The digital camera 1 is an imaging device having a photographing function, and is also a composite image creation device 5 that creates a composite image having a stereoscopic effect. The digital camera 1 includes a control unit 11, a program / data storage unit 12, an operation unit 13, an angle sensor 14, an imaging unit 15, an SDRAM 16, an image processing unit 17, a display driver unit 18, and a display unit. 19, a compression / decompression unit 20, a playback unit 21, a recording unit 22, and an image storage unit 23.

  The control unit (CPU) 11 operates according to a control program stored in the program / data storage unit 12 and performs overall control of the camera. In the control unit 11, a display control unit 111, a layer creation unit 112, a layer composition unit 113, and an inclination detection unit 114 are provided. Since the display control unit 111, the layer creation unit 112, the layer composition unit 113, and the inclination detection unit 114 are part of the processing executed by the control program, they will be described as functional units included in the control unit 11.

  The display control unit 111 performs display control when displaying a reproduced image including a still image and a moving image on the display unit 19 described later.

  The layer creation unit 112 detects the contour of each subject from the luminance difference and contrast of the image, and further specifies the shape of each subject in consideration of information on the color and shape of the subject. The layer creating unit 112 further creates a layer image based on the positional relationship between the identified subject shapes. Details of the layer image will be described later.

  The layer synthesizing unit 113 synthesizes a plurality of layer images created by the layer creating unit 112 to generate a three-dimensional synthesized image.

  The tilt detection unit 114 detects the tilt of the digital camera 1 based on the angle of the digital camera 1 detected by the angle sensor 14.

  The control unit 11 is connected to a program / data storage unit 12, an operation unit 13, an angle sensor 14, and a bus 25. As described above, the program / data storage unit 12 has the same configuration. A program to be executed in the control unit 11 is stored, and various data are also stored.

  The operation unit 13 includes various operation members and a touch panel for a user to give an instruction to the camera. The various operation members include, for example, a power button, a release button, an OK button, a cross button, a playback button, a tele wide button, and the like.

  In addition to the control unit 11, the imaging unit 15, SDRAM 16, image processing unit 17, display driver unit 18, compression / decompression unit 20, reproduction unit 21, and recording unit 22 are connected to the bus 25.

  The imaging unit 15 includes a photographing lens for forming a subject image, an imaging element for converting the subject image into image data, a processing circuit thereof, a diaphragm and a shutter interposed in the optical path of the photographing lens, and the like. including. Image data of still images and continuous images generated by the imaging unit 15 is output to the bus 25.

  The SDRAM 16 is an electrically rewritable volatile temporary storage memory, and is used for temporary storage of still image and continuous image data output from the imaging unit 15.

  The image processing unit 17 performs digital amplification (digital gain adjustment processing) of digital image data, white balance, color correction, gamma (γ) correction, contrast correction, live view display image generation, moving image generation, index image (reduction) Various image processing such as (image) generation is performed.

  The compression / decompression unit 20 is a circuit for compressing still image or continuous image data temporarily stored in the SDRAM 16 by a compression method such as JPEG or TIFF, and decompressing the data for display. Note that image compression is not limited to JPEG, TIFF, and MPEG, and other compression methods can be applied.

  The display driver unit 18 is connected to the display unit 19 and causes the display unit 19 to display an image. The display unit 19 is a display such as a liquid crystal monitor or an organic EL arranged on the back surface of the camera body.

  The reproduction unit 21 reproduces an image stored in the image storage unit 23. In particular, the playback unit 21 reads image data to be displayed on the display unit 19 from the image storage unit 23. The read image data is decompressed by the compression / decompression unit 20 and an image is reproduced and displayed on the display unit 19 based on the decompressed image data.

  The recording unit 22 causes the image storage unit 23 to record the image data compressed by the compression / decompression unit 20. The image storage unit 23 is connected to the playback unit 21 and the recording unit 22 and is a recording medium for image data that can be built in or loaded into the camera body.

  The digital camera 1 can set a stereoscopic shooting mode in which a stereoscopic image is created and displayed as a camera setting. When the stereoscopic shooting mode is set, image processing for generating a stereoscopic image is performed on the live view image for live view display.

  FIG. 2 is a diagram illustrating an example of a live view image. When the stereoscopic shooting mode is set, a live view image is displayed together with a character 31 “SE” (Stereo Effect) indicating that the stereoscopic shooting mode is set, and a layer setting window 32 for setting the number of layer images. Displayed on the unit 19.

  In the layer setting window 32, five types of settings “A”, “M”, “3”, “4”, and “5” are possible. When the user selects “A”, AUTO, that is, the number of layer images is automatically set. For example, the number of layer images is set according to the number of subjects in the image. When the user selects “M”, the user can determine MANUAL, that is, the layer image (number of layer images). When the user selects “3”, “4”, “5”, the number of layer images is set to 3, 4, and 5.

  Hereinafter, a method for generating an image with a stereoscopic effect using the live view image of FIG. 2 will be described.

  FIG. 3 is a diagram for explaining a layer image generation method when the number of layer images is set to five. The layer creation unit 112 detects the contour of each subject from the luminance difference and contrast of the image to be processed, and further specifies the shape of each subject in consideration of information on the color and shape of the subject. This can be done using known methods. In the live view image shown in FIG. 2, subjects p1 to p10 are specified. Here, p1 is a cat, p2 is a person, p3 is a flower, p4 is a tree, p5 is a car, p6 is a house, p7 to p9 are trees, and p10 is a mountain.

  The layer creating unit 112 further creates a layer image for each subject having a common perspective on the image based on the positional relationship between the identified subject shapes. Here, since the number of layer images is set to 5, the layer creating unit 112 creates five layer images. Specifically, five layer images are created based on the overlapping state of each subject and the positional relationship on the image.

  One layer image can include a plurality of subjects having a common perspective. Perspective is determined from the overlapping state of subjects on the image. That is, when a certain subject overlaps with another subject and part of the subject appears to be hidden, it is determined that the subject that is partially hidden is farther from the camera. The sense of perspective can also be determined from the positional relationship on the image. That is, generally, a subject that is far from the camera is positioned at the top of the image, and a subject that is close to the camera is positioned at the bottom of the image. Therefore, it is estimated that the subject located in the upper part of the image is farther from the camera than the subject located in the lower part of the image. Furthermore, the lens position is focused by the image capturing unit 15 at the time of shooting, but the front-rear positional relationship of each subject may be analyzed using this focusing information. Then, the perspective relationship of the subject is determined by combining one or several of these conditions.

  In the example shown in FIG. 2, it is determined that the perspectives of the subjects p2 and p5 do not share from the overlapping state of the subjects p2 and p5, and are included in different layer images. Further, from the overlapping state of the subjects p5 and p6 and the overlapping state of the subjects p4 and p6, the subject p4 and the subject p6 are included in different layer images, and the subject p5 and the subject p6 are included in different layer images. Similarly, the subject p6 and the subject p8 are included in different layer images from the overlapping state of the subjects p6 and p8, and the subject p8 and the subject p10 are included in different layer images from the overlapping state of the subjects p8 and p10.

  The layer creation unit 112 creates five layer images L1, L2, L3, L4, and L5 based on the principle described above. 3A to 3E are diagrams illustrating examples of the layer images L1 to L5.

  As shown in FIG. 3A, the layer image L1 includes subjects p1, p2, and p3 that are considered to have a common sense of perspective. The subjects p1, p2, and p3 are located in the lower part of the image, and are the subjects considered to be the shortest from the camera among the subjects p1 to p10.

  In creating a layer image, a process of trimming a subject that is considered to have a common sense of perspective on the image is performed. This process is a process of cutting out the subject from the image, but when cutting out, it is not cut out along the contour of the subject, but is cut out leaving the border portion. The size of the border portion may be set in advance. If a single layer image includes a plurality of subjects having a common perspective, each subject is cut out so as to include all subjects without being cut out separately. In the example shown in FIG. 3A, the subject is cut out to include the subjects p1, p2, and p3.

  As shown in FIG. 3B, the layer image L2 includes subjects p4 and p5 considered to have a common sense of perspective. Also in this case, the subjects p4 and p5 are cut out so as to include the subjects p4 and p5 without being cut out separately.

  As shown in FIG. 3C, the layer image L3 includes a subject p6.

  As shown in FIG. 3D, the layer image L4 includes subjects p7, p8, and p9 that are considered to have a common sense of perspective. Also in this case, the subjects p7, p8, and p9 are cut out so as to include the subjects p7, p8, and p9 without being cut out separately.

  As shown in FIG. 3E, the layer image L5 includes a subject p10 that is considered to be the farthest from the camera among the subjects p1 to p10. In the present embodiment, it is assumed that the edging process is not performed on the subject p10 that is considered to be the farthest from the camera. However, the edging process may be performed even on a subject that is considered to be the farthest from the camera.

  You may make it add a different color to a border part for every layer image. For example, the border portion 35 of the layer image L1 is red, the border portion 36 of the layer image L2 is blue, the border portion 37 of the layer image L3 is yellow, and the border portion L4 of the layer image L4 is green. In this way, by adding different colors to the border portion of each layer image, the combined image generated by combining the layer images becomes a colorful and fun image. Further, since the difference between the layer images is conspicuous, the stereoscopic effect of the composite image can be further enhanced.

  FIG. 4 is a diagram illustrating an example of a three-dimensional composite image generated by combining the five layer images L1 to L5 illustrated in FIG. The layer synthesis unit 113 synthesizes the five layer images L1 to L5 created by the layer creation unit 112.

  In the composition of layer images, composition is performed in order from the layer image of the subject that is considered to be far from the camera. In the example shown in FIG. 3, the composition is performed in the order of L4, L3, L2, and L1 from the layer image L5 including the subject p10 that is considered to be the farthest from the camera. Therefore, on the combined image, the layer image L5 is positioned at the bottom and the layer image L1 is positioned at the top. Note that the relative positional relationship of each subject does not change between the original captured image and the composite image.

  As shown in FIG. 4, a composite image generated by combining a plurality of layer images including subjects with a common perspective is a three-dimensional image such as a picture book that pops out.

  FIG. 5A is a diagram illustrating an example of a combined image generated by the combining process by the layer combining unit 113 when the number of layer images is nine. When the number of layer images is nine, the layer creating unit 112 creates nine layer images L1 to L9. In the example shown in FIG. 5A, the subject p1 of FIG. 2 is the layer image L1, the subject p2 is the layer image L2, the subject p3 is the layer image L3, the subject p4 is the layer image L4, and the subject p5 is the layer image. A layer image is created so that the subject p6 is included in the layer image L6 at L5. The layer images are created so that the subjects p8 and p9 are included in the layer image L7, the subject p7 is included in the layer image L8, and the subject p10 is included in the layer image L9.

  FIG. 5B is a diagram illustrating an example of a composite image when the number of layer images is five and a process of adding a shadow to a layer image is performed. The layer creation unit 112 creates five layer images by the method described above, and performs a process of adding a shadow to the created layer images. A known method can be used to add a shadow.

  The shadow is added to a layer image including a subject considered to be close to the camera. Also, a larger shadow is added to a layer image that includes a subject that is closer to the camera. In the example shown in FIG. 5B, shadows 51 and 52 are added to the layer images L1 and L2, respectively, and no shadow is added to the layer images L3, L4, and L5. Each of the shadows 51 and 52 represents a shadow generated when light is irradiated from the left direction. Further, the shadow 51 added to the layer image L1 is larger than the shadow 52 added to the layer image L2. Thereby, a three-dimensional effect can be expressed more in the synthesized image generated by synthesizing the layer images L1 to L5.

  In the present embodiment, a display method is performed in which the user's viewpoint changes according to the rotation operation of the digital camera 1 in the left-right direction. This display method will be described with reference to FIGS.

  FIG. 6 is a diagram for explaining the principle of expressing a display effect in which the viewpoint changes. FIG. 6A is a diagram illustrating a state in which three layer images L1, L2, and L3 are overlaid. In FIG. 6A, in order to display that the three images overlap each other in an easy-to-understand manner, the images are not viewed from the front of the images but are viewed from the upper right. Here, the viewpoint viewed from the front of the image is referred to as the front viewpoint, the front viewpoint, the viewpoint from which the image is viewed from the left is referred to as the L viewpoint, the viewpoint from which the image is viewed from the right is referred to as the R viewpoint.

  FIG. 6B is a diagram illustrating an example of a display method for expressing a display effect as seen from the L viewpoint. As shown in FIG. 6B, among the three layer images L1, L2, and L3 superimposed, the middle layer image L2 is shifted to the right with respect to the innermost layer image L3. The layer image L1 located at the forefront is shifted to the right side of the layer image L2. The process of changing the relative position of each layer image is performed by the layer composition unit 113. Also, the layer images L1, L2, and L3 are deformed so as to gradually expand as the vertical direction of the image goes to the left. The layer creation unit 112 performs a process of gradually enlarging the layer images L1, L2, and L3 toward the left side.

  In the present embodiment, the user holding the digital camera 1 rotates the camera in the right direction, that is, the operation of moving the right side of the digital camera 1 to the back (subject side) and the left side to the front (user side). When performed, a display expressing the display effect as seen from the L viewpoint is performed.

  FIG. 6C is a diagram illustrating an example of a display method for expressing the display effect as seen from the R viewpoint. As shown in FIG. 6C, among the three layer images L1, L2, and L3 superimposed, the middle layer image L2 is shifted to the left with respect to the innermost layer image L3. The layer image L1 located at the forefront is shifted to the left side of the layer image L2. The process of changing the relative position of each layer image is performed by the layer composition unit 113. Also, the layer images L1, L2, and L3 are deformed so as to gradually expand as the vertical direction of the image goes to the right side. The layer creation unit 112 performs the process of gradually enlarging the layer images L1, L2, and L3 as they go to the right side.

  In the present embodiment, the user holding the digital camera 1 rotates the camera in the left direction, that is, the operation of moving the left side of the digital camera 1 to the back (subject side) and the right side to the front side (user side). When performed, a display expressing the display effect as seen from the R viewpoint is performed.

  FIG. 7A is a diagram showing a state in which the composite image shown in FIG. 4 is displayed as seen from the L viewpoint. When an operation for rotating the digital camera 1 in the right direction is performed in a state where the composite image shown in FIG. 4 is displayed on the display unit 19, the composite image as shown in FIG. Is displayed. However, in FIG. 7A, for the sake of explanation, arrows 71 to 73 indicating that the layer image has been shifted are also displayed.

  As described above, when displaying as viewed from the L viewpoint, the layer images L1 to L4 are shifted to the right with respect to the layer image L5 located at the innermost position. At this time, the amount of shift to the right side is increased as the layer image is positioned in front. Accordingly, the amount to be shifted to the right is increased in the order of the layer images L4, L3, L2, and L1. In the present embodiment, the left end of the layer image L5 located at the innermost position is made to coincide with the left end of the display screen of the display unit 19. Further, the left side of each of the layer images L1 to L5 is enlarged in the vertical direction of the image.

  In particular, in the present embodiment, the display method is changed according to the angle of the digital camera 1 detected by the angle sensor 14. That is, display is performed so that the display effect as seen from the L viewpoint increases as the rotation angle of the digital camera 1 in the right direction increases. Specifically, as the rotation angle of the digital camera 1 in the right direction increases, the amount by which the layer image is shifted to the right is increased, and the amount by which the left side of each layer image is enlarged in the vertical direction is increased.

  FIG. 7B is a diagram showing a state in which the composite image shown in FIG. 4 is displayed as seen from the R viewpoint. When an operation for rotating the digital camera 1 in the left direction is performed in a state where the composite image shown in FIG. 4 is displayed on the display unit 19, the composite image as shown in FIG. Is displayed. However, in FIG. 7B, for the sake of explanation, arrows 74 to 76 indicating that the layer image has been shifted are also displayed.

  As described above, when displaying as viewed from the R viewpoint, the layer images L1 to L4 are shifted to the left with respect to the layer image L5 located at the innermost position. At this time, the amount of shift to the left is increased as the layer image is positioned closer to the front. Therefore, the amount of shifting to the left increases in the order of the layer images L4, L3, L2, and L1. In the present embodiment, the right end of the layer image L5 located at the innermost position is matched with the right end of the display screen of the display unit 19. Further, the right side of each layer image L1 to L5 is enlarged in the vertical direction of the image.

  Similarly to the case of performing display as viewed from the L viewpoint, display is performed such that the display effect as viewed from the R viewpoint increases as the rotation angle of the digital camera 1 in the left direction increases. Specifically, as the rotation angle of the digital camera 1 in the left direction increases, the amount by which the layer image is shifted to the left is increased, and the amount by which the right side of each layer image is enlarged in the vertical direction is increased.

  In the present embodiment, the display of the composite image is changed according to the tilting operation of the digital camera 1. A method of displaying a composite image according to the tilting operation of the digital camera 1 will be described with reference to FIG.

  FIG. 8A is a diagram illustrating an example of a composite image displayed on the display unit 19 when the digital camera 1 is not tilted. FIG. 8B is a diagram illustrating an example of a composite image displayed on the display unit 19 in a state where the digital camera 1 is tilted to the right.

  As shown in FIG. 8B, when the digital camera 1 is tilted to the right, the layer images L1 to L5 are shifted to the right according to the tilted angle, and the layer images L1 to L5 are shifted in a shifted state. Synthesize. The amount by which each layer image L1 to L5 is shifted to the right side is increased as the tilted angle is increased. Therefore, when the digital camera 1 is gradually tilted to the right, display is performed such that each of the layer images L1 to L5 gradually slides to the right.

  Although not shown, when the digital camera 1 is tilted to the left side, the layer images L1 to L5 are shifted to the left side according to the tilted angle, and a process of combining is performed. The amount by which each of the layer images L1 to L5 is shifted to the left side is increased as the tilted angle is increased. Accordingly, when the digital camera 1 is gradually tilted to the left, display is performed such that each of the layer images L1 to L5 gradually slides to the left.

  Further, when the digital camera 1 is returned from the tilted state to the non-tilted state, a process of returning each of the layer images L1 to L5 to the original position (see FIG. 8A) is performed.

  A method for the user to specify subjects to be included in the layer image when the layer setting window 32 is set to “M” (Manual) will be described. FIG. 9 is a diagram illustrating an example of a screen for the user to specify subjects to be included in the layer image. When the layer setting window 32 is set to “M”, a cursor 91 is displayed on the screen. The user can designate a subject to be included in the layer image by operating a cursor 91 displayed on the screen. FIG. 9 shows a state in which a person's subject 92 and a mountain subject 93 are respectively designated, and are assigned consecutive white numbers.

  The layer creation unit 112 creates the subject specified by the cursor 91 as a separate layer image. A subject not specified is automatically included in the same layer as a subject specified in the vicinity thereof.

  Note that the method of specifying the subject to be included in the layer image is not limited to the method using the cursor. For example, a touch panel may be provided in front of the display unit 19 or integrally with the display unit 19, and a subject to be included in the layer image may be designated by operating the touch panel.

  FIG. 10 is a flowchart showing the contents of processing performed by the digital camera 1 provided with the composite image creation device 5 in one embodiment. When the digital camera 1 is set to a shooting mode for shooting, the control unit 11 starts the process of step S11.

  In step S11, it is determined whether or not the digital camera 1 is set to the stereoscopic shooting mode. If it is determined that the digital camera 1 is not set to the stereoscopic shooting mode, the process proceeds to step S19.

  In step S19, a normal live view display process is performed. That is, the live view image is displayed on the display unit 19.

  On the other hand, if it is determined in step S11 that the digital camera 1 is set to the stereoscopic shooting mode, the process proceeds to step S12. In step S12, a stereoscopic live view display process is performed. Detailed contents of the stereoscopic live view display process will be described with reference to the flowchart shown in FIG.

  In step S31 of FIG. 11, it is determined whether or not the layer setting is set to manual, that is, whether or not the layer setting window 32 (see FIG. 2) is set to “M”. If it is determined that the layer setting is not set to manual, the process proceeds to step S32. If it is determined that the layer setting is set to manual, the process proceeds to step S40.

  In step S32, the number of layer images is set. When the user selects a number such as “3”, “4”, “5” in the layer setting window 32 (see FIG. 2), the selected number is set as the number of layer images. When the user selects “A” in the layer setting window 32, the number of layer images is arbitrarily set. For example, the subject on the image may be extracted, and the number of extracted subjects may be set as the number of layer images, and as described above, a plurality of subjects having a common sense of perspective may be included in one layer image. The number of layer images may be set.

  On the other hand, in step S40, it is determined whether or not the input operation for designating the layer image has been completed. As described above, the user can designate a subject to be included in a layer image, that is, a layer image, by operating the cursor 91 displayed so as to be superimposed on the image. When the user finishes specifying the layer image, the user presses a confirmation button (not shown). If it is determined that the confirm button has not been pressed and the input operation for specifying the layer image has not been completed, the process waits in step S40. If the confirm button has been pressed, it is determined that the input operation for specifying the layer image has been completed. Then, the process proceeds to step S33.

  In step S33, the layer creation unit 112 creates a layer image. Since the method of creating a layer image has already been described, detailed description thereof is omitted here.

  In step S34, it is determined whether or not a rotation operation in the left-right direction of the digital camera 1 has been detected. This determination is performed by the inclination detection unit 114 of the control unit 11 based on the angle of the digital camera 1 detected by the angle sensor 14. If it is determined that a rotation operation in the left-right direction of the digital camera 1 is detected, the process proceeds to step S41. If it is determined that a rotation operation is not detected, the process proceeds to step S35.

  In step S35, the viewpoint is set as a default. The default viewpoint is the front viewpoint (see FIG. 6A).

  In step S36, the layer compositing unit 113 generates a composite image having a stereoscopic effect by combining the plurality of layer images created in step S33.

  On the other hand, in step S <b> 41, the tilt detection unit 114 detects the rotation direction and rotation angle of the digital camera 1 based on the angle input from the angle sensor 14.

  In step S42, the layer composition unit 113 creates in step S33 so that a composite image in which the viewpoint of the user is changed according to the rotation direction and rotation angle of the digital camera 1 detected in step S41 is generated. Deform and combine multiple layer images.

  As described above, when a rotation operation in the right direction of the digital camera 1 is detected, a composite image (see FIG. 7A) as seen from the L viewpoint is generated. At this time, a composite image is generated so that the display effect as seen from the L viewpoint increases as the rotation angle of the digital camera 1 in the right direction increases.

  When a leftward rotation operation of the digital camera 1 is detected, a composite image (see FIG. 7B) as seen from the R viewpoint is generated. At this time, a composite image is generated so that the display effect as seen from the R viewpoint increases as the rotation angle of the digital camera 1 in the left direction increases.

  In step S37, the composite image generated in step S36 or step S42 is displayed on the display unit 19 as a live view image. As a result, a live view image with a stereoscopic effect is displayed on the display unit 19.

  In step S38, it is determined whether or not the tilt operation of the digital camera 1 has been detected. This determination is performed by the inclination detection unit 114 of the control unit 11 based on the angle of the digital camera 1 detected by the angle sensor 14. If it is determined that the tilt operation of the digital camera 1 has been detected, the process proceeds to step S44. If it is determined that the tilt operation has not been detected, the process returns to the process of the flowchart shown in FIG.

  In step S44, the layer composition unit 113 shifts the layer images in the tilted direction according to the tilt angle of the digital camera 1, and combines the layer images in the shifted state. Then, the generated composite image is displayed on the display unit 19 as a live view image.

  Returning to the flowchart shown in FIG. In step S13, it is determined whether or not the release button has been pressed by the user. If it is determined that the release button has not been pressed by the user, the process waits in step S13. If it is determined that the release button has been pressed, the process proceeds to step S14.

  In step S14, a photographing process is performed in response to the pressing operation of the release button. This photographing process is a process for photographing a still image.

  In step S15, the image data (layer combined image in the case of the stereoscopic shooting mode) obtained by the shooting process in step S14 is recorded in the image storage unit 23.

  In step S16, REC view display is performed in which the image obtained by the photographing process in step S14 is displayed on the display unit 19 for only a short time.

  In step S17, it is determined whether or not shooting has been completed. For example, when the user performs a power-off operation or an image reproduction operation, it is determined that shooting has ended, and the process of the flowchart ends. On the other hand, if it is determined that the shooting has not ended, the process returns to step S11.

  As described above, according to the composite image creating apparatus in one embodiment, a subject having a common perspective on the image is trimmed and cut into a single layer image, and a plurality of layer images are obtained according to the perspective of the subject. By creating and synthesizing the created layer images, a synthesized image having a stereoscopic effect is created. As a result, an image having a stereoscopic effect can be created from a normal two-dimensional image without requiring complicated image processing.

  Since the layer creation unit 112 creates the designated number of layer images, the user can change the stereoscopic effect on the composite image by designating the number of layer images.

  The layer creating unit 112 creates a plurality of layer images so that the designated subject is included in each of the different layer images. Thereby, the user can change the stereoscopic effect on the composite image by designating the subject to be included in the layer image.

  In particular, the layer creation unit 112 determines the perspective based on how the subjects on the image overlap, and creates each layer image. Therefore, an image with a three-dimensional effect can be obtained without acquiring subject distance information. Can be created.

  According to the composite image creation device in one embodiment, the layer creation unit 112 creates a layer image corresponding to the set viewpoint, and the layer synthesis unit 113 compares the relative image of each layer image according to the set viewpoint. Set the position and synthesize multiple layer images at the set relative position. Thereby, it is possible to express a display effect in which the viewpoint changes according to the set viewpoint.

  In addition, by displaying the three-dimensional synthesized image generated by the layer synthesizing unit 113 on the display unit 19 of the camera 1 as a live view image, an appropriate angle of view and subject can be set.

  Furthermore, since the relative position of each layer image is set according to the tilt of the camera 1 and a plurality of layer images are combined at the set relative position, the composition is impressive and interesting according to the tilt of the camera 1. Images can be created.

  In the above embodiment, it has been described that a layer composite image is created and recorded at the time of shooting. However, at the time of playback, a layer composite image may be created and recorded from image data that has already been shot. This is because the captured image data is equivalent to the live view image at the time of shooting in terms of image data.

  In the description of the embodiment described above, hardware processing is assumed as processing performed by the composite image creation apparatus, but it is not necessary to be limited to such a configuration. For example, a configuration in which software processing is performed by a computer is also possible. In this case, the computer includes a main storage device such as a CPU and a RAM, and a computer-readable storage medium storing a program for realizing all or part of the above processing. Here, this program is called a composite image creation program. Then, the CPU reads the composite image creation program stored in the storage medium and executes information processing / calculation processing, thereby realizing the same processing as that of the above-described composite image creation apparatus.

  Here, the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, and the like. Alternatively, the composite image creation program may be distributed to a computer via a communication line, and the computer that has received the distribution may execute the composite image creation program.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components of all the components shown by embodiment.

  In the above-described embodiment, a camera equipped with a composite image creation device has been described as a digital camera. However, a video camera or a movie camera may be used, and a mobile phone, a personal digital assistant (PDA), or a game device may be used. It may be a camera built in the camera.

  The layer creating unit 112 detects the contour of each subject from the luminance difference and contrast of the image, further specifies the shape of each subject in consideration of information on the color and shape of the subject, and A layer image was created based on the positional relationship. However, the subject distance of each subject may be detected at the time of shooting, and a layer image may be created based on the detected subject distance.

  In the above-described embodiment, display is performed such that the display effect as viewed from the L (R) viewpoint increases as the rotation angle of the digital camera 1 in the right (left) direction increases. As described above, display may be performed such that the display effect as seen from the L (R) viewpoint is increased in multiple stages.

  In the embodiment described above, a stereoscopic composite image is displayed as a live view image during live view display set in the stereoscopic shooting mode. However, as described above, when a still image is reproduced, the stereoscopic effect is displayed. A composite image may be reproduced and displayed.

DESCRIPTION OF SYMBOLS 11 ... Control part 14 ... Angle sensor 15 ... Imaging part 19 ... Display part 111 ... Display control part 112 ... Layer preparation part 113 ... Layer composition part 114 ... Inclination detection part

Claims (8)

A layer creation unit that creates a single layer image by trimming and cropping subjects with a common perspective on the image, and creates a plurality of layer images according to the perspective of the subject,
A layer compositing unit that creates a three-dimensional composite image by compositing a plurality of layer images created by the layer creating unit;
A composite image creating apparatus comprising: The layer creation unit creates a specified number of layer images.
The composite image creating apparatus according to claim 1. The layer creating unit creates a plurality of layer images so that the specified subject is included in each of the different layer images;
The composite image creating apparatus according to claim 1, wherein the composite image creating apparatus is a composite image creating apparatus according to claim 1. The layer creation unit judges the perspective based on how the subjects on the image overlap, and creates each layer image,
The composite image creating apparatus according to claim 1, wherein the composite image creating apparatus is a composite image creating apparatus according to claim 1. The layer creation unit creates a layer image corresponding to the set viewpoint,
The layer composition unit sets the relative position of each layer image according to the set viewpoint, and synthesizes the layer image created by the layer creation unit at the set relative position.
The composite image creation apparatus according to claim 1, wherein the composite image creation apparatus is a composite image creation apparatus. The composite image creation device according to any one of claims 1 to 5,
A display for displaying an image;
A display control unit for displaying a live view image on the display unit during shooting;
With
The display control unit causes the display unit to display a three-dimensional composite image created by the layer composition unit as the live view image.
An imaging apparatus characterized by that. A tilt detection unit for detecting the tilt of the imaging device;
The layer composition unit sets a relative position of each layer image according to the inclination of the imaging device detected by the inclination detection unit, and the layer image created by the layer creation unit is set at the set relative position. To synthesize,
The imaging apparatus according to claim 6. Creating a single layer image by trimming and cropping subjects with a common perspective on the image, and creating a plurality of layer images according to the perspective of the subject;
Creating a three-dimensional composite image by synthesizing the created plurality of layer images;
A composite image creation program for causing a computer to execute.