JP2014135784A - Imaging apparatus, imaging method and imaging program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of presenting a user with information to be a guideline for selecting the combination of special effects when generating a composite image by individually performing special effects on the respective data of two images.SOLUTION: In an imaging apparatus 1 capable of performing a special effect on an image displayed by a display unit 7, a special effect image generator 11c performs a special effect on one of a subject image and a background image. When a composite image generator 11d synthesizes the special effect image having the performed special effect with the other one of the subject image and the background image, a display controller 11e controls the display unit 7 to display the synthesized composite image. Also, by referring to a special effect information table stored in a special effect information storage 9d, the display controller 11e controls the display unit 7 to display information related to the special effect to be performed on the subject image or the background image, in order from the highest recommendation degree to the lowest, according to combination with the special effect performed immediately before.

Description

  The present invention relates to an imaging apparatus, an imaging method, and an imaging program that shoot a subject and generate electronic image data.

  In recent years, in an imaging apparatus such as a digital camera or a digital video, there has been known a technique that meets various demands of consumers by applying special effects to image data obtained by imaging a subject. For example, a technique is known in which photographed image data can be recorded with a special effect such as wipe or sepia (see Patent Document 1). In this technology, the user can display an image on which a special effect selected when the user operates an effect switch for selecting a special effect on a part of the image displayed on the display monitor. It is possible to easily select a desired special effect.

  Also, the same subject is shot multiple times, and the subject image and the background image included in the image are separated using the captured images, and then the background image is blurred and synthesized again with the subject image. There is known an imaging apparatus capable of performing (see Patent Document 2).

JP-A-2005-347886 Japanese Patent No. 4321287 Japanese Patent No. 4464079

  By the way, in the conventional imaging device, when a composite image is generated using two image data, a composite image is not generated by combining different special effects with each of the two image data. Therefore, it is conceivable to apply different special effects to each of the two image data. However, not all combinations of different special effects are good, and there are combinations in which the effects are canceled out, and information that serves as a guide for selecting the combination has been required.

  The present invention has been made in view of the above, and information that serves as a guideline for selecting a combination of special effects when a composite image is generated by performing special effects individually on two pieces of image data. An object is to provide an imaging apparatus, an imaging method, and an imaging program that can be presented to a user.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention images a subject, generates electronic image data of the subject, and generates an image corresponding to the generated image data. An imaging apparatus having a display unit for displaying and capable of applying a special effect to the image,
An image separation unit that separates a subject image and a background image from the image displayed by the display unit, and an input of information regarding special effects to be applied to the subject image and the background image separated by the image separation unit, respectively. An input unit; a special effect image generation unit that generates a special effect image of each image by individually applying a special effect according to information received by the input unit to the subject image and the background image; and the special effect A composite image generation unit that generates a composite image using the special effect image generated by the image generation unit; and the display unit when the image separation unit cannot separate the subject image and the background image from the image. And a display control unit for displaying a warning on the unit.

  The imaging apparatus according to the present invention is characterized in that, in the above-mentioned invention, the background image can set a plurality of aspect ratios, and the input unit has an operation switch for instructing the aspect ratio of the background image. And

  In the imaging device according to the present invention, in the above invention, the display control unit is combined with the special effect applied most recently on the composite image generated by the composite image generation unit displayed by the display unit. Only the information related to the special effects that can be displayed is superimposed on the display unit.

  In the imaging device according to the present invention, in the above invention, the display control unit displays a plurality of menu icons in the imaging device on the display unit, and the input unit is provided on a display screen of the display unit. And a touch panel that detects a position where an object that comes into contact with the outside comes into contact and receives an input of a signal corresponding to the menu icon displayed at the detected position.

  In addition, an imaging method according to the present invention includes a display unit that captures an image of a subject, generates electronic image data of the subject, and displays an image corresponding to the generated image data. An imaging method executed by an imaging apparatus capable of applying an effect, the image separation step separating a subject image and a background image from the image displayed by the display unit, and the subject separated in the image separation step An input step for receiving information on special effects to be applied to the image and the background image respectively, and a special effect corresponding to the information received in the input step is individually applied to the subject image and the background image A special effect image generating step for generating a special effect image of the image, and the special effect generated in the special effect image generating step A composite image generation step for generating a composite image using an image, and a display control step for displaying a warning on the display unit when the subject image and the background image cannot be separated from the image in the image separation step. It is characterized by including these.

  The program according to the present invention includes a display unit that captures an image of a subject, generates electronic image data of the subject, and displays an image corresponding to the generated image data. An image separation step of separating the subject image and the background image from the image displayed by the display unit, and the subject image and the background image separated in the image separation step. An input step for receiving information on special effects to be applied, and a special effect corresponding to the information received in the input step are individually applied to the subject image and the background image to generate a special effect image for each image A special effect image generation step, and a synthesized image using the special effect image generated in the special effect image generation step. When the synthesized image generation step to be performed and the subject image and the background image cannot be separated from the image in the image separation step, a warning is displayed on the display unit and a display control step is executed. Features.

  According to the present invention, the special effect image generation unit applies a special effect to one of the subject image and the background image, and the special effect image to which the special effect is applied is combined with the other of the subject image and the background image by the image composition unit. When the images are combined, the display control unit displays the combined image on the display unit and refers to the special effect information table stored in the storage unit, and information on the special effect to be applied to the subject image or the background image Are displayed on the display unit in descending order of the recommended degree according to the combination with the special effect applied most recently. Thus, the user can intuitively recognize information serving as a guideline for selecting a combination of special effects when generating a composite image by performing special effects individually on each of the two image data. There is an effect.

FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a perspective view showing the configuration of the front side of the imaging apparatus according to Embodiment 1 of the present invention. FIG. 3 is a perspective view showing the configuration of the back side of the imaging apparatus according to Embodiment 1 of the present invention. FIG. 4 is a schematic diagram illustrating a situation when the imaging unit included in the imaging device according to Embodiment 1 of the present invention generates two image data in which one end in the left-right direction of each field of view overlaps. . FIG. 5 is a diagram illustrating an example of an image corresponding to the image data generated by the imaging unit under the situation illustrated in FIG. FIG. 6 is a diagram illustrating an example of an image corresponding to image data generated by the imaging unit under the situation illustrated in FIG. FIG. 7 is a diagram illustrating an example of an image obtained by virtually superimposing images generated at different shooting positions by the imaging unit under the situation illustrated in FIG. 4. FIG. 8 is a diagram illustrating an example of a background image generated by the image separation unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 9 is a diagram illustrating an example of a subject image generated by the image separation unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 10 is a schematic diagram for explaining an overview of processing performed by the special effect image generation unit and the composite image generation unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 11 is a flowchart showing an outline of processing performed by the imaging apparatus according to Embodiment 1 of the present invention. FIG. 12 is a flowchart showing an overview of menu operation processing performed by the imaging apparatus according to Embodiment 1 of the present invention. FIG. 13 is a diagram illustrating an example of an image displayed by the display unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 14 is a diagram illustrating an example of an image displayed on the display unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 15 is a diagram illustrating an example of an image displayed by the display unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 16 is a diagram illustrating an example of the special effect information table stored in the special effect information storage unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 17 is a flowchart illustrating an overview of the imaging process performed by the imaging apparatus according to Embodiment 1 of the present invention. FIG. 18 is a flowchart showing an outline of the swing shooting process performed by the imaging apparatus according to Embodiment 1 of the present invention. FIG. 19 is a diagram illustrating an example of an image displayed by the display unit included in the imaging apparatus according to Embodiment 1 of the present invention. FIG. 20 is a schematic diagram illustrating a panorama shooting method in a shooting mode performed by the imaging apparatus according to Embodiment 2 of the present invention. FIG. 21 is a schematic diagram illustrating an HDR imaging method in an imaging mode performed by the imaging apparatus according to Embodiment 2 of the present invention. FIG. 22 is a diagram illustrating an example of an image generated by the imaging device under the situation illustrated in FIG. FIG. 23 is a flowchart illustrating an overview of menu operation processing performed by the imaging apparatus according to Embodiment 2 of the present invention. FIG. 24 is a diagram illustrating an example of an image displayed on the display unit included in the imaging apparatus according to Embodiment 2 of the present invention. FIG. 25 is a diagram illustrating an example of an image displayed on the display unit included in the imaging apparatus according to Embodiment 2 of the present invention. FIG. 26 is a diagram illustrating an example of the special effect information table stored in the special effect information storage unit included in the imaging apparatus according to Embodiment 2 of the present invention. FIG. 27 is a diagram illustrating an example of an image displayed by the display unit included in the imaging apparatus according to Embodiment 2 of the present invention. FIG. 28 is a diagram illustrating an example of the special effect information table stored in the special effect information storage unit included in the imaging apparatus according to Embodiment 2 of the present invention. FIG. 29 is a diagram illustrating an example of an image displayed on the display unit included in the imaging apparatus according to Embodiment 2 of the present invention. FIG. 30 is a diagram illustrating an example of a composite image generated by the composite image generation unit included in the imaging apparatus according to Embodiment 2 of the present invention.

  DESCRIPTION OF EMBODIMENTS Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described with reference to the drawings. The present invention is not limited to the embodiments described below. In the description of the drawings, the same parts are denoted by the same reference numerals.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a diagram illustrating a configuration of the imaging device according to Embodiment 1 on the side facing the subject (front side). FIG. 3 is a diagram illustrating a configuration of a side (back side) facing the user of the imaging apparatus according to the first embodiment.

  As shown in FIGS. 1 to 3, the imaging device 1 captures a subject, generates an electronic image data of the photographed subject, and an attitude detection unit 3 that detects acceleration of the imaging device 1. , A strobe 4 that irradiates light to a field of view imaged by the imaging unit 2, a clock 5 having a shooting date determination function and a timer function, an operation input unit 6 that receives input of various information of the imaging device 1, and an imaging unit 2 stores a display unit 7 that displays an image corresponding to the image data generated by 2, a touch panel 8 that receives an input of a signal according to an external contact position, and various types of information including image data generated by the imaging unit 2. Storage unit 9, a communication unit 10 that performs transmission / reception to / from various databases via the Internet, and a control unit 11 that controls the operation of the imaging apparatus 1.

  The imaging unit 2 includes a lens unit 2a, a lens driving mechanism 2b, a diaphragm 2c, a diaphragm driving mechanism 2d, a shutter 2e, a shutter driving mechanism 2f, an imaging element 2g, and a signal processing unit 2h. The lens unit 2a is composed of a focus lens, a zoom lens, and the like, and collects light from a predetermined visual field region. The lens driving mechanism 2b is configured by a DC motor or the like, and changes the focus position, the focal length, and the like of the lens unit 2a by moving the focus lens, the zoom lens, and the like of the lens unit 2a on the optical axis L1. The diaphragm 2c adjusts the exposure by limiting the amount of incident light collected by the lens unit 2a. The aperture drive mechanism 2d is configured by a stepping motor or the like, and drives the aperture 2c. The shutter 2e sets the state of the image sensor 2g to an exposure state or a light shielding state. The shutter drive unit 2f is configured by a stepping motor or the like, and drives the shutter 2e according to a release signal. The imaging device 2g is realized by a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like that receives the light collected by the lens unit 2a and converts it into an electrical signal (analog signal). The signal is output to the signal processing unit 2h. The signal processing unit 2h performs signal processing such as amplification on the electrical signal output from the image sensor 2g, and then performs A / D conversion to convert it into digital image data and outputs the digital image data to the control unit 11.

  The posture detection unit 3 is configured by an acceleration sensor, and detects the posture state and / or movement of the imaging device 1 by detecting the acceleration of the imaging device 1. Specifically, the posture detection unit 3 detects the posture of the imaging device 1 with respect to the horizontal plane.

  The strobe 4 is configured by a xenon lamp, an LED (Light Emitting Diode), or the like. The strobe 4 irradiates strobe light that is auxiliary light toward the field of view of the imaging unit 2.

  The timepiece 5 generates a time signal that serves as a reference for the operation of the imaging apparatus 1. Thereby, the control part 11 can set the acquisition time of image data, the exposure time of the image pick-up element 2g, etc.

  The operation input unit 6 switches between a power switch 6a for switching the power state of the imaging apparatus 1 to an on state or an off state, a release switch 6b for inputting a release signal for giving a shooting instruction, and various shooting modes of the imaging apparatus 1. A shooting mode switch 6c, an operation switch 6d for switching various settings of the imaging device 1, a menu switch 6e for displaying information on various shooting modes of the imaging device 1 on the display unit 7, and a zoom for performing a zoom operation of the imaging unit 2 And a switch 6f.

  The display unit 7 is realized using a display panel made of liquid crystal, organic EL (Electro Luminescence), or the like. The display unit 7 displays an image corresponding to the image data generated by the imaging unit 2. The display unit 7 appropriately displays operation information of the imaging apparatus 1 and information related to shooting. The display unit 7 displays a three-dimensional image (hereinafter referred to as “3D image”) corresponding to the image data generated by the imaging unit 2. In general, a display unit that displays a 3D image includes a parallax barrier method or a lenticular lens method. In this embodiment mode, any type of display unit is applicable.

  The touch panel 8 is provided so as to overlap the display screen of the display unit 7 (see FIG. 3). The touch panel 8 detects a position touched by the user based on information displayed on the display unit 7 and accepts an input of an operation signal corresponding to the contact position. In general, the touch panel includes a resistance film method, a capacitance method, an optical method, and the like. In the present embodiment, any type of touch panel is applicable. In the first embodiment, the touch panel 8 functions as an input unit.

  The storage unit 9 stores an image data storage unit 9a that stores image data captured by the imaging unit 2, a program storage unit 9b that stores various programs executed by the imaging device 1 and the imaging program according to the first embodiment, According to a combination of the temporary image data storage unit 9c that temporarily stores the image data captured by the imaging unit 2 or the settings of the various imaging devices 1, and the special effect applied to the subject image and the background image when performing the special effect. A special effect information storage unit 9d for storing a special effect information table for giving a recommended degree. The storage unit 9 is realized using a semiconductor memory such as a flash memory or a RAM (Random Access Memory) that is fixedly provided inside the imaging apparatus 1. The storage unit 9 may be provided with a function as a recording medium interface that reads information stored in the storage medium while storing information in a storage medium such as a memory card mounted from the outside.

  The communication unit 10 has a function as a communication interface, and transmits and receives to and from an external processing device such as a server (not shown) or a personal computer via a network (not shown). The communication unit 10 acquires data related to various program data and special effect information table of the imaging device 1 by performing transmission and reception with the external processing device. The communication unit 10 is connected to a network via a wired or wireless LAN (Local Area Network).

  The control unit 11 is realized by a CPU (Central Processing Unit) or the like. The control unit 11 reads out and executes a program from the program storage unit 9b of the storage unit 9 in response to an operation signal from the operation input unit 6 and performs instructions and data transfer to each unit constituting the imaging device 1. The operation of the image pickup apparatus 1 is comprehensively controlled. The control unit 11 includes an image processing unit 11a, an image separation unit 11b, a special effect image generation unit 11c, a composite image generation unit 11d, and a display control unit 11e.

  The image processing unit 11a performs various types of image processing on the image data output from the signal processing unit 2h and outputs the processed image data to the storage unit 9. Specifically, the image processing unit 11a performs processing such as edge enhancement, color correction, and γ correction on the image data output from the signal processing unit 2h.

  The image separation unit 11b separates the subject image and the background image from the image corresponding to the image data generated by the imaging unit 2. Specifically, the image separation unit 11b uses the well-known triangulation principle and corresponds to two pieces of image data corresponding to two pieces of image data generated by the imaging unit 2 in which one end portion in the left-right direction overlaps each other. The subject image and the background image are separated based on the parallax of the two images.

  The special effect image generation unit 11c generates a special effect image for each image by individually applying a special effect corresponding to the information received by the touch panel 8 to the subject image and the background image separated by the image separation unit 11b. To do. Specifically, the special effect image generation unit 11c arranges various optical filters such as a color conversion filter and a contrast adjustment filter in front of the lens unit 2a of the imaging unit 2 for each of the subject image and the background image. A special effect image obtained by applying a special effect obtained in a case or a special effect obtained when a photographing lens for a specific purpose such as a fish-eye lens is disposed is generated. The special effect image generation unit 11c may generate a special effect image by applying a special effect to one image corresponding to the image data generated by the imaging unit 2. Further, special effect images may be successively generated by sequentially applying special effects to the image data continuously generated by the imaging unit 2.

  The composite image generation unit 11d generates a composite image using the special effect image generated by the special effect image generation unit 11c. Specifically, the composite image generation unit 11d combines the subject image separated by the image separation unit 11b with the special effect image generated by the special effect image generation unit 11c applying the special effect to the background image, and generates the composite image. Generate.

  The display control unit 11 e controls display on the display unit 7. In the display control unit 11e, the special effect image generation unit 11c applies a special effect to one of the subject image and the background image, and the special effect image to which the special effect is applied is converted into a subject image and a background image by the composite image generation unit 11d. When the image is combined with the other, the combined image is displayed on the display unit 7, and the special effect information table stored in the special effect information storage unit 9d is referred to and the special image to be applied to the subject image or the background image. Information on the effect is displayed on the display unit 7 in descending order of recommendation according to the combination with the special effect applied most recently. The display control unit 11e displays information on the special effect on the display unit 7 using icons. Specifically, the display control unit 11e displays the special effect icons corresponding to the special effect information in descending order of recommendation according to the combination with the special effect most recently applied with reference to the special effect information table. 7 is displayed.

  In the imaging apparatus 1 having the above configuration, a situation when the image separation unit 11b separates an image generated by the imaging unit 2 into a subject image and a background image will be described. FIG. 4 is a schematic diagram illustrating a situation when the imaging unit 2 generates two image data in which one end in the horizontal direction of each field of view overlaps. FIG. 5 is a diagram illustrating an example of an image corresponding to the image data generated by the imaging unit 2 at the shooting position P1 under the situation illustrated in FIG. FIG. 6 is a diagram illustrating an example of an image corresponding to the image data generated by the imaging unit 2 at the shooting position P2 under the situation illustrated in FIG. FIG. 7 is a diagram illustrating an example of an image obtained by virtually superimposing images respectively generated at different shooting positions by the imaging unit 2 under the situation illustrated in FIG. In FIG. 7, an image W1 is an image superimposed on the background, and an image W2 is an image superimposed on the subject. In FIG. 4, the user moves the imaging device 1 from the left side to the right side, and the imaging device 1 continuously shoots at minute time intervals during this movement period. For this reason, a plurality of images exist between the image WL1 and the image WR1 shown in FIGS.

  As shown in FIG. 4, the imaging unit 2 continuously captures images of a subject A1 (distance d1) and a subject A2 (distance d2) having different distances from the imaging unit 2 at a plurality of shooting positions. A plurality of image data having different areas are generated. Specifically, as illustrated in FIGS. 5 and 6, the imaging unit 2 generates an image WL1 at the shooting position P1 in FIG. 4 and generates an image WR1 at the shooting position P2 in FIG. As shown in FIG. 4, since the distance d2 between the imaging unit 2 and the subject A2 is larger than the distance d1 between the imaging unit 2 and the subject A1, the area of the subject A2 that is the background almost overlaps. Specifically, as shown in FIG. 7, the area of the subject A2 almost overlaps in the image W1. On the other hand, in the image W1, the area of the subject A1 does not overlap and has a parallax a1 with respect to the subject A1.

  As described above, as illustrated in FIG. 7A, the image separation unit 11b identifies an overlapping region as a background image when the image WL1 and the image WR1 are virtually overlapped. In addition, as shown in FIG. 7B, the image separation unit 11b overlaps by moving the image WL1 and the image WR1 in different directions and superimposing the regions where the luminance of the image WL1 and the image WR1 is high as a reference. The region is specified as the subject image of the subject A1. As a result, the image separation unit 11b can separate the background image W3 and the subject image W4 from the image corresponding to the image data generated from the imaging unit 2, as shown in FIGS. When the image WL1 and the image WR1 are virtually overlapped with the subject A1 as a reference, the image separation unit 11b identifies the subject image by superimposing the regions with high contrast between the image WL1 and the image WR1. Also good.

  Next, an outline of processing by the special effect image generation unit 11c and the composite image generation unit 11d will be described. FIG. 10 is a schematic diagram for explaining an overview of processing by the special effect image generation unit 11c and the composite image generation unit 11d.

  As shown in FIG. 10, first, a special effect set by the user, for example, a “soft effect” described later is applied to the background image W10 stored in the special effect image generation unit 11c and the image data storage unit 9a, and the special effect image W11. (FIG. 10 (a) → FIG. 10 (b)). Thereafter, the composite image generation unit 11d generates the composite image W12 by combining the subject image W4 separated by the image separation unit 11b with the special effect image W11 (FIG. 10 (b) → FIG. 10 (c)). Finally, the special effect image generation unit 11c applies a special effect set by the user to the subject image W4 included in the composite image W12, for example, a “sepia effect” described later to obtain a composite image W13 (FIG. 10C). (FIG. 10 (d)). Thereby, the imaging device 1 can generate a composite image in which a special effect is applied to each of the subject image and the background image. In FIG. 10, an image subjected to the “soft effect” is represented by being blurred, and an image subjected to the “sepia effect” is represented by hatching.

  Next, processing performed by the imaging apparatus 1 according to Embodiment 1 will be described. FIG. 11 is a flowchart illustrating an outline of processing performed by the imaging apparatus 1.

  In FIG. 11, first, the control unit 11 determines whether or not the power of the imaging device 1 is turned on (step S <b> 101). When the power supply of the imaging device 1 is on (step S101: Yes), the imaging device 1 proceeds to step S102. On the other hand, when the power of the imaging apparatus 1 is not turned on (step S101: No), the imaging apparatus 1 ends this process.

  Subsequently, the control unit 11 determines whether or not the imaging device 1 is set to the shooting mode (step S102). When the imaging device 1 is set to the shooting mode (step S102: Yes), the imaging device 1 proceeds to step S103 described later. On the other hand, when the imaging device 1 is not set to the imaging mode (step S102: No), the imaging device 1 proceeds to step S108 described later.

  A case where the imaging apparatus 1 is set to the shooting mode in step S102 (step S102: Yes) will be described. In this case, the display control unit 11e causes the display unit 7 to display a through image of an image corresponding to the image data continuously generated by the imaging unit 2 at a constant minute time interval (step S103).

  Subsequently, the control unit 11 determines whether or not a release signal instructing photographing is input by operating the release switch 6b by the user (step S104). When the release signal instructing photographing is not input (step S104: No), the control unit 11 determines whether or not the menu switch 6e is operated by the user (step S105). When the menu switch 6e is not operated by the user within a predetermined time, for example, 30 seconds (step S105: No), the imaging device 1 returns to step S101. On the other hand, when the menu switch 6e is operated by the user (step S105: Yes), the imaging device 1 executes menu operation processing for setting a menu included in the imaging mode (step S106), and the imaging device 1 Return to step S101.

  A case where a release signal instructing photographing is input in step S104 (step S104: Yes) will be described. In this case, the imaging device 1 executes a shooting process for shooting the subject (step S107), and the imaging device 1 returns to step S101.

  In step S102, the case where the imaging device 1 is not in the shooting mode (step S102: No) will be described. In this case, the display control unit 11e causes the display unit 7 to display an image selection screen that collectively displays a plurality of images stored in the image data storage unit 9a (step S108).

  Subsequently, the control unit 11 determines whether an image is selected from the image selection screen displayed on the display unit 7 when the user touches the touch panel 8 (step S109). When an image is not selected from the image selection screen by the user (step S109: No), the imaging apparatus 1 proceeds to step S113 described later. On the other hand, when an image is selected from the image selection screen by the user (step S109: Yes), the display control unit 11e causes the display unit 7 to display the image selected by the user on the full screen (step S110) and capture the image. The apparatus 1 moves to step S111.

  In step S111, it is determined whether an image switching operation has been performed by the user. When the user performs an image switching operation (step S111: Yes), the display control unit 11e switches the image currently displayed on the display unit 7 (step S112), and the imaging apparatus 1 proceeds to step S110. Return. On the other hand, when the image switching operation is not performed by the user (step S111: No), the control unit 11 determines whether or not the image reproduction end operation is performed (step S113). Specifically, the control unit 11 determines whether or not the imaging device 1 has been switched from the playback mode to the shooting mode by operating the shooting mode switch 6c by the user. When the user has not performed the image reproduction end operation (step S113: No), the imaging apparatus 1 returns to step S108. On the other hand, when the user performs an operation for ending image reproduction (step S113), the imaging apparatus 1 returns to step S101.

  Next, the menu operation process in step S106 in FIG. 11 will be described. FIG. 12 is a flowchart showing an outline of menu operation processing.

  In FIG. 12, first, the display control unit 11e displays a menu icon on the image displayed on the display unit 7 (step S201). Specifically, as shown in FIG. 13, the display control unit 11e displays a shooting mode icon Q1, a strobe icon Q2, a special shooting mode icon Q3, and a return icon Q4 on an image W21 displayed on the display unit 7. The shooting mode icon Q1 is an icon to which an instruction signal for causing the display unit 7 to display detailed parameters of the operating conditions of the shooting mode is input. As detailed parameters of the shooting operation conditions, parameters such as shutter speed, exposure correction range, ISO sensitivity, and aperture value are displayed. The strobe icon Q2 is an icon to which an instruction signal for displaying detailed parameters of strobe operation conditions of the strobe (flash) on the display unit 7 is input. As detailed parameters of the strobe operating conditions, parameters such as automatic light emission, light emission inhibition, red-eye light emission, and forced light emission are displayed. The special shooting mode icon Q3 is an icon to which an instruction signal for displaying a lower menu (submenu) of the special shooting mode on the display unit 7 is input.

  Subsequently, the control unit 11 determines whether or not the special shooting mode icon Q3 is operated by the user (step S202). Specifically, as shown in FIG. 13, the control unit 11 touches the special shooting mode icon Q3 displayed on the image W21 of the display unit 7, so that the touch panel 8 receives an input. It is determined whether or not the special shooting mode icon Q3 has been operated. If the special shooting mode icon Q3 is not operated by the user within a predetermined time, for example, 30 seconds (step S202: No), the imaging apparatus 1 returns to the main routine shown in FIG. On the other hand, when the special shooting mode icon Q3 is operated by the user (step S202: Yes), the imaging device 1 proceeds to step S203.

  Thereafter, the display control unit 11e displays a lower menu icon in the special shooting mode on the image displayed on the display unit 7 (step S203). Specifically, as illustrated in FIG. 14, the display control unit 11 e displays a swing composition icon Q 5, an art filter icon Q 6, a swing art icon Q 7, and a return icon Q 4 on the image W 21 displayed on the display unit 7.

  Here, the various icons shown in FIG. 14 will be described. The swing composition icon Q5 is an icon for inputting an instruction signal for setting the photographing mode of the imaging apparatus 1 to the swing composite photographing mode. The swing composite shooting mode is a shooting mode in which a subject image or background image separated from an image shot by the image separation unit 11b is combined with another image to generate a combined image. The art filter icon Q6 is an icon for inputting an instruction signal for setting the shooting mode of the imaging apparatus 1 to special effect shooting. The special effect shooting is a shooting mode in which a special effect image generation unit 11c applies a special effect set by a user to a shot image to generate a special effect image. The swing art icon Q7 is an icon for inputting an instruction signal for setting the shooting mode of the imaging apparatus 1 to the swing art shooting mode. In the swing art shooting mode, the special effect image generation unit 11c applies a special effect set by the user to the subject image or the background image separated from the image captured by the image separation unit 11b to generate a special effect image. This is a shooting mode in which the generated special effect image is combined with another image to generate a combined image.

  Returning to FIG. 12, in step S204, the control unit 11 determines whether or not the swing art icon Q7 is operated by the user. Specifically, as illustrated in FIG. 14, the control unit 11 performs a swing according to a signal received by the touch panel 8 when the user touches a swing art icon Q <b> 7 displayed on the screen W <b> 21 of the display unit 7. It is determined whether or not the art icon Q7 has been operated. When the swing art icon Q7 is not operated by the user within the predetermined time (step S204: No), the imaging device 1 returns to the main routine shown in FIG. On the other hand, when the swing art icon Q7 is operated by the user (step S204: Yes), the imaging device 1 proceeds to step S205.

  Subsequently, the display control unit 11e causes the display unit 7 to display a background image corresponding to the image data stored in the image data storage unit 9a and a plurality of special effect icons (step S205). Specifically, as shown in FIG. 15, the display control unit 11e displays the background image W22 on the display unit 7 and refers to the special effect information table stored in the special effect information storage unit 9d, so that the normal icon Q8, soft icon Q9 and sepia icon Q10 are displayed on the display unit 7.

  Here, with reference to FIG. 16, the display method of the icon for special effects which the display control part 11e displays on the display part 7 is demonstrated. FIG. 16 is a diagram illustrating an example of the special effect information table stored in the special effect information storage unit 9d.

  As shown in FIG. 16, the special effect information table T1 describes special effects applicable to the subject image and the background image. Specifically, special effects applicable to the subject image are “normal”, “pop”, “fisheye”, “soft”, “sepia”, “black and white”, “sketch”, and “miniature”. On the other hand, special effects to which the background image can be applied are “normal”, “pop”, “fisheye”, “soft”, “sepia”, “black and white”, “sketch”, and “miniature”. In addition, the special effect information table T1 describes the degree of recommendation (0 to 5) corresponding to a combination of special effects when a composite image is generated by applying a special effect to each of the subject image and the background image. Specifically, when the special effect of the subject image is “normal”, “5” is described as the recommended degree of the combination of “normal”, “soft”, and “sepia” in the special effect of the background image. Furthermore, “4” is described as the recommended degree of the combination of “pop”, “black and white”, and “sketch”, and “1” is described as the recommended degree of the combination of “fisheye” and “miniature”. That is, the degree of recommendation is set higher stepwise as the combination of special effects is better. Note that the numerical values shown in FIG. 16 are merely examples, and are not limited thereto. For example, the numerical value of the recommendation degree can be changed with the operation switch 6d or the like. In addition, the recommendation level may be set to be lower in steps as the combination of special effects is better.

  Thus, when displaying the special effect icon on the display unit 7, the display control unit 11 e refers to the special effect information table T 1 and displays the special effect combinations in descending order of recommendation. For example, as illustrated in FIG. 15, the display control unit 11 e does not apply a special effect to the background image W22 currently displayed by the display unit 7 and the background image W22 is “normal”. Control is performed to display the normal icon Q8, the soft icon Q9, and the sepia icon Q10 on the display unit 7 as effect icons. Thereby, the imaging device 1 can present to the user information that serves as a guideline for selecting a combination of special effects that is optimal for the image currently displayed on the display unit 7.

  Next, each of the special effects shown in FIG. 16 will be described. Normal is a state in which no special effect is applied by the special effect image generation unit 11c. Pop is an effect that highlights colors colorfully and impressively. For example, processing for increasing the hue and saturation of image data is performed. Fisheye is the effect of finishing a straight line distorted in an arc shape as photographed using a fisheye lens. For example, the image data is changed to an image distorted by coordinate transformation or the like. Software is an effect that softens the entire image as if it were taken using a soft focus lens. For example, processing for reducing the contrast and saturation levels of the image data is performed. Sepia is an effect that produces a sepia tone monochrome. For example, processing for lowering the level of white balance and tone curve is performed. Black and white is the effect of finishing an image in black and white expressed in gray tones. For example, processing with a reduced white balance level is performed. A sketch is an effect that finishes an image like a sketch in which outlines are drawn with line drawings. Miniature is the effect of focusing on the center of the screen and blurring by lowering the contrast around the top and bottom.

  Returning to FIG. 12, in step S <b> 206, the control unit 11 determines whether a background image switching operation has been performed by the user. Specifically, as shown in FIG. 15, the control unit 11 determines whether or not the user has touched an image switching icon Q11 that receives an input of an instruction signal instructing switching to another background image. When the switching operation of the background image is performed by the user (step S206: Yes), the display control unit 11e changes from the background image currently displayed on the display unit 7 to another background image stored in the image data storage unit 9a. The image is switched and displayed on the display unit 7 (step S207), and the imaging apparatus 1 returns to step S205. On the other hand, when the background image switching operation is not performed by the user (step S206: No), the imaging device 1 proceeds to step S208.

  Subsequently, the control unit 11 determines whether or not the special effect icon has been operated by the user (step S208). Specifically, as illustrated in FIG. 15, the control unit 11 determines whether or not the user touches any one of the normal icon Q8, the soft icon Q9, and the sepia icon Q10 displayed on the screen of the display unit 7. Judging. When the special effect icon is operated by the user (step S208: Yes), the special effect image generation unit 11c corresponds to the special effect icon touched by the user with respect to the background image currently displayed on the display unit 7. The special effect, for example, a special effect image subjected to software is generated (step S209), and the display control unit 11e displays the special effect image (see, for example, FIG. 10B) on the display unit 7 (step S210).

  Thereafter, the control unit 11 temporarily stores the special effect image currently displayed on the display unit 7 in the temporary image data storage unit 9c (step S211), and the imaging apparatus 1 returns to the main routine illustrated in FIG.

  A case where the special effect icon is not operated by the user in step S208 (step S208: No) will be described. In this case, the control unit 11 determines whether or not a special effect icon switching operation has been performed by the user within a predetermined time (step S212). Specifically, as shown in FIG. 15, the control unit 11 inputs an instruction signal for instructing the user to switch from the special effect icon currently displayed on the display unit 7 to the next special effect icon with a high recommendation level. It is determined whether or not the switch icon Q12 that accepts is touched. If the switching operation of the special effect icon is not performed by the user within a predetermined time, for example, within 30 seconds (step S212: No), the imaging device 1 returns to the main routine shown in FIG. On the other hand, when the special effect icon switching operation is performed by the user (step S212: Yes), the display control unit 11e refers to the special effect information table T1 stored in the special effect information storage unit 9d, and then recommends it. The special effect icons having high degrees, for example, “pop”, “black and white” and “sketch” special effect icons are switched to be displayed on the display unit 7 (step S213), and the imaging apparatus 1 returns to step S205.

  Next, the photographing process in step 107 in FIG. 11 will be described. FIG. 17 is a flowchart showing an outline of the photographing process. In the following description, a case where the shooting mode of the imaging apparatus 1 is set to swing art shooting will be described.

  As shown in FIG. 17, first, the control unit 11 determines whether or not the shooting mode of the imaging apparatus 1 is set to swing art shooting (step S301). When the imaging mode of the imaging device 1 is set to swing art shooting (step S301: Yes), the imaging device 1 performs a swing art shooting process of generating a composite image by applying a special effect to each of the subject image and the background image. This is executed (step S302), and the imaging apparatus 1 returns to the main routine shown in FIG.

  A case where the shooting mode of the imaging apparatus 1 is not set to swing art shooting in step S301 (step S301: No) will be described. In this case, the imaging apparatus 1 captures an image of the subject (step S303), and the display control unit 11e displays an image corresponding to the captured image data on the display unit 7 (step S304).

  Subsequently, the control unit 11 stores the captured image data in the image data storage unit 9a of the storage unit 9 (step S305), and the imaging apparatus 1 returns to the main routine illustrated in FIG. In step S305, compressed data (JPEG) and uncompressed data (RAW) may be stored together as image data. In this case, the user may be able to select one of these image data at the time of image composition described later. In addition, various types of aspect ratios such as 4: 3 and 16: 9 can be set as the background image, and the user can select a desired aspect ratio according to the expression and purpose.

  Next, the swing art photographing process in step S302 of FIG. 18 will be described. FIG. 18 is a flowchart showing an outline of the swing art shooting process.

  In FIG. 18, first, the control unit 11 determines whether or not the imaging apparatus 1 has started moving by the user (step S <b> 401). Specifically, the control unit 11 determines whether or not the imaging device 1 has started moving based on the acceleration detected by the posture detection unit 3 when the user moves the imaging device 1 (see FIG. 4). . When the imaging apparatus 1 has not started moving (step S401: No), the control unit 11 repeats this determination. On the other hand, when the imaging device 1 starts moving (step S401: Yes), the imaging device 1 proceeds to step S402.

  Subsequently, the imaging apparatus 1 continuously shoots the subject at a minute time interval (step S402), and the control unit 11 temporarily stores the captured image in the temporary image data storage unit 9c of the storage unit 9. (Step S403).

  Thereafter, the control unit 11 determines whether or not the imaging apparatus 1 has stopped moving (step S404). When the imaging device 1 has not stopped moving (No at Step S404), the imaging device 1 returns to Step S402. On the other hand, when the imaging device 1 stops moving (step S404: Yes), the imaging device 1 proceeds to step S405.

  Subsequently, the control unit 11 determines whether or not the subject image and the background image can be separated from the plurality of images taken by the image separation unit 11b (step S405). Specifically, in the control unit 11, the image separation unit 11b is based on a plurality of images stored in the temporary image data storage unit 9c, for example, the parallax of the subject included in each adjacent image (see FIG. 7). Then, it is determined whether or not the subject image and the background image can be separated from the captured image. When it is not possible to separate the subject image and the background image from the image captured by the image separation unit 11b (step S405: No), the display control unit 11e separates the subject image and the background image from the captured image. Is displayed on the display unit 7 (step S406), and the imaging apparatus 1 returns to the main routine shown in FIG.

  A case will be described in which it is possible to separate the subject image and the background image from the image captured by the image separation unit 11b in step S405 (step S405: Yes). In this case, the image separation unit 11b separates the subject image from the captured image (step S407), and the composite image generation unit 11d generates the subject image separated by the image separation unit 11b by the special effect image generation unit 11c. A composite image (see, for example, FIG. 10C) is generated by combining with the special effect image (step S408).

  Subsequently, the display control unit 11e displays the composite image generated by the composite image generation unit 11d on the display unit 7 (step S409), and the control unit 11 determines whether or not the menu switch 6e is operated by the user. (Step S410). When the menu switch 6e is operated by the user (step S410: Yes), the imaging device 1 proceeds to step S411 described later. On the other hand, when the menu switch 6e is not operated by the user within a predetermined time, for example, within 30 seconds (step S410: No), the imaging device 1 proceeds to step S419 described later.

  A case where the menu switch 6e is operated by the user in step S410 (step S410: Yes) will be described. In this case, the display control unit 11e displays an image selection icon and a special effect icon on the image displayed by the display unit 7 (step S411). Specifically, as illustrated in FIG. 19, the display control unit 11 e displays a background selection icon Q <b> 13 that instructs switching to a special effect icon of a special effect to be applied to the background image on the composite image W <b> 23 displayed by the display unit 7. Display. Further, the display control unit 11e refers to the special effect information table T1 and displays information on the special effects to be applied to the subject image in descending order of recommendation according to the combination with the special effect applied most recently. For example, the display control unit 11e causes the display unit 7 to display the normal icon Q14 and the soft icon Q15 when the special effect on the background image most recently applied by the special effect image generation unit 11c is “soft”. Furthermore, the display control unit 11e causes the display unit 7 to display a switching icon Q16 that receives input of a special effect icon switching instruction signal and an end icon Q17 that receives input of an instruction signal to end swing art shooting.

  Subsequently, the control unit 11 determines whether or not the image selection icon has been operated by the user (step S412). Specifically, as shown in FIG. 19, the control unit 11 determines whether or not the background selection icon Q13 has been operated by the user. When the user operates the background icon (step S412: Yes), the display control unit 11e switches between the image selection icon and the special effect icon (step S413), and the imaging device 1 returns to step S412. Specifically, the display control unit 11e displays a subject icon (not shown) as an image selection icon on the composite image W23 displayed by the display unit 7, and refers to the special effect information table T1 to determine the background. Information on special effects to be applied to the image is displayed in order of the degree of recommendation according to the combination with the special effect applied most recently.

  A case where the image selection icon is not operated by the user in step S412 (step S412: No) will be described. In this case, the control unit 11 determines whether or not the special effect icon has been operated by the user (step S414). When the special effect icon is operated by the user (step S414: Yes), the special effect image generation unit 11c corresponds to the special effect icon in which the touch panel 8 receives an input in the contact area located on the display area of the special effect icon. A special effect image is generated by applying the special effect to the subject image (step S415).

  Subsequently, the composite image generation unit 11d generates a composite image using the special effect image generated by the special effect image generation unit 11c (step S416), and the display control unit 11e generates the composite image generated by the composite image generation unit 11d. The image is displayed on the display unit 7 (step S417).

  Thereafter, the control unit 11 determines whether or not the end icon Q17 has been operated by the user within a predetermined time (step S418). When the end icon Q17 is not operated by the user within a predetermined time, for example, 30 seconds (step S418: No), the imaging device 1 returns to step S412. On the other hand, when the end icon Q17 is operated by the user within a predetermined time (step S418: Yes), the control unit 11 stores image data corresponding to the image currently displayed on the display unit 7 in the image data storage unit 9a. The image is stored (step S419), and the imaging apparatus 1 returns to the main routine shown in FIG.

  The case where the special effect icon is not operated by the user in step S414 (step S414: No) will be described. In this case, the control unit 11 determines whether or not a special effect icon switching operation has been performed by the user (step S420). Specifically, the control unit 11 determines whether or not the switching icon Q16 (see FIG. 19) has been operated by the user. When there is no special effect icon switching operation by the user (step S420: No), the imaging apparatus 1 proceeds to step S418. On the other hand, when the special effect switching operation is performed by the user (step S420: Yes), the display control unit 11e refers to the special effect information table T1 and switches to the special effect icon having the next highest recommendation level. The image is displayed on the unit 7 (step S421), and the imaging apparatus 1 returns to step S412. Specifically, the display control unit 11e causes the display unit 7 to display special effect icons of “sepia” and “black and white” as the special effect icons having the next highest recommendation.

  In the first embodiment described above, the special effect image generation unit 11c applies a special effect to one of the subject image and the background image, and the special effect image subjected to this special effect is processed by the composite image generation unit 11d. When combined with the other of the image and the background image, the display control unit 11e displays the combined image on the display unit 7 and refers to the special effect information table T1 stored in the special effect information storage unit 9d. Thus, information related to the special effect to be applied to the subject image or the background image is displayed on the display unit 7 in descending order of recommendation according to the combination with the special effect applied most recently. Thus, when a special effect is individually applied to each of the two image data to generate a composite image, information serving as a guideline for selecting a combination of special effects can be presented to the user.

  Furthermore, in the first embodiment, the display control unit 11e first displays a special effect icon with a highly recommended combination on the display unit 7, so that the user misses the photo opportunity of the subject through the special effect selection operation. In addition, it is possible to obtain a composite image in which a special effect is applied to each of the subject image and the background image.

(Embodiment 2)
Next, the second embodiment will be described. In the first embodiment described above, the display control unit refers to the special effect information table, and recommends information related to the special effect to be applied to the subject image or the background image according to the combination with the special effect that was most recently applied. They are displayed on the display unit 7 in descending order. On the other hand, in the second embodiment, the second special effect information storage unit gives a recommendation degree according to a combination of a shooting mode that can be set by the imaging apparatus and a special effect applied to the subject image or the background image. The effect information table is stored, and when the shooting control mode is set by the input from the input unit, the recommended degree corresponding to the combination with the set shooting mode is referred to the second special effect information table. Information on special effects is displayed on the display unit in descending order. Note that the imaging apparatus of the second embodiment has the same configuration as the imaging apparatus 1 of the first embodiment described above. For this reason, in the second embodiment, a plurality of shooting modes respectively set according to various shooting scenes provided in the imaging apparatus 1 will be described.

  First, a plurality of shooting modes respectively set according to various shooting scenes provided in the imaging apparatus 1 according to the second embodiment will be described. The imaging apparatus 1 according to the second embodiment is provided with normal imaging, panoramic imaging, swing 3D imaging, 3D imaging, HDR (High Dynamic Range) imaging, and S strobe imaging as imaging modes. Note that the above-described shooting modes are merely examples, and are not limited thereto.

  FIG. 20 is a schematic diagram illustrating a panorama shooting method in the shooting mode performed by the imaging apparatus 1. As illustrated in FIG. 20, the imaging apparatus 1 generates a plurality of image data continuously captured at a plurality of imaging positions by being moved from the left side to the right side by the user. Specifically, as illustrated in FIG. 20, the imaging device 1 generates images W31 to W33 (FIG. 20A). Thereafter, the composite image generation unit 11d generates a single panoramic image W34 in which the ends in the short direction of the images W31 to W33 are overlapped with each other (FIG. 20B). Accordingly, the user can obtain a panoramic image W34 by shooting while moving the imaging device 1. In FIG. 20A, representative three images W31 to W33 among a plurality of images generated by the imaging unit 2 are shown, and there are a plurality of images between the images W31 to W33. .

  Further, the imaging device 1 can perform 3D shooting and swing 3D shooting in the shooting mode of the imaging device 1 by shooting while moving the imaging device 1 by the user. The 3D shooting is shooting in which the user continuously shoots a subject while moving the imaging device 1 to generate two image data in which one end in the left-right direction of each field of view overlaps. Specifically, as illustrated in FIG. 20, when the user moves the imaging device 1 from the left to the right, the imaging device 1 continuously captures the subject A <b> 1, whereby one end in the left-right direction of each field of view. Two pieces of image data having overlapping portions are generated. Thereby, the imaging device 1 can generate a three-dimensional image (hereinafter referred to as “3D image”) that is virtually recognized by the user. Furthermore, the swing 3D shooting is a shooting in which a special effect is applied to the shot 3D image to generate a special effect 3D image. Specifically, the image separation unit 11b separates the subject image and the background image from the 3D image, and the special effect image generation unit 11c applies a special effect to the separated subject image and background image to generate a special effect image. . Thereafter, the composite image generation unit 11d generates a special effect 3D image by combining the special effect images. Thereby, the imaging device 1 can generate a special effect 3D image obtained by applying a special effect to the 3D image.

  FIG. 21 is a schematic diagram illustrating an HDR imaging method in the imaging mode performed by the imaging apparatus 1. FIG. 22 is a diagram illustrating an example of an image generated by the imaging apparatus 1 under the situation illustrated in FIG. In FIG. 22, an image W40 is an image generated by the imaging device 1 under the situation shown in FIG. 21A, and an image W41 is generated by the imaging device 1 under the situation shown in FIG. It is an image. HDR imaging is imaging in which the reproduction range of brightness is expanded by combining an image for a dark part and an image for a bright part in order to obtain an image with a large dynamic range.

  As shown in FIG. 21, first, the imaging apparatus 1 irradiates the subject A3 with the light of the strobe 4 to photograph the subject A3 and generates an image W40 (FIG. 22A). Subsequently, the imaging apparatus 1 shoots the subject A3 with an exposure time that is longer than normal shooting, for example, twice as long as normal shooting, and generates an image W41 (FIG. 22B). Thereafter, the image separation unit 11b separates the subject image and the background image from the images W40 and W41 based on the difference between the luminance value of the image W40 and the luminance value of the image W41, respectively. Specifically, the image separation unit 11b identifies and separates a region where the difference between the luminance value of the image W40 and the luminance value of the image W41 is equal to or greater than a threshold value as a subject image. On the other hand, the image separation unit 11b identifies and separates a region where the difference between the luminance value of the image W40 and the luminance value of the image W41 is less than the threshold value as a background image. Subsequently, the composite image generation unit 11d generates a composite image by combining the subject image separated from the image W40 by the image separation unit 11b and the background image separated from the image W41. Thereby, the imaging device 1 can generate a composite image having an optimal luminance value between the subject A3 and the background.

  In addition, the imaging apparatus 1 can perform S strobe shooting in which the flash emission is set stronger than normal shooting or background blur shooting with a blurred background by increasing the aperture value compared to normal shooting. Here, the S strobe is a process that combines the image with the strobe light and the image with a longer exposure time without causing the strobe light to prevent the background from becoming dark and reproduce the background and the person correctly. It is. Background blur is a process for reducing the change in brightness of an image and reducing the contrast.

  Processing performed by the imaging apparatus 1 according to the second embodiment having the above imaging mode will be described. The process performed by the imaging apparatus 1 according to the second embodiment performs the same process as the first embodiment described above (see FIG. 11), and the menu operation process and the shooting process are different. For this reason, after describing the menu operation process performed by the imaging apparatus 1 according to the second embodiment, the photographing process performed by the imaging apparatus 1 according to the second embodiment will be described.

  FIG. 23 is a flowchart illustrating an overview of menu operation processing performed by the imaging apparatus 1 according to the second embodiment.

  23, first, the display control unit 11e displays a menu icon (see FIG. 13) on the image displayed on the display unit 7 (step S501), and the control unit 11 operates the special shooting mode icon by the user. It is determined whether or not it has been done (step S502). If the special shooting mode icon has not been operated by the user within the predetermined time (step S502: No), the imaging apparatus 1 returns to the main routine shown in FIG. On the other hand, when the special shooting mode icon is operated by the user (step S502: Yes), the imaging apparatus 1 proceeds to step S503.

  Subsequently, the display control unit 11e displays a lower menu icon in the special shooting mode on the image displayed on the display unit 7 (step S503). Specifically, as illustrated in FIG. 24, the display control unit 11e includes a panorama icon Q21 to which an instruction signal for instructing the imaging device 1 to set panorama shooting is input on an image W50 displayed on the display unit 7. A swing 3D icon Q22 to which an instruction signal for instructing setting of swing 3D shooting is input is displayed on the imaging apparatus 1. In FIG. 24, only the panorama icon Q21 and the swing 3D icon Q22 are the lower-level menu icons in the shooting mode, but the present invention is not limited to this. Specifically, when the switching icon Q23, 24 that receives an input of an instruction signal for instructing switching of the shooting mode is operated by the user, the display control unit 11e performs 3D shooting on another lower menu icon, for example, the imaging device 1. A 3D shooting icon to which an instruction signal for instructing the setting is input is displayed on the display unit 7.

  Subsequently, the control unit 11 determines whether or not the swing 3D icon has been operated by the user (step S504). When the swing 3D icon is not operated by the user within the predetermined time (step S504: No), the imaging device 1 returns to the main routine shown in FIG. On the other hand, when the swing 3D icon is operated by the user (step S504: Yes), the imaging device 1 proceeds to step S505.

  Thereafter, the display control unit 11e displays the image selection icon and the special effect icon on the image displayed on the display unit 7 (step S505). Specifically, as shown in FIG. 25, the display control unit 11e displays a subject selection icon Q26 on the display unit 7 as an image selection icon, and a second special effect information table stored in the special effect information storage unit 9d. The pop icon Q27 is displayed as information on special effects in descending order of recommendation according to the combination with the set shooting mode. In FIG. 25, an icon Q25 is an icon indicating information on the current shooting mode of the imaging apparatus 1 set by the user and information on an image to be subjected to the special effect set by the user.

  Here, with reference to FIG. 26, the display method of the special effect icon which the display control part 11e displays on the display part 7 is demonstrated. FIG. 26 is a diagram illustrating an example of a second special effect information table stored in the special effect information storage unit 9d.

  As shown in FIG. 26, the special effect information table T2 describes special effects applicable to shooting modes that can be set by the imaging apparatus 1. Specifically, special effects applicable to the shooting mode are “miniature”, “pop”, “fisheye”, “soft”, “sepia”, “black and white”, and “sketch”. Furthermore, a recommendation level corresponding to a combination of an imaging mode that can be set by the imaging apparatus 1 and a special effect applied to the subject image or the background image is described as a numerical value (0 to 5). Specifically, when the shooting mode is “swing 3D”, “5” is described as the recommended degree of special effect “pop”. Furthermore, when the shooting mode is “swing 3D”, “0” is described as the recommendation degree of the special effect “fisheye”. The combination in which the recommended degree of the imaging mode and the special effect is “0” is a combination in which a remarkable visual effect due to the special effect cannot be obtained with respect to the captured image. Therefore, in the second embodiment, the display control unit 11e refers to the second special effect information table T2 stored in the special effect information storage unit 9d, and displays the special effect icons in descending order of the shooting mode and the recommendation level. Control to be displayed on the unit 7 is performed. Further, the display control unit 11e performs control for causing the display unit 7 to display only special effect icons that can be combined with the shooting mode. That is, the display control unit 11e performs control so that the display unit 7 does not display the special effect icon having the recommended degree of imaging mode and special effect of “0”.

  Returning to FIG. 23, in step S506, the control unit 11 determines whether or not the end icon Q29 (see FIG. 25) has been operated by the user. When the end icon Q29 is operated by the user (step S506: Yes), the imaging apparatus 1 returns to the main routine shown in FIG. On the other hand, when the end icon Q29 is not operated by the user (step S506: No), the imaging device 1 proceeds to step S507.

  Subsequently, the control unit 11 determines whether or not the image selection icon has been operated by the user (step S507). Specifically, as shown in FIG. 25, the control unit 11 determines whether or not a subject selection icon Q26 that accepts an input of an instruction signal for switching to setting of a special effect to be applied to the subject image has been operated. When the image selection icon is not operated by the user (step S508: No), the control unit 11 determines whether or not the special effect icon is operated by the user (step S508). Specifically, as illustrated in FIG. 25, the control unit 11 determines whether or not the pop icon Q27 is operated by the user. When the special effect icon is operated by the user (step S508: Yes), the control unit 11 sets the special effect of the background image according to the special effect icon operated by the user, and the set content is the temporary image. The data is stored in the data storage unit 9c (step S509), and the imaging device 1 returns to step S505.

  A case where the special effect icon is not operated by the user in step S508 (step S508: No) will be described. In this case, the control unit 11 determines whether or not a special effect icon switching operation has been performed by the user (step S510). Specifically, as shown in FIG. 25, the control unit 11 inputs an instruction signal instructing the user to switch from the special effect icon currently displayed on the display unit 7 to the special effect icon with the next highest recommendation level. It is determined whether or not the switching icon Q28 that accepts is operated. When the switching operation of the special effect icon is not performed by the user within the predetermined time (step S510: No), the imaging device 1 returns to step S505. On the other hand, when the special effect icon switching operation is performed by the user (step S510: Yes), the display control unit 11e refers to the special effect information table T2 stored in the special effect information storage unit 9d and then recommends it. A special effect icon having a high degree, for example, a “black and white” special effect icon, is displayed on the display unit 7 (step S511), and the imaging apparatus 1 returns to step S508.

  A case where the image selection icon is operated by the user in step S507 (step S507: Yes) will be described. In this case, the display control unit 11e refers to the image selection icon and the special effect information table stored in the special effect information storage unit 9d to display the special effect icon on the image displayed on the display unit 7 (step S512). ). Specifically, as shown in FIG. 27, the display control unit 11e displays the image selection icon Q30, and refers to the special effect information table stored in the special effect information storage unit 9d, so that the normal icon Q31, sepia An icon Q32 and a monochrome icon Q33 are displayed on the display unit 7. In FIG. 27, the image selection icon Q30 is displayed with the current shooting mode of the imaging apparatus 1 set by the user and information on special effects set for the background, for example, a black and white effect added as a special effect. . Thereby, the user can set the special effect to be applied to the subject image while confirming the special effect set for the background image.

  Here, with reference to FIG. 28, the display method of the special effect icon which the display control part 11e displays on the display part 7 is demonstrated. FIG. 28 is a diagram illustrating an example of the special effect information table stored in the special effect information storage unit 9d.

  As shown in FIG. 28, the special effect information table T3 describes special effects applicable to the subject image and the background image. Specifically, in the special effect information table T3, the recommended degree of the combination of the special effects when the special effect is applied to each of the subject image and the background image to generate the composite image is described with a numerical value (0 to 5). Yes. Specifically, when the special effect of the subject image is “normal”, “5” is described as the recommended degree of “normal”, “soft”, and “monochrome” in the special effect of the background image. Furthermore, “4” is described as the recommended level of “Sepia”, and “0” is described as the recommended level of “Pop”, “Fisheye”, “Sketch”, and “Miniature”. A special effect combination in which the recommended degree of special effect is “0” is a combination in which a remarkable visual effect due to the special effect cannot be obtained with respect to a captured image. Therefore, in the second embodiment, the display control unit 11e refers to the special effect information table T3 stored in the special effect information storage unit 9d, and displays only the special effect icons that can be combined with the special effects. 7 is controlled. In other words, the display control unit 11e performs control (restriction) so that the special effect icon with the special effect combination recommendation degree “0” is not displayed on the display unit 7. Thereby, the user can reliably prevent the image from becoming unnatural when the special effect is set for each of the background image and the subject image to generate the composite image.

  Returning to FIG. 23, in step s513, the control unit 11 determines whether or not the special effect icon is operated by the user. When the special effect icon is operated by the user (step S513: Yes), the control unit 11 sets the special effect of the subject image according to the special effect icon operated by the user, and the set content is the temporary image. It memorize | stores in the data storage part 9c (step S514), and the imaging device 1 returns to step S505.

  On the other hand, when the special effect icon is not operated by the user (step S513: No), the control unit 11 determines whether the special effect icon switching operation is performed by the user (step S515). Specifically, as shown in FIG. 27, the control unit 11 inputs an instruction signal instructing the user to switch from the special effect icon currently displayed on the display unit 7 to the next special effect icon having a high recommendation level. It is determined whether or not the switching icon Q34 that accepts the touch has been touched. When the switching operation of the special effect icon is not performed by the user within the predetermined time (step S515: No), the imaging device 1 returns to step S505. On the other hand, when the switching operation of the special effect icon is performed by the user (step S515: Yes), the display control unit 11e refers to the special effect information table T3 stored in the special effect information storage unit 9d and recommends it next. A special effect icon having a high degree, for example, a “pop” special effect icon is switched to display on the display unit 7 (step S516), and the imaging apparatus 1 returns to step S505.

  Next, imaging processing performed by the imaging apparatus 1 according to the second embodiment will be described. FIG. 29 is a flowchart showing an outline of the photographing process. In the following description, a case where the shooting mode of the imaging apparatus 1 is set to swing 3D shooting will be described.

  As shown in FIG. 29, the control unit 11 first determines whether or not the shooting mode of the imaging device 1 is set to swing 3D shooting (step S601). When the shooting mode of the imaging device 1 is set to swing 3D shooting (step S601: Yes), the control unit 11 determines whether or not the imaging device 1 has started moving (step S602). When the imaging device 1 has not started moving (step S602: No), the control unit 11 repeats this determination. On the other hand, when the imaging apparatus 1 starts moving by the user (step S602: Yes), the imaging apparatus 1 proceeds to step S603.

  Subsequently, the imaging apparatus 1 continuously shoots at a minute time interval (step S603), and the control unit 11 temporarily stores the captured image in the temporary image data storage unit 9c (step S604).

  Thereafter, the control unit 11 determines whether or not the imaging apparatus 1 has stopped moving (step S605). When the imaging device 1 has not stopped moving (step S605: No), the imaging device 1 returns to step S603. On the other hand, when the imaging apparatus 1 stops moving (step S605: Yes), the imaging apparatus 1 proceeds to step S606.

  Subsequently, the control unit 11 determines whether or not the subject image and the background image can be separated from the plurality of images taken by the image separation unit 11b (step S606). When the subject image and the background image cannot be separated from the plurality of images taken by the image separation unit 11b (step S606: No), the display control unit 11e separates the subject image and the background image from the taken image. A warning to the effect that it cannot be displayed is displayed on the display unit 7 (step S607), and the imaging apparatus 1 returns to the main routine shown in FIG.

  A case where the subject image and the background image can be separated from the image captured by the image separation unit 11b in step S606 (step S606: Yes) will be described. In this case, the image separation unit 11b separates the subject image and the background image from the captured image (step S608), and the special effect image generation unit 11c stores the subject image and the background image stored in the temporary image data storage unit 9c. A special effect image is generated by applying the special effect set to each (step S609).

  Subsequently, the composite image generation unit 11d generates a composite image by combining the special effect images of the subject image and the background image generated by the special effect image generation unit 11c (step S610), and the display control unit 11e A 3D special effect image is displayed on the display unit 7 using the composite image generated by the image generation unit 11d (step S611). Specifically, as illustrated in FIG. 30, the display control unit 11e causes the display unit 7 to display a 3D image using two image data of the composite image WL2 and the composite image WR2. Accordingly, the user can virtually visually recognize a 3D image obtained by applying a special effect to each of the subject image and the background image.

  Thereafter, the control unit 11 stores the image currently displayed on the display unit 7 in the image data storage unit 9a (step S612), and the imaging apparatus 1 returns to the main routine shown in FIG.

  A case where the shooting mode of the imaging device 1 is not set to swing 3D shooting in step S601 (step S601: No) will be described. In this case, the imaging device 1 captures the subject (step S613), and the display control unit 11e displays the captured image on the display unit 7 in a rec view (step S614).

  Thereafter, the control unit 11 stores the captured image in the image data storage unit 9a (step S615), and the imaging device 1 returns to the main routine shown in FIG.

  According to the second embodiment described above, the special effect information storage unit 9d provides the degree of recommendation according to the combination of the shooting mode that can be set by the imaging apparatus 1 and the special effect applied to the subject image or the background image. When the special effect information table T2 is stored, and the shooting control mode is set by the display control unit 11e by the input from the touch panel 8, the second special effect information table is referred to according to the combination with the set shooting mode. Information regarding special effects is displayed on the display unit 7 in descending order of recommendation. Thereby, when the user performs shooting by combining the shooting mode and the special effect, information serving as a guideline for selecting a combination of the shooting mode and the special effect can be presented to the user.

  Further, according to the second embodiment, the display control unit 11e performs control for causing the display unit 7 to display only the special effect icons corresponding to the special effects capable of combining the shooting mode and the special effects. Thereby, the user can surely prevent an image to be taken from becoming unnatural when shooting is performed by combining the shooting mode and the special effect.

  In the second embodiment described above, the image capturing apparatus 1 captures the image while being moved by the user, thereby generating two pieces of image data in which the left and right ends of each field of view overlap each other. The image pickup unit may include two optical systems to generate two pieces of image data in which one end in the horizontal direction of each field of view overlaps.

(Other embodiments)
In the first and second embodiments, the display control unit 11e refers to the special effect information table stored in the special effect information storage unit 9d, and the special effect icon colors in descending order of recommendation degree of special effect combinations. In addition, the display unit 7 may be highlighted in a display mode in which the size is changed. Accordingly, the user can intuitively select an optimal combination of special effects based on information serving as a guideline for selecting a combination with special effects.

  In the first and second embodiments, the composite image generation unit 11d synthesizes special effect images in which special effects are applied to the subject image and the background image separated by the image separation unit 11b. A special effect is applied to the image (compressed data or non-compressed data, compressed data or non-compressed data classified by aspect ratio) stored in the image data storage unit 9a and the image immediately after shooting with multiple exposure, respectively. Two special effect images may be generated, and the generated two special effect images may be combined to generate a combined image.

  In the first and second embodiments, the posture detection unit 3 detects the posture state of the imaging device 1. For example, the posture detection unit 3 detects acceleration generated when the user taps the display screen of the display unit 7. Accordingly, an operation signal of a touch operation for switching various shooting modes or various settings of the imaging apparatus 1 may be received, and the operation signal may be output to the control unit 11.

  In the first and second embodiments, the special effect information table stored in the special effect information storage unit 9d is updated or rewritten by connecting to an external processing device such as a personal computer or a server via the communication unit 10. You may go. Thereby, the display control part 11e can display a special effect icon with the recommendation degree which considered the combination of the newly added special effect and imaging | photography mode. Furthermore, the special effect newly added to the storage medium that can be attached to the storage unit 9, for example, a memory card, or the recommended degree of the combination of the shooting mode is stored, and the control unit 11 acquires the special effect information from the storage medium. The special effect information table may be updated or rewritten by being stored in the unit 9d.

  In the first and second embodiments, the imaging apparatus 1 has been described as a digital camera. However, the imaging apparatus 1 is applied to various electronic devices having a shooting function and a display function such as a digital video camera and a mobile phone with a camera. Can do.

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Imaging part 2a Lens part 2b Lens drive mechanism 2c Aperture 2d Aperture drive mechanism 2e Shutter 2f Shutter drive mechanism 2g Image sensor 2h Signal processing part 3 Attitude detection part 4 Strobe 5 Clock 6 Operation input part 6a Power switch 6b Release switch 6c Shooting mode selection switch 6e Menu switch 6f Zoom switch 7 Display unit 8 Touch panel 9 Storage unit 9a Image data storage unit 9b Program storage unit 9c Temporary image data storage unit 9d Special effect information storage unit 10 Communication unit 11 Control unit 11a Image processing unit 11b Image separation unit 11c Special effect image generation unit 11d Composite image generation unit 11e Display control unit

Claims (6)

An imaging apparatus that has a display unit that captures an image of a subject, generates electronic image data of the subject, and displays an image corresponding to the generated image data, and can apply a special effect to the image. There,
An image separation unit for separating a subject image and a background image from the image displayed by the display unit;
An input unit that receives input of information relating to special effects applied to the subject image and the background image separated by the image separation unit;
A special effect image generating unit that generates a special effect image of each image by individually applying a special effect according to information received by the input unit to the subject image and the background image;
A composite image generation unit that generates a composite image using the special effect image generated by the special effect image generation unit;
When the image separation unit cannot separate the subject image and the background image from the image, a warning is displayed on the display unit, and a display control unit;
An imaging apparatus comprising: The background image can set a plurality of aspect ratios,
The imaging apparatus according to claim 1, wherein the input unit includes an operation switch that instructs an aspect ratio of the background image.   The display control unit superimposes only information related to special effects that can be combined with the special effect applied most recently on the composite image generated by the composite image generation unit displayed by the display unit. The imaging apparatus according to claim 1, wherein the imaging apparatus is displayed on a display unit. The display control unit displays a plurality of menu icons in the imaging apparatus on the display unit,
The input unit is provided on a display screen of the display unit, and detects a position where an object touching from the outside contacts, and a touch panel that receives an input of a signal corresponding to the menu icon displayed at the detected position. The imaging apparatus according to claim 1, wherein the imaging apparatus is provided. An imaging apparatus that has a display unit that captures an image of a subject, generates electronic image data of the subject, and displays an image corresponding to the generated image data, and can apply a special effect to the image. An imaging method to perform,
An image separation step of separating a subject image and a background image from the image displayed by the display unit;
An input step for accepting input of information relating to special effects applied to the subject image and the background image separated in the image separation step;
A special effect image generation step of generating a special effect image of each image by individually applying a special effect according to the information received in the input step to the subject image and the background image;
A composite image generation step of generating a composite image using the special effect image generated in the special effect image generation step;
When the subject image and the background image cannot be separated from the image in the image separation step, a warning is displayed on the display unit and a display control step;
An imaging method comprising: An imaging apparatus that has a display unit that captures an image of a subject, generates electronic image data of the subject, and displays an image corresponding to the generated image data, and can apply a special effect to the image. ,
An image separation step of separating a subject image and a background image from the image displayed by the display unit;
An input step for accepting input of information relating to special effects applied to the subject image and the background image separated in the image separation step;
A special effect image generation step of generating a special effect image of each image by individually applying a special effect according to the information received in the input step to the subject image and the background image;
A composite image generation step of generating a composite image using the special effect image generated in the special effect image generation step;
When the subject image and the background image cannot be separated from the image in the image separation step, a warning is displayed on the display unit and a display control step;
The imaging program characterized by performing.

Images