JP5181832B2 - Photographed image composition method and digital camera - Google Patents

Description

  The present invention relates to a technique for capturing an image of a subject with a clear background and a blurred background using a digital camera not equipped with a sufficient aperture mechanism such as a camera-equipped mobile phone.

  When a photograph is taken with the subject in focus, the subject appears sharp, but the foreground and background located before and after the subject are blurred images. At this time, the distance range that is sharply viewed from the photographic lens is called the depth of field. If the focal length and the shooting distance are the same, the depth of field becomes deeper as the lens aperture (F value) is reduced. The more you open, the shallower the depth of field.

  For example, when shooting a specific object, such as portrait shooting, only the main subject is sharpened by setting the lens to the open state and focusing on the subject with a shallow depth of field. In this case, it is sufficient to take a photograph using a large-aperture lens with a bright F-number as a lens. Pictures with a blur effect cannot be obtained.

  On the other hand, although digital cameras and camera-equipped mobile phones have recently become widespread, large-aperture lenses with a bright F value have a large lens size and are expensive, and are used for cameras mounted on camera-equipped cellular phones and the like. It is almost impossible to adopt as a lens. For this reason, it is generally difficult to take a picture with a clear subject and blurred background due to the characteristics of the lens of the camera mounted on the camera-equipped mobile phone or the like.

  As means for solving such a problem, Patent Document 1 discloses that in a digital camera that records an optical image formed on a imaging element by a photographic lens on a recording medium as digital data, the optical image is read from the imaging element. According to the distance between the main subject and a foreground and / or background area of the main subject. An imaging control unit that captures a plurality of frames with a focal position shifted from the in-focus position, a display unit that displays a captured image, and each of the captured images of the plurality of frames displayed on the display unit. An image selecting means for selecting an image constituting the area, and images of each area selected by the image selecting means are extracted from the captured images of the plurality of frames, and A technique for enabling photographing of an image having equivalent blurring effect without using a large-aperture lens having a bright F value is provided by including an image composition means for composing a rear image into one. Has been.

JP 2003-209727 A

  According to the digital camera described in Patent Document 1, the main subject is imaged at the in-focus position, and a plurality of frames are imaged with the focal position shifted from the in-focus position. By synthesizing the image, the foreground or background image of the main subject can be set as a desired defocused image. Therefore, even if a large aperture lens with a bright F value is not used, shooting with the same defocusing effect can be performed. Images can be acquired.

  However, in the invention described in Patent Document 1, an optical image formed on the image sensor by the photographic lens is first classified by the sectioning unit according to the distance from the image sensor and the area including the main object. The image is divided into a plurality of areas including the foreground and / or background area, and then the main subject is imaged at the in-focus position, and the focus position is shifted forward or backward with respect to the in-focus position. Since this is a configuration in which frames are captured and image data of each area selected by the user from among the captured images of the plurality of frames is combined to generate one combined image, the configuration is complicated and operation Is also troublesome.

  An object of the present invention is to provide a means that makes it possible to acquire a photographed image having a blur rendering effect by image synthesis in a relatively simple manner.

  The captured image composition method of the present invention moves the photographic lens from the shortest shooting distance position (macro edge) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction when shooting the subject, The subject is photographed at each in-focus position, and the subject region and the background region of the subject are extracted from the contrast value of each photographed image, and the subject is focused on the extracted subject region. An image of a corresponding area photographed at a position is fitted, and an image of the corresponding area photographed at the shortest photographing distance position is fitted to the extracted background area to be combined into one image.

  In addition, the method of synthesizing a captured image according to the present invention is configured to move the focus of the photographing lens from the shortest photographing distance position (macro end) to the ∞ position direction or from the ∞ position to the shortest photographing distance position direction when photographing a subject, The subject is photographed at the in-focus position and the infinite position of the subject, respectively, the subject area and the background area of the subject are extracted from the contrast value of each photographed image, and in accordance with a synthesis method selection instruction, The extracted subject area and background area are combined into one image by combining the subject area and background area selected from the images taken at the shortest shooting distance position, the in-focus position of the subject, and the ∞ position. It is characterized by doing.

  In addition, the method of synthesizing a captured image according to the present invention is configured to move the focus of the photographing lens from the shortest photographing distance position (macro end) to the ∞ position direction or from the ∞ position to the shortest photographing distance position direction when photographing a subject, The subject is photographed at the in-focus position and the infinite position of the subject, and the subject area, the background area and the foreground area of the subject are extracted from the contrast value of each photographed image, and a synthesis method selection instruction In accordance with the extracted subject area, background area and foreground area, the subject area, background area and foreground selected from the images taken at the shortest shooting distance position, the focus position of the subject and the ∞ position It is characterized by combining regions into one image.

  The digital camera of the present invention moves the photographic lens from the shortest shooting distance position (macro end) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction during subject shooting, while focusing on the shortest shooting distance position and the subject. Subject photographing means for photographing the subject at each position; subject / ambient scene extracting means for extracting the subject region and the background region of the subject from the contrast value of each photographed image; and the extracted subject Fit the image of the corresponding area photographed at the in-focus position of the subject to the area, and fit the image of the corresponding area photographed at the shortest shooting distance position to the extracted background area. And image combining means for combining the images into one image.

  In addition, the digital camera of the present invention moves the photographic lens from the shortest shooting distance position (macro end) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction when shooting the subject, Subject photographing means for photographing the subject at the focal position and the infinite position; and a subject / ambient scene extracting means for extracting the subject area and the background area of the subject from the contrast value of each photographed image; The subject area and background area selected from the images taken at the shortest shooting distance position, the in-focus position of the subject, and the ∞ position with respect to the extracted subject area and background area in accordance with a synthesis method selection instruction And a photographed image synthesizing means for synthesizing them into one image.

  In addition, the digital camera of the present invention moves the photographic lens from the shortest shooting distance position (macro end) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction when shooting the subject, Subject photographing means for photographing the subject at the focal position and the ∞ position, and subject / ambient scene extraction for extracting the subject region, the background region of the subject, and the foreground region from the contrast value of each photographed image, respectively. And the selected subject area, the background area, and the foreground area are selected from the images taken at the shortest shooting distance position, the in-focus position of the subject, and the ∞ position in accordance with the means and the synthesis method selection instruction. Photographed image combining means for combining the subject area, the background area, and the foreground area into a single image. It is characterized in.

  According to the present invention, the lens is controlled to move in one direction from the ∞ position to the macro position or from the macro position to the ∞ position with respect to a digital camera not equipped with a sufficient aperture mechanism such as a camera-equipped mobile phone. In the meantime, the photographing of the subject can be completed at the macro position and the subject in-focus position, or at the macro position, the subject in-focus position and the ∞ position, respectively. Etc. can be minimized.

  In addition, since the captured images at the macro position and the ∞ position are used as the composite image, it is possible to obtain a captured image with a large blur effect with a relatively simple configuration and operation.

  In addition, images such as full focus (both near and far) and full out-of-focus (all out of focus) can be easily captured by the composition processing method.

  FIG. 1 is a functional block diagram showing the main configuration of a digital camera to which the present invention is applied. The camera function of a single digital camera having the functional block shown in the figure or a mobile phone equipped with the digital camera is shown. It can be realized as an attached mobile communication terminal.

  The digital camera of this embodiment includes a lens control unit 11, an image sensor unit 12, a control unit 13, a memory unit 14, an image processing unit 15, a display unit 16, a storage unit 17, and an operation unit 18. When implemented as a portable communication terminal equipped with the digital camera of the present embodiment, the control unit 13, the memory unit 14, the image processing unit 15, the display unit 16, the storage unit 17, and the operation unit 18 are connected to the portable communication terminal. The terminal can also be used as a functional block corresponding to these.

  The lens control unit 11 moves the lens in accordance with an instruction from the control unit 13, and returns the result of movement to the designated position to the control unit 13 as a return value. The image sensor unit 12 outputs a captured image to the control unit 13 in accordance with an instruction from the control unit 13. The control unit 13 outputs the image data output from the image sensor unit to the memory unit 14. The image processing unit 15 performs composition processing using the image data in the memory unit.

  The display unit 16 displays the image synthesized by the image processing unit 15, and the image displayed by the operation of the operation unit 18 is stored in the storage unit 17. Further, whether or not to use the present method is instructed to the control unit 13 by the operation of the operation unit 18, and when the present method is not used, the subject is photographed once by a normal AF (autofocus) function, and the image processing unit 15 is not performed.

  FIG. 2 is a flowchart showing the first embodiment of the present invention, and FIGS. 3 to 4 are operation explanatory views of this embodiment. Hereinafter, the operation of the present embodiment will be described with reference to FIGS.

  The control unit 13 determines whether or not to use the subject enhancement mode of the present embodiment in accordance with the operation of the operation unit 18 while the camera is activated (S1). When the subject emphasis mode is not used, the process shifts to the normal shooting mode. In the normal photographing mode, the lens position is set to the subject in-focus state by the AF function and photographing is performed once (S8). After the captured focused image is output to the display unit as it is and displayed (S9), the captured image is stored in accordance with a storage instruction (S10) from the operation unit 18 (S11).

  When the subject emphasis mode is selected, the lens control unit 11 moves the photographing lens position from the ∞ position (∞ end) toward the shortest shooting distance position (macro end), and the lens position becomes the subject in-focus position. Automatic continuous shooting is performed in which the same subject is shot at two locations at the time and when the macro position is reached (S2) and (S3). Next, the images photographed in (S2) and (S3) are combined (S4).

  As shown in FIG. 3, the contrast value is large when the subject is in focus (lens position B), but is small when the subject is out of focus (lens position A). Further, a distant background image captured at the lens position A in the macro mode has a smaller contrast value and a greater degree of out-of-focus than a distant background image captured at the lens position B focused on the subject. When the difference between the contrast values of the two images taken in (S2) and (S3) is taken, the difference between the areas focused on the subject is larger than the difference between the background areas not focused. Therefore, the composite part is extracted by determining the subject and the background from the difference between the contrast values of the two images.

  Note that, as shown in FIG. 3, in an image captured in the subject in-focus state (lens position B), the subject contrast value is larger than the background contrast value. It is also possible to extract a region having a high contrast value by determining it as a subject and extracting a region having a low contrast value as a background from an image captured with setting B).

  In the image composition of (S4), as shown in FIG. 4, the extracted subject portion is fitted with a portion having a large contrast value obtained by photographing the lens position in the subject in-focus state, and the extracted background portion is A composition process is performed by fitting a portion having a small contrast value captured in the macro mode imaging state (S5). Therefore, the synthesized image is an image in which the blur of the background is more emphasized compared to an image (image B) that was shot with the lens position set to the subject in-focus state by the AF function. The synthesized image is output and displayed on the display unit 16 (S5), and is stored according to a storage instruction (S6) from the operation unit 18 (S7).

  FIG. 5 is a flowchart showing a second embodiment of the present invention, and FIGS. 6 to 7 are operation explanatory views of the present embodiment.

  In the present embodiment, the same subject is used in a mode in which automatic continuous shooting is performed by setting the lens position to three positions of the shortest shooting distance position (macro edge) / subject focus position / ∞ position (hereinafter referred to as multi-stage AF mode). The lens is photographed at the ∞ focus position, the subject focus position, and the macro focus position, and by combining these three shot images as appropriate, in addition to the subject-enhanced image, The user can obtain a favorite composite image such as an image. Hereinafter, the operation of the present embodiment will be described with reference to FIGS. 1 and 5 to 7.

  Whether or not to use the multi-stage AF mode is determined according to the operation of the operation unit 18 while the camera is activated (S21). When the multi-stage AF mode is not used, the normal shooting mode is shifted. In the normal shooting mode, the subject is shot once in the AF mode focused on the subject (S29). The captured image is output to the display unit as it is, displayed (S30), and stored in accordance with a storage instruction (S31) from the operation unit 18 (S32).

  When the multi-stage AF mode is selected by operating the operation unit 18, the multi-stage AF mode is entered. In multi-stage AF mode shooting, the lens position is moved from the ∞ position (∞ end) to the shortest shooting distance position (macro end) by the lens control unit 11 while the lens position is ∞ position / subject focus position / macro position. Then, automatic continuous shooting is performed in which the same subject is shot at three locations (S21).

  In this case, as shown in FIG. 6, the lens starts moving from the ∞ side, shoots at the ∞ position (C), shoots at the subject in-focus position (B), and finally the macro position (A ) The photographing result is stored in the memory unit 14 and used in the subsequent composition processing. In the embodiment, the lens movement is started from the ∞ side. However, the present invention is not limited to this, and the operation may be started from the macro side.

  In the present embodiment, the synthesis process differs depending on the synthesis method selection result (S23). In the composition processing when subject emphasis focus is performed, the images photographed in FIGS. 6A and 6B are composed (S24). In the synthesis process in the case of performing full focus, the images photographed in FIGS. 6B and 6C are synthesized (S25). In the synthesis process in the case of performing full blurring, the images photographed in FIGS. 6A and 6C are synthesized (S26).

  As shown in FIG. 7, an image taken at the position (A) is out of focus, and an image taken at the position (C) is always obtained as an image with a close range being out of focus. Further, an image in which the subject is in focus is obtained at the position (B). By combining these three patterns of images, image synthesis in each case is realized. The composition location in the image is determined according to the composition processing method by comparing the contrast values of the captured images.

  The image composition processing method using the contrast value is the same as that in the first embodiment described with reference to FIG. In FIG. 3, the composition processing method at the time of subject emphasis focus is described as an example. However, at the time of full focus, a portion where the contrast value of two images is large is superimposed, and at the time of full blur, a portion where the contrast value is small Are superimposed. After the composition, the image is displayed on the display unit 16 (S27), and is stored according to a storage instruction (S28) from the operation unit 18 (S29).

  According to this embodiment, in addition to the subject emphasis focus, it is possible to obtain a composite image of full focus or full focus. Further, as a mode other than that described in the present embodiment, it is also possible to select and perform synthesis of an image in which only the background is clear and the subject is out of focus.

  In the above embodiment, two images are combined, but it is also possible to combine three images taken at the lens positions (A), (B), and (C).

  For example, when there is a foreground between the subject and the camera, the contrast value of the captured image of (A) is compared with the captured image of (B) by comparing the captured images captured at the lens positions of (A) and (B). An area larger than the contrast value is extracted as the foreground. At the time of image composition, the corresponding area of the photographed image photographed at the lens position (C) is inserted into the extracted foreground area. In this way, it is possible to efficiently blur the surrounding scene of the subject including the foreground and the background, and it is possible to synthesize an image in which the subject is more emphasized.

  When full focus composition is selected, the foreground, subject, and background are covered by fitting the corresponding area of the photographed image taken at the lens position (A) into the extracted foreground area. This makes it possible to synthesize a more focused image.

It is a functional block diagram which shows the principal part structure of the digital camera to which this invention is applied. It is a flowchart which shows the 1st Embodiment of this invention. It is operation | movement explanatory drawing of 1st Embodiment. It is operation | movement explanatory drawing of 1st Embodiment. It is a flowchart which shows the 2nd Embodiment of this invention. It is operation | movement explanatory drawing of 2nd Embodiment. It is operation | movement explanatory drawing of 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Lens control part 12 Image sensor part 13 Control part 14 Memory part 15 Image processing part 16 Display part 17 Storage part 18 Operation part

Claims (9)

  While shooting the subject, the focus of the taking lens is moved from the shortest shooting distance position (macro end) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction, while the shortest shooting distance position, the in-focus position of the subject, and the ∞ position. Each of the subject is photographed, the subject region and the background region of the subject are extracted from the contrast value of each photographed image, and the extracted subject region and the background region according to the synthesis method selection instruction, A photographed image composition method comprising combining the subject area and the background area selected from the images photographed at the shortest photographing distance position, the in-focus position of the subject and the ∞ position into one image.   While shooting the subject, the focus of the taking lens is moved from the shortest shooting distance position (macro end) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction, while the shortest shooting distance position, the in-focus position of the subject, and the ∞ position. Each of the subject is photographed, and the subject region, the background region and the foreground region of the subject are respectively extracted from the contrast value of each photographed image, and the extracted subject region and background region and For the foreground area, combining the subject area, the background area, and the foreground area selected from the images captured at the shortest shooting distance position, the in-focus position of the subject and the ∞ position, and combining them into one image. A method for synthesizing a photographed image. 3. An area having a large contrast difference between an image photographed at the shortest photographing distance position and an image photographed at an in-focus position of the subject is extracted as the subject region. 3. A method for synthesizing a captured image described in 1. 3. The captured image composition method according to claim 1, wherein a region having a large contrast value is extracted as the subject from an image photographed at the in-focus position of the subject. While shooting the subject, the photographic lens is moved from the shortest shooting distance position (macro edge) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction. Subject photographing means for photographing the subject;
Subject / ambient scene extraction means for extracting a subject region and a background region of the subject from the contrast value of each photographed image;
The subject area and background area selected from the images taken at the shortest shooting distance position, the in-focus position of the subject, and the ∞ position with respect to the extracted subject area and background area in accordance with a synthesis method selection instruction Photographic image composition means for combining the images into one image;
A digital camera characterized by comprising: While shooting the subject, the photographic lens is moved from the shortest shooting distance position (macro edge) to the ∞ position direction or from the ∞ position to the shortest shooting distance position direction. Subject photographing means for photographing the subject;
Subject / ambient scene extraction means for extracting a subject area, a background area of the subject, and a foreground area from the contrast value of each captured image;
The subject area selected from the images taken at the shortest shooting distance position, the in-focus position of the subject, and the ∞ position with respect to the extracted subject area, background area, and foreground area in accordance with a synthesis method selection instruction And a photographed image composition means for combining the background region and the foreground region into one image,
A digital camera characterized by comprising: The subject / ambient scene extracting means extracts a region having a large difference in contrast value as the subject region between an image photographed at the shortest photographing distance position and an image photographed at the in-focus position of the subject. The digital camera according to claim 5 or 6 , characterized by comprising: The object / ambient scene extraction unit, from the image captured by the focus position of the subject, according to claim 5 or, characterized in that it has a function of extracting a larger area of the contrast value as the subject 6. The digital camera according to 6 . A mobile communication terminal with a camera function, wherein the digital camera according to any one of claims 5 to 8 is mounted.

Images