JP5239280B2 - Imaging apparatus, imaging control program, and imaging method - Google Patents

Description

  The present invention relates to an imaging apparatus that images a subject, an imaging control program used in the imaging apparatus, and an imaging method.

2. Description of the Related Art Conventionally, an image pickup apparatus for recording a better image is known from Patent Document 1 below. In this imaging apparatus, when the eye-meditation detection mode is set, continuous shooting is performed and a plurality of images are captured. Subsequently, the number of persons with eyes closed is detected from each of the plurality of images, and the image with the least number of persons with eyes closed is recorded on the memory card.
JP 2005-39365 A

However, an image in which all the subjects (each person) are captured in a good state does not always exist in a plurality of continuously shot images. Therefore, even if one of the best images is selected from a plurality of continuously shot images and recorded, some subjects may be unclear.
In other words, in the conventional technique for selecting the best image from a plurality of continuously shot images, when there are a plurality of subjects in the image, the best image is obtained even if some subjects are unclear. The case where it is selected as an image inevitably occurs. Therefore, it is impossible to surely obtain a clear image for all of the plurality of subjects present in the image.

  An object of the present invention is to provide an imaging apparatus, an imaging control program, and an imaging method that can obtain a clear image from a plurality of subjects present in an image.

In order to solve the above-described problem, the invention according to claim 1 is common to an imaging unit that captures a plurality of images including a plurality of subjects while changing the depth of field , and each image captured by the imaging unit. A subject recognizing unit for recognizing a plurality of subject portions, a selecting unit for selecting the most in- focus subject portion in each of a plurality of common subject portions in each image recognized by the subject recognizing unit, and the selecting unit. the selected object portion by an extraction means for respectively extracting from an image having the object portion, by combining the respective extracted object portion by the extracting means, and an image synthesizing means for generating a single combined image, It is characterized by providing.

According to a second aspect of the present invention, the image pickup means picks up a plurality of images while gradually changing the depth of field.

According to a third aspect of the present invention, the imaging means changes the depth of field by controlling a focal length of an optical system that forms a subject image on the imaging means. .

According to a fourth aspect of the present invention, the imaging means changes the depth of field by controlling a diaphragm arranged in front of itself.

The invention according to claim 5 further includes setting means for setting the number of images picked up by the image pickup means.

In the invention according to claim 6 , the image synthesizing unit selects any image having the subject portion selected by the selection unit as a reference image, and the selection unit selects the reference image. A single composite image is generated by combining subject portions of other images.

The invention according to claim 7 further includes recording means for recording a single synthesized image synthesized by the image synthesizing means.

The invention according to claim 8 further includes image display means for displaying a single synthesized image synthesized by the image synthesizing means.

According to the ninth aspect of the present invention, the imaging control means for controlling the computer included in the imaging apparatus including the imaging means to capture a plurality of images including a plurality of subjects while changing the depth of field. in each image captured by the imaging control unit, a plurality of object part recognizing object recognition means common in each of the plurality of object portion husband common in each image recognized by the object recognition unit, the best focus match selection means for selecting an object moiety, selected subject portion by the selection means, extracting means for respectively extracting from an image having the object portion, by combining the respective extracted object portion by the extraction means, single imaging control program for the combined image the image synthesizing means for generating a, characterized in that to the to function.

In the invention of claim 10 , the image capturing step of capturing a plurality of images including a plurality of subjects while changing the depth of field, and the images captured in the image capturing step are common. A subject recognition step for recognizing a plurality of subject portions, a selection step for selecting a subject portion that is most in focus in each of a plurality of common subject portions in each image recognized in the subject recognition step, and this selection the selected subject portion in step, an extraction step of respectively extracted from the image having the object portion, by combining the respective extracted object portion in this extraction step, the image synthesis to generate a single composite image And including a step

  According to the present invention, it is possible to obtain a clearer image of all subjects present in the image.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a schematic configuration of a digital camera to which an embodiment of the present invention is applied. This digital camera is configured around a CPU 1, and a ROM 2, a RAM 3, a key input unit 4, and an I / O port 5 are connected to the CPU 1. Further, the DRAM 1, the flash memory 8, the compression / expansion unit 9, the TG (Timing Generator) 10, the SG (Signal Generator) 11, the aperture driving unit 12, and the AF control unit 13 are connected to the CPU 1 through a bus. The ROM 2 stores an operation program of the CPU 1 and the like. The CPU 1 operates based on the operation program, and controls each unit using the RAM 3 as a work memory.

  The key input unit 4 includes a power key that is operated when the power is turned on / off, a shutter key that is operated when the video is stored in the flash memory 8, a switching from the recording mode (REC mode) to the reproduction mode, and the reproduction mode. Various keys such as a mode switching key operated when switching from the recording mode to the recording mode and a reproduction key operated when reproducing an image based on the image data stored in the flash memory 8 are provided. The shutter key has a so-called half shutter function that can be half-pressed and fully pressed. A serial I / O terminal 14 is connected to the I / O port 5. The I / O port 5 is an interface for inputting and outputting a video signal converted into a serial signal to other devices such as a general personal computer via the serial I / O terminal 14.

  The DRAM 7 is a memory that temporarily stores the captured digital video signal, and the flash memory 8 is a memory that stores the compressed video signal. The compression / decompression unit 9 performs compression / decompression processing by encoding / decoding the captured digital video signal. A CCD 16 is connected to the TG 10 via a drive circuit (Driver) 15. The drive circuit 15 is controlled by a timing signal generated by the TG 10 and drives the CCD 16 based on the timing signal.

  A fixed lens 17, a focus lens 18, and a diaphragm 19 are disposed on the front side of the CCD 16. The focus lens 18 is held by a drive mechanism 20 composed of an AF motor or the like that operates according to a drive signal generated by the AF control unit 13, and is measured using an AF sensor 21 connected to the AF control unit 13. A focusing operation for moving back and forth on the optical axis is performed based on the subject distance.

  The diaphragm 19 is driven by a driving signal generated by the diaphragm driving unit 12 and adjusts the amount of light of the subject image incident on the CCD 16. The CCD 16 is connected to the TG 10 via an amplifier 22 and an A / D converter 23. The A / D converter 23 converts the analog video signal captured by the CCD 16 into a digital signal and outputs it to the TG 10.

  A VRAM 24 and a D / A converter 25 are connected to the SG 11. The SG 11 adds a synchronization signal or the like to the luminance signal and color signal read from the flash memory 8 and then expanded by the compression / decompression unit 9, and the luminance signal and color signal transferred from the TG 10, that is, the digital video signal. Thus, the digital video signal is created, and the digital video signal created in SG11 is stored in the VRAM 24.

  The D / A converter 25 converts the digital video signal fetched from the SG 11 into an analog video signal. A video output terminal 27 and an LCD (Liquid Crystal Display) 28 are connected to the D / A converter 25 via an amplifier 26. The LCD 28 drives the liquid crystal based on the analog video signal sent from the D / A converter 25 to display an image. Further, a touch panel 29 is stacked on the LCD 28, and touch position data detected by the touch panel 29 is input to the CPU 1.

  The RAM 3 stores a subject-photographing order table 31 as shown in FIG. This subject-photographing order table 31 stores the recognized subjects A, B, C,... In the image captured by continuous shooting described later in the vertical axis direction and continuous shooting in the horizontal axis direction. The first image, the second image, the third image,... In the intersection area between the vertical axis and the horizontal axis, all “0” s are stored in the initial state, and the first to Nth sheets are the clearest (the most out of focus). The subjects A, B, C,... Are rewritten from “0” to “1” as shown in FIG.

  In the present embodiment with the above configuration, when the continuous shooting composition mode in the recording mode is set, the CPU 1 operates according to the flowchart shown in FIG. 3 based on the program. First, a through image is displayed on the LCD 28 (step S101). That is, the drive circuit 15 drives the CCD 16 based on the timing signal from the TG 10, and the electrical signal from the CCD 16 is input to the SG 11 via the amplifier 22, the A / D converter 23, and the TG 10. In SG 11, a synchronization signal or the like is added to the digital image signal to create a digital video signal, which is written into the VRAM 24 and is given to the LCD 28 via the D / A converter 25 and the amplifier 26. As a result, as shown in FIG. 4, a through image 281 formed by the fixed lens 17 and the focus lens 18 is displayed on the LCD 28.

  In this example, it is assumed that subjects A, B, C, and D located at different distances from the digital camera exist in the through image 281 as shown in FIG.

  Next, it is determined whether or not the shutter key is half-pressed (step S102). If the shutter key is half-pressed, AF processing is executed (step S103). That is, in the through image 281, the AF sensor 21 measures the distance up to the subject A located at the center, and the focus lens 18 is driven according to the measured subject distance to focus on the subject A.

Next, it is determined whether or not the shutter key has been fully pressed (step S104). If the shutter key is fully pressed, the depth of field is changed (step S105). The depth of field is changed N times as will be described later. For example, in the first change, the depth of field is reduced by a predetermined amount from the state in which the AF is performed (step S103). Thereafter, the depth of field may be gradually changed in a gradually increasing direction, or conversely, the depth of field may be increased by a predetermined amount and then gradually changed in a gradually decreasing direction.
Thereby, the depth of field can be changed efficiently.

In changing the depth of field, any of the following methods (1) to (3) may be used.
(1) The diaphragm drive unit 12 drives only the diaphragm 19 to change the depth of field.
(2) Only the focus lens 18 is driven by the AF control unit 13 to change the depth of field.
(3) The diaphragm drive unit 12 drives the diaphragm 19 and the AF control unit 13 drives the focus lens 18 to change the depth of field.
In any of (1), (2), and (3), the depth of field can be changed.

  Note that one of the depths of field changed in step S105 preferably includes the depth of field during AF executed in step S103.

  Subsequently, the image data output from the CCD 16 with the depth of field changed is captured (step S106) and stored in the DRAM 7 (step S107). Next, the value of the counter C that counts the number of shots is incremented (step S108), and it is determined whether or not the incremented value of the counter C reaches the set number N (step S109). Here, the set number N is the number of continuous shots set in advance by the user.

  Therefore, the user can arbitrarily set the number of continuous shots in this continuous shooting combination mode.

  If the result of determination in step S109 is C ≠ N, the process returns to step S105. Therefore, the loop of steps S105 to S109 is repeated N times until C = N, and N images taken with different depths of field are stored in the DRAM 7. Therefore, if the user has set “N = 5”, for example, five images captured with different depths of field are stored in the DRAM 7.

  Next, in the plurality of images stored in the DRAM 7, a common subject portion is recognized and stored in the subject-photographing order table 31 (step S110). The recognition of the subject in step S110 is, for example, based on the luminance in each image stored in the DRAM 7 and detecting the edge of the subject based on the luminance difference between the portion and the other portion and surrounded by the edge. This is done by recognizing the selected area as one subject.

  Therefore, when this digital camera is fixed and the shutter key is fully pressed in the continuous shooting composition mode, the DRAM 7 has N sheets (5 sheets) each having subjects A, B, C, and D as shown in FIG. ) Common objects A, B, C, and D are recognized. As shown in FIG. 2A, the subject-photographing order table 31 stores the recognized subjects A, B, C, and D, and the photographing order of images taken continuously in the horizontal axis direction. The first sheet, the second sheet, the third sheet, the fourth sheet, and the fifth sheet are stored. At this time, “0” is all stored in the intersection area of the vertical axis and the horizontal axis.

  Subsequently, subject selection processing is executed (step S111). In this subject selection process, the clearest subject (the least in-focus blur) is selected from the common subjects A, B, C, and D in the images with different depths of field recognized in step S110. To do. When selecting the clearest subject (the least in-focus), for example, the contrast at the center of each subject is detected based on the luminance signal at that location, and the subject with the highest contrast peak is selected. Make the subject clear (the most out of focus). In the subject-photographing order table 31, “0” in the area corresponding to the selected subject and photographing order is rewritten to “1”.

  Therefore, for example, the first image for subject A, the second image for subject B, the fourth image for subject C, and the fifth image for subject D were the clearest (the most in-focus). 2 (b), in the subject-photographing order table 31, the subject A and the first image, the subject B and the second image, the subject C and the fourth image, and the subject “1” is written in each intersection area of D and the fifth sheet.

  Next, subject part extraction processing is executed (step S112). In the subject portion extraction process, the corresponding subject portion is extracted from the image having the photographing order “1” with reference to the subject-photographing order table 31 shown in FIG. That is, as shown in FIG. 5, (a) the subject A portion is extracted from the first image, (b) the subject B portion is extracted from the second image, and (c) the third image is extracted. No subject portion is extracted from (d) the subject C portion is extracted from the fourth image, and (e) the subject D portion is extracted from the fifth image.

  Then, these extracted subject portions are combined to generate (f) a single combined image (step S113). This makes it possible to obtain a single composite image in which all subjects existing in the captured image are clearer.

  When generating the composite image, any one of the images (first to fifth images) from which the subject is selected in step S111, for example, the first image is used as a reference image, and the reference image is displayed. The selected subject portion (subjects B, C, and D) is cut out from the other images (second, fourth, and fifth images). The cut out subject portions (subject portions B, C, and D) are combined with the corresponding subject portions B, C, and D of the first image as the reference image.

  In this way, by synthesizing the subject portion using a whole image as a reference image, the cut out subject portions (subject portions B, C, and D) are arranged at appropriate positions in the background image by a simple synthesis process. And can be synthesized.

  Of course, the image composition is not limited to such image composition, and other image composition methods such as cutting out all selected subject portions (subjects A, B, C, and D portions) and compositing them with the background image may be used.

  Next, the synthesized image is displayed on the LCD 28 (step S114). Thereby, the user can confirm the synthesized image.

  Subsequently, the synthesized image is recorded in the flash memory 8 (step S115). As a result, a clear image can be recorded in the flash memory 8 with all the subjects A to D existing at different positions.

  Also, it is determined whether or not switching from the continuous shooting composition mode to another mode has been performed (step S116). If the continuous shooting composition mode is set, the processing from step S101 is repeated, and the processing according to this flow is terminated when the mode is shifted to another mode.

  FIG. 6 is a flowchart showing a processing procedure in another embodiment of the present invention. In this flowchart, the processes in steps S201 to S209 are the same as those in steps S101 to S109 in the flowchart of FIG. 3 described above. That is, a through image is displayed on the LCD 28 (step S201), and it is determined whether or not the shutter key is half-pressed (step S202). If the shutter key is half-pressed, AF processing is executed (step S203). . Next, it is determined whether or not the shutter key is fully pressed (step S204). If the shutter key is fully pressed, the depth of field is changed (step S205). For the change of the depth of field, any of the methods (1) to (3) may be used as in the above-described embodiment.

  Subsequently, the image data output from the CCD 16 with the depth of field changed is captured (step S206) and stored in the DRAM 7 (step S207). Further, the value of the counter C that counts the number of shots is incremented (step S208), and it is determined whether or not the incremented value of the counter C reaches the set number N (step S209). If the result of determination in step S209 is C ≠ N, the process returns to step S205. Therefore, the loop of steps S205 to S209 is repeated N times until C = N, and N images captured with different depths of field are stored in the DRAM 7. Therefore, if the user has set “N = 5”, for example, five images captured with different depths of field are stored in the DRAM 7.

  Next, in the plurality of images stored in the DRAM 7, a common subject portion is recognized and stored in the subject-photographing order table 31 (step S210). As a result, the DRAM 7 recognizes the common subjects A, B, C, and D from N (five) images each having the subjects A, B, C, and D as shown in FIG. . As shown in FIG. 2A, the subject-photographing order table 31 stores the recognized subjects A, B, C, and D, and the photographing order of images taken continuously in the horizontal axis direction. The first sheet, the second sheet, the third sheet, the fourth sheet, and the fifth sheet are stored. At this time, “0” is all stored in the intersection area of the vertical axis and the horizontal axis.

  Subsequently, an image display process is executed, and the first image of the plurality of images stored in the DRAM 7 is displayed on the LCD 28 (step S211). Then, in the state where the subject image is displayed on the LCD 28 for the first image, the user visually recognizes this and determines that the plurality of subjects A to D are the clearest (the focus is the least in focus). The subject is touched with a finger or the like to select the subject. Then, the determination in step S212 is YES, and the subject selection process in step 214 is executed. In this subject selection process, the CPU 1 detects a touch position based on an output signal from the touch panel 29, selects a subject corresponding to the detected touch position, and shoots the selected subject and the image in the subject-photographing order table 31. Rewrite “0” in the area corresponding to the rank to “1”.

  Therefore, if the subject A is touched in the state where the first image 282 is displayed on the LCD 28 as shown in FIG. 5A, the subject-photographing order table is shown in FIG. 2B. In 31, “0” in the intersection area corresponding to “first image” and “subject A” is rewritten to “1”.

  Thereafter, it is determined whether or not all the subjects (subjects A to D) recognized in step S210 have been selected (step S215). If they have not been selected, the process returns to step S211. The next image of the plurality of images stored in the DRAM 7 is displayed on the LCD 28. If the determination in step S212 is NO and there is no subject selection operation by touch, when a predetermined time has elapsed (step S213; YES), the process returns to step S211 and the plurality of sheets stored in the DRAM 7 are stored. The next image in the image is displayed on the LCD 28.

  Accordingly, as shown in FIG. 5, for example, the user displays (a) the subject A from the first image, (b) the subject B from the second image, and (c) three images If no subject is selected from the eye image, (d) subject C is selected from the fourth image, and (e) subject D is selected from the fifth image, FIG. Thus, in the subject-photographing order table 31, “1” is written in each intersection area of the subject A and the first image, the subject B and the second image, the subject C and the fourth image, and the subject D and the fifth image. .

  In this manner, in the subject-photographing order table 31, when “1” is written corresponding to the subjects A to D, all the subjects (subjects A to D) have been selected, and the determination in step S215 is YES. It becomes. Therefore, the process proceeds from step S215 to step S216, and in steps S216 to S220, the same processing as steps S112 to S1116 in the above-described embodiment is executed.

  That is, the subject portion extraction process is executed (step S216), and the subject portion corresponding to the subject order from the image with the photographing order “1” is referred to with reference to the subject-photographing order table 31 shown in FIG. Extract. That is, as shown in FIG. 5, (a) the subject A portion is extracted from the first image, (b) the subject B portion is extracted from the second image, and (c) three images are extracted. No subject portion is extracted from the eye image, (d) the subject C portion is extracted from the fourth image, and (e) the subject D portion is extracted from the fifth image.

  Then, these extracted subject portions are combined to generate (e) a single combined image (step S217). In generating the composite image, any one of the images (first to fifth images) in which the subject is selected in step S214, for example, the first image is used as a reference image, and the reference image is displayed. The selected subject portions (subject portions B, C, and D) are cut out from the other images (second, fourth, and fifth images). The cut out subject portions (subject portions B, C, and D) are combined with the corresponding subject portions B, C, and D of the first image as the reference image. Next, the synthesized image is displayed on the LCD 28 (step S218). Thereby, the user can confirm the synthesized image.

  Subsequently, the synthesized image is recorded in the flash memory 8 (step S219). As a result, a clear image can be recorded in the flash memory 8 by all the subjects A to D selected by the user and present at different positions.

  Further, it is determined whether or not switching from the continuous shooting composition mode to another mode has been performed (step S220). If the continuous shooting composition mode is set, the processing from step S201 is repeated, and the processing according to this flow is terminated when the mode is shifted to another mode.

  In the embodiment, AF is performed in step S103 or step S203. However, continuous shooting may be performed by sequentially changing the depth of field without performing AF. In this case, if the number of continuous shots is not settable by the user and the continuous shooting is performed with a larger number of predetermined numbers, a clear (in-focus, less out-of-focus) subject in the continuously shot images It is possible to reliably include an image having

  In the embodiment, the composite image displayed in step S114 or step S218 is always recorded in the flash memory 8 in step S115 or step S219. However, an OK key is provided in the key input unit 4, and the OK is performed. The composite image may be recorded in the flash memory 8 only when the key is operated.

  In this embodiment, the depth of field is changed in step S105 or step S205. However, continuous shooting may be performed without changing the depth of field. In this case, when a dynamic subject is included in a plurality of subjects, a clear subject portion is combined with a subject portion that has become blurred due to the movement of the dynamic subject, so that all subjects are clear. A simple composite image can be generated. For example, in a group photo, even if a person blinks and there is a subject person whose eyes are unclear, by combining the person subject part before blinking on the person subject, Group photos with open eyes can be recorded.

  Further, in the other embodiment shown in FIG. 5, the subject is selected based on the touch position data detected by the touch panel 29. However, a pointer is displayed on the LCD 28 and a cursor for moving the pointer is displayed. The subject may be selected by operating the key and the enter key.

It is a schematic block diagram of a digital camera to which an embodiment of the present invention is applied. 3 is a conceptual diagram illustrating a memory configuration of a subject-photographing order table. It is a flowchart which shows the process sequence in one embodiment of this invention. It is a figure which shows the example of 1 embodiment of this invention, and another embodiment display screen. It is explanatory drawing which shows selection of an image, extraction of a subject part, and single image composition. It is a flowchart which shows the process sequence in other embodiment of this invention.

Explanation of symbols

1 CPU
2 ROM
3 RAM
4 Key input part 7 DRAM
8 Flash memory 10 TG
11 SG
12 Aperture Drive Unit 13 AF Control Unit 14 Serial I / O Terminal 16 CCD
17 Fixed Lens 18 Focus Lens 20 Drive Mechanism 21 AF Sensor 28 LCD
29 Touch Panel 31 Subject-Sequence Ranking Table 281 Through Image 282 First Image A Subject B Subject C Subject D Subject

Claims (10)

Imaging means for capturing a plurality of images including a plurality of subjects while changing the depth of field ;
Subject recognition means for recognizing a plurality of common subject portions in each image taken by the imaging means;
A selection means for selecting the most in- focus subject portion in each of a plurality of common subject portions in each image recognized by the subject recognition means;
Extraction means for extracting the subject portions selected by the selection means from the images having the subject portions ,
By combining the respective extracted object portion by the extracting means, and an image synthesizing means for generating a single combined image,
An imaging apparatus comprising: The imaging apparatus according to claim 1 , wherein the imaging unit captures a plurality of images while gradually changing the depth of field . The imaging apparatus according to claim 1, wherein the imaging unit changes the depth of field by controlling a focal length of an optical system that forms a subject image on the imaging unit. The imaging apparatus according to claim 1, wherein the imaging unit changes the depth of field by controlling a diaphragm disposed in front of the imaging unit. The imaging apparatus according to claim 1, further comprising a setting unit that sets the number of images to be captured by the imaging unit. The image synthesizing unit uses any image having the subject portion selected by the selection unit as a reference image, and synthesizes the subject portion of the other image selected by the selection unit with the reference image. The imaging apparatus according to claim 1, wherein one composite image is generated . The imaging apparatus according to claim 1, further comprising recording means for recording a single composite image synthesized by the image synthesis means . The imaging apparatus according to claim 1, further comprising image display means for displaying a single composite image synthesized by the image synthesis means . A computer included in an imaging apparatus including an imaging unit,
An imaging control means for controlling to capture a plurality of images including a plurality of subjects while changing the depth of field;
Subject recognition means for recognizing a plurality of common subject portions in each image taken by the imaging control means;
Selection means for selecting the most in-focus subject part in each of a plurality of common subject parts in each image recognized by the subject recognition means;
Extraction means for extracting the subject parts selected by the selection means from the images having the subject parts,
Image combining means for generating a single composite image by combining the subject portions respectively extracted by the extraction means;
An imaging control program that functions as a computer program. An imaging step of imaging a plurality of images including a plurality of subjects while changing the depth of field;
A subject recognition step for recognizing a plurality of common subject portions in each of the images taken in this imaging step;
A selection step for selecting the most in-focus subject portion in each of a plurality of common subject portions in each image recognized in the subject recognition step;
An extraction step for extracting the subject portions selected in the selection step from the images having the subject portions, respectively;
An image synthesis step for synthesizing the subject portions extracted in this extraction step to generate a single synthesized image;
An imaging method comprising:

Images