JP2009200924A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of separating a subject and a non-subject in an image even when a light scene is photographed. <P>SOLUTION: The imaging apparatus (11 to 26) which photographs one scene with high stroboscope light intensity and low stroboscope light intensity, and detects the border between a subject area and a non-subject area in the photographed image based on comparison between luminances of images acquired by the above photographing, includes a control means (20). The control means (20) controls timings of exposure of an imaging element (14) and emission of stroboscope light so that a period overlapping with a light non-emission period of stroboscope light in an exposure period of the imaging element (14) is shorter when the photography with high stroboscope light is performed than when automatic exposure mode photography is performed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of separating a subject and a non-subject in an image with high accuracy. Hereinafter, separation of a subject and a non-subject in an image is referred to as “subject separation”.

Many techniques for separating a subject and a non-subject in an image have been proposed. As an example, Patent Document 1 captures an image with illumination on an object and an image with no illumination, and extracts a region where the luminance changes from the two images as a subject region. Technology is known.
JP-A-10-210340

  However, with conventional technologies, for example, when shooting in a bright scene, such as when shooting portraits outdoors on a sunny day or indoors under bright illumination, the amount of ambient light is less than the amount of illumination light. Since it becomes large, the luminance change amount of the subject between the illuminated image and the non-illuminated image becomes small, and as a result, it is difficult to separate the subject and the non-subject with high accuracy.

  The present invention solves the above-described problems of the prior art. An object of the present invention is to provide an imaging apparatus capable of separating a subject and a non-subject in an image with high accuracy even when shooting a bright scene.

  The imaging device according to the first aspect of the present invention performs shooting with a high strobe light intensity and shooting with a low strobe light intensity for the same scene, and based on a luminance comparison of images acquired by the shooting, The imaging device that detects the boundary between the subject area and the non-subject area includes a control unit. This control means, when shooting with high strobe light intensity, the image sensor so that the period of the exposure period of the image sensor that overlaps the non-flash period of the strobe light is shorter than the overlap period in automatic exposure mode shooting. The timing of the exposure and the flash light emission is controlled.

  Here, “photographing with low strobe light intensity” includes non-flash photography.

  In a second aspect based on the first aspect, the control means makes the time difference between the start of exposure and the start of light emission smaller than the time difference between the start of exposure and the start of light emission in automatic exposure mode photography.

  In a third aspect based on the first aspect or the second aspect, the control means determines that the time difference between the end of exposure and the end of light emission is greater than the time difference between the end of exposure and the end of light emission in automatic exposure mode photography. Make it smaller.

  In a fourth invention according to any one of the first to third inventions, the control means matches the time from the start to the end of exposure with the time from the start to the end of light emission.

  In a fifth aspect based on any one of the first to fourth aspects, the control means matches the timing at which the opening degree of the shutter for controlling the exposure period of the image sensor becomes maximum with the timing of light emission.

  According to a sixth invention, in any one of the first to fifth inventions, there is provided noise reduction means for performing a noise reduction process on an image other than the actual photographed image used for luminance comparison.

  According to a seventh aspect of the present invention, in any one of the first to sixth aspects of the invention, a normalization for normalizing an image used for luminance comparison so that signal levels of non-subject areas between images used for luminance comparison are aligned. Further comprising:

  In an eighth invention according to any one of the first to seventh inventions, in photographing with high strobe light intensity, strobe light is emitted with a light emission amount larger than a light emission amount in automatic exposure mode photography.

  According to a ninth invention, in the eighth invention, there is further provided discrimination means for preferentially considering a whiteout area in an image acquired by photographing with high strobe light intensity as a subject area regardless of luminance comparison.

  A tenth aspect of the present invention is the image processing apparatus according to any one of the first to ninth aspects, further comprising image processing means for performing different image processing on the subject area and the non-subject area in the captured image.

  In an eleventh aspect based on the tenth aspect, the image processing means performs blur processing on the non-subject area.

  In the present invention, when subject separation is performed based on a luminance comparison of a plurality of images having different strobe light intensities captured in the same scene, when shooting an image having a high strobe light intensity among a plurality of images, The timing of the exposure of the image sensor and the emission of the strobe light is controlled so that the period in which the strobe light is not emitted in the exposure period is shorter than that in the automatic exposure mode shooting. Therefore, according to the present invention, by reducing the exposure amount of the image sensor due to ambient light and relatively increasing the ratio of the exposure amount of the image sensor due to the strobe light, it is possible to obtain an image with high and low strobe light intensity. Since the amount of deviation between the luminance change amount of the subject and the luminance change amount of the background can be increased, the accuracy of subject separation can be increased.

(Description of the first embodiment)
Hereinafter, the electronic camera of the first embodiment will be described.

  FIG. 1 is a block diagram of an electronic camera to which an imaging apparatus of the present invention is applied.

  The electronic camera includes an imaging lens 11 and a lens driving unit 12, a mechanical shutter 13, an imaging device 14, a signal processing unit 15, a timing generator (TG) 16, a buffer memory 17, a data processing unit 18, and a compression. / Decoding unit 19, control unit 20, photometry unit 21, monitor 22, recording medium 23, light emitting unit 24, operation unit 25, and bus 26. Here, the buffer memory 17, the data processing unit 18, the compression / decoding unit 19, the control unit 20, the photometry unit 21, the monitor 22, and the recording medium 23 are connected via a bus 26. In addition, the lens driving unit 12, the mechanical shutter 13, the signal processing unit 15, the TG 16, the light emitting unit 24, and the operation unit 25 are each connected to the control unit 20.

  The imaging lens 11 includes a plurality of lens groups including a focus lens and a zoom lens. For simplicity, the imaging lens 11 is illustrated as a single lens in FIG.

  The lens driving unit 12 generates a lens driving signal in accordance with an instruction from the control unit 20, performs focus adjustment and zoom adjustment by moving the imaging lens 11 in the optical axis direction, and subjects the light flux that has passed through the imaging lens 11 to the subject. An image is formed on the imaging surface of the image sensor 14.

  The mechanical shutter 13 is, for example, a lens shutter (behind the lens shutter) having a shutter blade that also serves as a diaphragm blade. The mechanical shutter 13 adjusts the light amount of the light beam reaching the imaging surface of the imaging element 14 by adjusting the opening amount of the shutter blades and controlling the opening / closing operation in accordance with an instruction from the control unit 20. The exposure time of the image sensor 14 by the light flux is controlled.

  The imaging element 14 is a CCD type or CMOS type imaging element, and is disposed on the image space side of the imaging lens 11. The image sensor 14 photoelectrically converts a subject image formed on the imaging surface to generate an analog image signal. The output of the image sensor 14 is connected to the signal processing unit 15.

  The signal processing unit 15 performs CDS (correlated double sampling), gain adjustment, A / D conversion, white balance adjustment, color separation on the analog image signal output from the image sensor 14 in accordance with an instruction from the control unit 20. Signal processing such as (interpolation) and gamma conversion is performed, and the processed image signal is output. Further, the signal processing unit 15 sets an adjustment amount for gain adjustment based on an instruction from the control unit 20, and thereby adjusts the photographing sensitivity corresponding to the ISO sensitivity. The output of the signal processing unit 15 is connected to the buffer memory 17.

  The TG 16 supplies timing pulses to the image sensor 14 and the signal processing unit 15 based on instructions from the control unit 20. The drive timing of the image sensor 14 and the signal processor 15 is controlled by the timing pulse.

  The buffer memory 17 temporarily stores the image signal output from the signal processing unit 15 as image data. The buffer memory 17 temporarily stores image data created in the course of processing by the control unit 20.

  The data processing unit 18 performs image processing such as blur processing on the image data in accordance with an instruction from the control unit 20.

  The compression / decoding unit 19 performs a compression process on the image data in accordance with an instruction from the control unit 20. The compression process is performed in JPEG (Joint Photographic Experts Group) format or the like.

  The photometry unit 21 is based on the image data recorded in the buffer memory 17 in accordance with an instruction from the control unit 20 by a known photometry method such as multi-segment photometry (multi-pattern photometry), center-weighted photometry, or spot photometry. An evaluation value indicating the brightness of the subject and the entire shooting scene and the brightness distribution of the shooting scene is calculated.

  The monitor 22 is an LCD monitor provided on the back surface of the electronic camera housing or the like, an electronic viewfinder having an eyepiece, and the like, and displays various images according to instructions from the control unit 20.

  Image data after compression processing is recorded on the recording medium 23 by the control unit 20. The recording medium 23 is a memory card incorporating a semiconductor memory, a small hard disk, or the like.

  The light emitting unit 24 drives an electronic flash or the like provided in the electronic camera housing based on an instruction from the control unit 20 to emit strobe light for illuminating the subject toward the object scene.

  The operation unit 25 includes operation members such as a release button and a shooting mode switching button, and sends an operation signal according to the content of the member operation by the user to the control unit 20.

  The control unit 20 performs overall control of each unit of the electronic camera according to the operation signal sent from the operation unit 25. For example, when receiving a half-press operation signal of the release button of the operation unit 25, the control unit 20 performs focus adjustment control (AF) of the imaging lens 11 in cooperation with the lens driving unit 12 prior to photographing. Further, the control unit 20 drives the photometry unit 21 to calculate the evaluation value of the shooting scene, and adjusts the setting contents of the signal processing unit 15 and the like based on the evaluation value.

  At the time of shooting, the control unit 20 determines the exposure condition based on the evaluation value calculated by the photometry unit 21, and under the determined condition, the lens driving unit 12, the mechanical shutter 13, the signal processing unit 15, the TG 16 and the light emitting unit. 24 is driven to take a picture. The shutter speed is set by a combination of the opening time of the mechanical shutter 13 and the charge accumulation time of the image sensor 14. Further, the image data of the photographed image is recorded in the buffer memory 17 via the signal processing unit 15. The control unit 20 drives the data processing unit 18 and the compression / decoding unit 19 to perform image processing and compression processing on the image data of the image recorded in the buffer memory 17 and record the processed image data. Recording on the medium 23.

  The subject separation operation of the electronic camera of the first embodiment will be described below with reference to the flowchart of FIG. The flow in FIG. 2 is executed when shooting in a “bright scene” where it is difficult to separate the subject in the prior art. Note that the “subject” here is an object that the user wants to shoot close to the camera, and the “background” is located farther from the camera than the subject. It is an object that is difficult for light to reach.

  S101: When the control unit 20 detects that the user has fully pressed the release button based on the operation signal received from the operation unit 25, the control unit 20 determines the exposure condition for the preliminary shooting. Here, the preliminary photographing is photographing for acquiring a preliminary photographing image, and the preliminary photographing image is an image used only for separating the subject and the background by the luminance comparison in S106 below. As the determination of the exposure conditions for the preliminary shooting, specifically, the timing of the opening / closing operation of the mechanical shutter 13, the exposure of the image sensor 14, and the strobe emission of the light emitting unit 24 are determined.

  Here, for description of exposure control for preliminary shooting, for comparison, exposure control when the electronic camera is in the automatic exposure mode will be described with reference to FIGS. 3 and 4.

  FIG. 3 shows the relationship between the control of the opening / closing operation of the mechanical shutter 13 and the exposure amount of the image sensor 14 thereby. Note that the light quantity indicated by the vertical axis in FIG. 3 is the brightness, that is, the light quantity incident on the image sensor 14 per unit time.

  The mechanical shutter 13 is opened based on the shutter opening function Fo (t) from time tc1 when photographing is started by fully pressing the release button, and is fully opened at time to1. The mechanical shutter 13 is maintained in a fully open state for a certain time (time to1 to to2), then closed based on the shutter closing function Fc (t) at the time to2 and completely closed at time tc2. It becomes. The shutter opening function Fo (t) is a function representing the characteristics when the mechanical shutter 13 is opened, and the shutter closing function Fc (t) is a function representing the characteristics when the shutter is closed. In FIG. 3, the outputs of these functions are shown as straight lines for simplicity.

  If the charge accumulation time of the image sensor 14 covers the period of tc1 to tc2, the exposure amount S of the image sensor 14 is incident on the image sensor 14 from such control of the opening / closing operation of the mechanical shutter 13. The total light amount is obtained by integrating the light amount over time tc1 to time tc2. This corresponds to the area of the trapezoidal portion of FIG. When the electronic camera is in the automatic exposure mode, the shutter opening / closing operation is normally controlled so that the exposure amount S becomes an appropriate light amount.

  FIG. 4 shows the relationship between the opening / closing operation of the mechanical shutter 13, the exposure of the image sensor 14, the strobe emission of the light emitting unit 24, and the exposure amount of the image sensor 14 in the shooting of the “bright scene” in the automatic exposure mode. Show.

  In the shooting of “bright scene”, since an appropriate exposure amount can be obtained in a short time, the opening / closing operation of the mechanical shutter 13 is controlled as shown in the portion of << shutter opening / closing >> of FIG. Specifically, the mechanical shutter 13 opens from the time tc1 based on the shutter opening function Fo (t) when shooting is started by fully pressing the release button, but the time before the fully opened state is reached. It starts to close at tc0. The mechanical shutter 13 is closed based on the shutter closing time function Fc (t) and is completely closed at time tc2. Further, the exposure of the image sensor 14 is controlled as shown in the portion of << CCD exposure >> in FIG. 4, and the image sensor 14 starts exposure as soon as the mechanical shutter 13 starts to open (time tc1). The exposure is finished simultaneously with the complete closing (time tc2). At this time, the exposure amount Se of the image sensor 14 due to the ambient light is expressed by Expression (1) based on the shutter opening function Fo (t) and the shutter closing function Fc (t).

  Here, in this shooting, if the strobe light is forcibly emitted to separate the subject, the strobe light emission of the light emitting unit 24 is controlled as shown in the portion of <strobe light emission> in FIG. The Rukoto. Specifically, the light emitting unit 24 adjusts the timing so that the amount of light emission becomes maximum when the mechanical shutter 13 is opened most widely (time tc0), and from the exposure time (time tc1 to tc2) of the image sensor 14. The flash light is emitted only for an extremely short time (time ts1 to ts2). In this case, the exposure amount Ss of the image sensor 14 by the strobe light is expressed by Expression (2) based on the shutter opening function Fo (t), the shutter closing function Fc (t), and the strobe light amount function Fs (t). The

  By the way, the separation of the subject and the background is based on the fact that when the strobe light is emitted, the subject near the electronic camera is more effective in applying the strobe light than the background far from the electronic camera. Is going. In other words, when comparing an image shot with flash light (flash emission image) and an image shot without flash light (flash non-flash image), the subject has a larger amount of brightness change than the background. . Therefore, the subject and the background can be separated by using the difference in luminance change amount.

  This will be specifically described below.

  According to the above equation (2), the exposure amount of the image sensor 14 by the strobe light reflected by the subject is Ss0, and the exposure amount of the image sensor 14 by the strobe light reflected by the background is Ssb. Then, according to the above formula (1), the exposure amount of the image sensor 14 by the ambient light reflected from the subject or the background is set to Se. Then, the exposure amount of the image sensor 14 due to the reflected light from the subject at the time of strobe light emission becomes Ss0 + Se, and the exposure amount of the image sensor 14 due to the reflected light from the background becomes Ssb + Se. In addition, the exposure amount of the image sensor 14 by the reflected light from the subject and the background when the strobe is not lighted is Se. Usually, the luminance of an image (here, the amount of signal generated by photoelectric conversion of the imaging surface of the imaging device 14 is determined as luminance) is determined by the exposure amount of the imaging device 14 by reflected light from an object present in the screen. The Therefore, the luminance of the subject in the flash emission image can be represented by (Ss0 + Se), and the luminance of the background can be represented by (Ssb + Se). In addition, the luminance of the subject and the background in the non-flash image can be represented by Se. Therefore, if the luminance change amount of the subject and the background is obtained based on the luminance ratio (luminance ratio), the luminance ratio of the subject in both images is (Ss0 + Se) / Se, and the luminance ratio of the background is (Ssb + Se) / Se. It becomes.

  From the above, the amount of deviation between the luminance ratio of the subject and the luminance ratio of the background in the strobe light emission image and the strobe non-light emission image is expressed by Expression (3).

  That is, if this deviation amount is increased, subject separation can be performed with high accuracy.

  However, in the shooting of “bright scenes”, the exposure amount Se of the image sensor 14 due to the ambient light is increased, and the deviation amount of the above formula (3) is decreased. As a result, the accuracy of subject separation is lowered. .

  Therefore, in the preliminary shooting of the first embodiment, exposure control is performed so as to reduce the exposure amount Se of the image sensor 14 by the ambient light in order to increase the accuracy of subject separation.

  The control unit 20 determines the exposure conditions for preliminary shooting as shown in FIG. Specifically, the opening / closing operation of the mechanical shutter 13 is the same as that in the automatic exposure mode, and the timing of exposure of the image sensor 14 and strobe light emission of the light emitting unit 24 is determined as follows.

  The control unit 20 determines the exposure start time as the time ts1 and the exposure end as the time ts2 as the exposure timing of the image sensor 14 (see the portion of << CCD exposure >> in FIG. 5). In addition, as the strobe light emission timing of the light emitting unit 24, the light emission start time is determined as the time ts1, and the light emission end time is determined as the time ts2 (refer to the portion of <strobe light emission> in FIG. 5). That is, the timing is determined so that the start and end of exposure of the image sensor 14 and the start and end of strobe light emission of the light emitting unit 24 are matched.

  By performing exposure control at such timing, the exposure of the image sensor 14 by ambient light is maintained while maintaining the exposure amount Ss of the image sensor 14 by strobe light as compared to the exposure control in the automatic exposure mode of FIG. The quantity Se can be set to the minimum value (Se1). At this time, the difference between the luminance ratio of the subject and the luminance ratio of the background in the strobe light emission image and the strobe non-light emission image is (Ss0−Ssb) / Se1 from the above equation (3). Here, the exposure amount of the image sensor 14 by the ambient light is Se1, and Se> Se1 as compared to the same exposure amount Se in the exposure control in the automatic exposure mode. Therefore, the relationship between the deviation amounts is {(Ss0−Ssb ) / Se1}> {(Ss0−Ssb) / Se}. Therefore, according to the exposure control of the electronic camera of the first embodiment, the amount of divergence can be made larger than the exposure control in the automatic exposure mode even when shooting a “bright scene”, so that the accuracy of subject separation can be improved.

  The control unit 20 proceeds to S102 after determining the exposure condition for the preliminary shooting as described above.

  S102: The control unit 20 performs the first preliminary shooting under the exposure condition determined in S101. However, in the first preliminary photographing, the control unit 20 performs photographing by driving the light emitting unit 24 to forcibly emit strobe light. The image data of the first preliminary image captured in the first preliminary image is recorded in the buffer memory 17.

  In the present embodiment, a through image is acquired as the first preliminary image. Here, the through image is an image obtained by a specific usage of the image sensor. Although the through image has a lower resolution than the image acquired at the time of actual photographing, it can be read out from the image sensor in a short time, and thus is suitable for obtaining information of a scene to be photographed in substantially real time.

  S103: The control unit 20 performs the second preliminary photographing under the exposure condition determined in S101. However, in the second preliminary shooting, the control unit 20 performs shooting while prohibiting the emission of strobe light. The image data of the second preliminary image captured in the second preliminary image is recorded in a different area from the first preliminary image in the buffer memory 17.

  In the present embodiment, a through image is acquired for the second preliminary captured image as in S102.

  S104: The control unit 20 determines the exposure condition for the main photographing to be a condition for appropriate exposure. The conditions for the proper exposure are the same as those determined in the automatic exposure mode. The main photographing is a step of photographing an ornamental image (main photographing image). In addition, the proper exposure means an exposure in which the maximum luminance and the minimum luminance in the screen to be photographed fall within the dynamic range of the image sensor 14 and an image without blackout or whiteout is photographed.

  S105: The control unit 20 performs the main photographing under the exposure condition determined in S104. The image data of the actual captured image captured by the actual capturing is recorded in a different area from the first and second preliminary captured images in the buffer memory 17.

  S106: The control unit 20 compares the luminance of the first preliminary image (strobe emission image) and the second preliminary image (strobe non-emission image) recorded in the buffer memory 17, and determines the luminance between the images. A region where the amount of change exceeds a predetermined threshold is extracted as a subject region, and subject separation information for separating the subject and the background is created. Note that the luminance comparison can be performed based on the luminance difference or the luminance ratio, but here it is performed based on the luminance ratio. When subject separation is performed based on the luminance ratio, the reflectance of the subject is canceled in the process of luminance comparison, and therefore, high-accuracy separation that does not depend on the reflectance of the subject is possible (luminance based on the luminance ratio). A technique for extracting a subject by comparison has been proposed in Japanese Patent Laid-Open No. 2005-130268).

  S107: The control unit 20 separates the subject and the background from the actual captured image in the buffer memory 17 based on the subject separation information created in S106. Here, since the subject separation information obtained in S106 is obtained by luminance comparison between through images, the resolution is lower than that of the actual captured image and cannot be directly applied to subject separation of the actual captured image. Therefore, the subject separation information is converted into the resolution of the main photographic image by resolution conversion processing, and after the subject separation information and the resolution of the main photographic image are matched, processing for separating the subject and background of the main photographic image is performed.

  S108: The control unit 20 drives the data processing unit 18 to perform different image processing on the subject and background regions separated in S107. Here, for example, blur processing is performed only on the background. The blur processing is executed by, for example, a filter operation (convolution operation) of a point spread function, but is not limited thereto, and can be performed using various digital filters such as an averaging filter and a Gaussian filter.

  S109: The control unit 20 drives the monitor 22 to display the image after the image processing in S108 on an LCD monitor or the like. The user can confirm the quality of the image by the display.

  S110: The control unit 20 drives the compression / decoding unit 19, performs compression processing on the image after image processing, and records the compressed image on the recording medium 23.

(Operational effects of the first embodiment)
As described above, in the electronic camera according to the first embodiment, the exposure condition is determined so that the exposure timing of the image sensor 14 and the start and end timings of the strobe emission of the light emitting unit 24 are matched, and the preliminary shooting is performed under the exposure condition. Done. By performing exposure control in this way, it is possible to minimize the exposure amount of the image sensor 14 by ambient light while maintaining the exposure amount of the image sensor 14 by strobe light. Therefore, in the electronic camera according to the first embodiment, even when “bright scene” is captured, the difference between the luminance ratio of the subject and the luminance ratio of the background between the first preliminary captured image and the second preliminary captured image. The amount can be increased, and the accuracy of subject separation is increased.

(Description of Second Embodiment)
Next, an electronic camera according to a second embodiment will be described. In the following description, the same reference numerals are given to the components of the electronic camera that are common to the first embodiment, and redundant description is omitted. Further, the description of the subject separation operation of the electronic camera common to the first embodiment is also omitted.

  The second embodiment is a modification of the first embodiment. In the electronic camera according to the second embodiment, the exposure condition for the preliminary shooting is determined as follows by changing the process of S101 of the first embodiment.

  S101: When the control unit 20 detects that the release button is fully pressed by the user based on the operation signal received from the operation unit 25, the control unit 20 determines the exposure condition for the preliminary shooting as shown in FIG. Specifically, the opening / closing operation of the mechanical shutter 13 is the same as that in the automatic exposure mode, and the timing of exposure of the image sensor 14 and strobe light emission of the light emitting unit 24 is determined as follows.

  The control unit 20 makes the timing difference between the start of exposure of the image sensor 14 and the start of strobe emission of the light emitting unit 24 smaller than the timing difference (ts1−tc1) in the exposure control in the automatic exposure mode (FIG. 4). The exposure start of the image sensor 14 is determined at time tc1 ′, and the strobe light emission start of the light emitting unit 24 is determined at the timing of time ts1. As a result, the timing difference becomes ts1−tc1 ′. In addition, the control unit 20 captures an image so that the timing difference between the end of exposure of the image sensor 14 and the end of strobe emission of the light emitting unit 24 is also smaller than the timing difference (tc2-ts2) in the exposure control in the automatic exposure mode. The end of exposure of the element 14 is determined at time tc2 ′, and the end of strobe light emission of the light emitting unit 24 is determined at the timing of time ts2 (see “CCD exposure” and “Strobe light emission” in FIG. 6). As a result, the timing difference becomes tc2'-ts2.

  By performing exposure control at such timing, the exposure amount of the image sensor 14 by the ambient light is set to Se while maintaining the exposure amount Ss of the image sensor 14 by the strobe light as compared to the exposure control in the automatic exposure mode. To Se2. At this time, the divergence amount between the luminance ratio of the subject and the luminance ratio of the background in the first preliminary image (strobe light emission image) and the second preliminary image (strobe non-light emission image) is expressed by the above equation (3) ( Ss0−Ssb) / Se2. Here, the exposure amount of the image sensor 14 due to the ambient light is Se2, and Se> Se2 as compared with the same exposure amount Se in the exposure control in the automatic exposure mode. Therefore, the relationship between the deviation amounts is {(Ss0−Ssb ) / Se2}> {(Ss0−Ssb) / Se}. Therefore, according to the exposure control of the electronic camera of the second embodiment, the amount of divergence can be made larger than the exposure control in the automatic exposure mode even in the shooting of “bright scene”, so that the accuracy of subject separation is improved.

  The exposure control of the electronic camera of the second embodiment can be used as a more practical means when it is difficult to accurately realize the exposure control of the electronic camera of the first embodiment, which is an ideal exposure control. it can. When this is used, the timing difference (ts1−tc1 ′) between the start of exposure of the image sensor 14 and the start of strobe light emission of the light emitting unit 24 and its exposure so that the exposure amount Se2 of the image sensor 14 due to the ambient light described above becomes as small as possible. It is desirable to minimize the timing difference (tc2′−ts2) between the end of exposure and the end of strobe light emission. However, if the exposure control is performed so that the exposure amount Se2 of the image sensor 14 by the ambient light is smaller than the exposure amount Se of the image sensor 14 by the ambient light in the exposure control in the automatic exposure mode, the accuracy of subject separation is reduced. The effect which improves can be acquired.

  After determining the exposure conditions for the preliminary shooting in this way, the control unit 20 proceeds to S102.

(Operational effect of the second embodiment)
As described above, in the electronic camera according to the second embodiment, the exposure condition is set so that the difference between the start timings of the exposure of the image sensor 14 and the strobe emission of the light emitting unit 24 is reduced and the difference between the end timings is reduced. Then, preliminary shooting is performed under the exposure conditions. By performing exposure control in this way, it is possible to reduce the exposure amount of the image sensor 14 by ambient light while maintaining the exposure amount of the image sensor 14 by strobe light. Therefore, in the electronic camera of the second embodiment, even in the case of shooting a “bright scene”, the difference between the luminance ratio of the subject and the luminance ratio of the background between the first preliminary shooting image and the second preliminary shooting image. The amount can be increased, and the accuracy of subject separation is increased.

(Description of the third embodiment)
Next, an electronic camera according to a third embodiment will be described. In the following description, the same reference numerals are given to the components of the electronic camera that are common to the first embodiment, and redundant description is omitted. Further, the description of the subject separation operation of the electronic camera common to the first embodiment is also omitted.

  The third embodiment is a modification of the first embodiment. In the electronic camera of the third embodiment, the exposure condition for the preliminary shooting is determined as follows by changing the process of S101 of the first embodiment.

  S101: When the control unit 20 detects that the release button is fully pressed by the user from the operation signal received from the operation unit 25, the control unit 20 determines the exposure condition for the preliminary shooting as shown in FIG. Specifically, the opening / closing operation of the mechanical shutter 13 is the same as that in the automatic exposure mode, and the timing of exposure of the image sensor 14 and strobe light emission of the light emitting unit 24 is determined as follows.

  The control unit 20 makes only the timing difference between the exposure start of the image sensor 14 and the strobe emission start of the light emitting unit 24 smaller than the timing difference (ts1−tc1) in the exposure control in the automatic exposure mode (FIG. 4). Then, the exposure start of the image sensor 14 is determined at time tc1 ′, and the strobe light emission start of the light emitting unit 24 is determined at the time ts1 (see “CCD exposure” and “Strobe light emission” in FIG. 7). As a result, the timing difference becomes ts1−tc1 ′.

  By performing exposure control at such timing, the exposure amount of the image sensor 14 by the ambient light is set to Se while maintaining the exposure amount Ss of the image sensor 14 by the strobe light as compared to the exposure control in the automatic exposure mode. To Se3. At this time, the divergence amount between the luminance ratio of the subject and the luminance ratio of the background in the first preliminary image (strobe light emission image) and the second preliminary image (strobe non-light emission image) is expressed by the above equation (3) ( Ss0−Ssb) / Se3. Here, the exposure amount of the image sensor 14 by the ambient light is Se3, and Se> Se3 as compared to the same exposure amount Se in the exposure control in the automatic exposure mode, so the relationship between the deviation amounts is {(Ss0−Ssb ) / Se3}> {(Ss0−Ssb) / Se}. Therefore, according to the exposure control of the electronic camera of the third embodiment, the amount of divergence can be made larger than the exposure control in the automatic exposure mode even in the shooting of “bright scene”, so that the accuracy of subject separation is improved. In general, when light is incident on the imaging surface after the exposure of the imaging device 14 is completed, there is a possibility that problems such as smear may occur. However, in the electronic camera of the third embodiment, as shown in FIG. 7, the exposure of the image sensor 14 is completed at the time tc2 at the same time as the mechanical shutter 13 is completely closed. Light does not enter the imaging surface. Therefore, according to the exposure control of the electronic camera of the third embodiment, the accuracy of subject separation can be improved without the occurrence of problems such as smear.

After determining the exposure conditions for the preliminary shooting in this way, the control unit 20 proceeds to S102.
(Operational effect of the third embodiment)
As described above, in the electronic camera of the third embodiment, the exposure condition is determined so as to reduce only the timing difference between the exposure start of the image sensor 14 and the strobe light emission of the light emitting unit 24, and preliminary shooting is performed under the exposure condition. Done. By performing exposure control in this way, it is possible to reduce the exposure amount of the image sensor 14 by ambient light while maintaining the exposure amount of the image sensor 14 by strobe light. Therefore, in the electronic camera according to the third embodiment, even in shooting a “bright scene”, the difference between the luminance ratio of the subject and the luminance ratio of the background between the first preliminary shooting image and the second preliminary shooting image. The amount can be increased, and the accuracy of subject separation is increased.

  Furthermore, in the electronic camera of the third embodiment, the exposure of the image sensor 14 is completed at the same time as the mechanical shutter 13 is completely closed. For this reason, light does not enter the imaging surface after the exposure of the imaging element 14 is completed. Therefore, in the electronic camera of the third embodiment, the accuracy of subject separation is improved without the occurrence of defects such as smear.

  The object separation operation of the electronic camera according to the first to third embodiments has been described above.

When preliminary shooting is performed under the exposure conditions determined by the electronic cameras of the first to third embodiments described above, exposure with ambient light is performed as compared with exposure control in the automatic exposure mode as shown in FIG. Since the amount is reduced, an underexposed pre-photographed image may be taken. However, since the pre-photographed image is an image that is used only for subject separation, unlike the main photographed image for viewing, it is only necessary to perform subject separation by luminance comparison, and the exposure state is not particularly limited. However, when the image is underexposed, the S / N ratio generally decreases, so that luminance comparison cannot be performed accurately in subject separation, and the separation accuracy may decrease. Therefore, for example, it is desirable to improve the S / N ratio of the preliminary photographed image by the following method.
(Method 1) A method of performing preliminary shooting at a timing when the shutter is opened wider than the exposure condition in the automatic exposure mode. The method of improving the S / N ratio of the preliminary captured image is the same as that of the electronic camera of the first embodiment (FIG. 5). An example will be described.

  When determining the exposure conditions for preliminary shooting in S101, as shown in FIG. 8, the start and end of the exposure of the image sensor 14 and the strobe emission of the light emitting unit 24 are fully opened after the mechanical shutter 13 starts to open. Is determined as late as possible within the range of the time until (the shutter opening time in FIG. 3). Specifically, the start timing of exposure of the image sensor 14 and strobe light emission of the light emitting unit 24 is set to ts1 ′ (which was ts1 in the electronic camera of the first embodiment), and the end timing thereof is set to ts2 ′ (first In the electronic camera of the embodiment, it was determined as ts2. In accordance with these determinations, the timing at which the mechanical shutter 13 starts to close is determined as a time tc0 '.

  By determining the timing in this way, the timing at which the mechanical shutter 13 starts to close is delayed from the time tc0 to the time tc0 ′, and preliminary shooting is performed with the mechanical shutter 13 being opened wider than the electronic camera of the first embodiment. Will come to be. Therefore, compared with the electronic camera of the first embodiment, the exposure time of the image sensor 14 does not change ((ts2-ts1) = (ts2'-ts1 ')), but the exposure amount of the image sensor 14 by ambient light is Se1. Since it increases from Se4 to Se4, the S / N ratio of the preliminary photographed image is improved. At this time, the exposure amount of the image sensor 14 by the strobe light also increases from Ss to Ss ′. The increase rate Ss ′ / Ss is the same as the rate Se4 / Se1 in which the exposure amount of the image sensor 14 by the ambient light is increased. Since the ratio does not change, the accuracy of subject separation is not different from that of the electronic camera of the first embodiment. Accordingly, it is possible to improve the S / N ratio of the preliminary photographed image while maintaining the effect of improving the subject separation accuracy by the electronic camera of the first embodiment.

Note that this technique can be applied not only to the electronic camera of the first embodiment but also to the electronic cameras of the second and third embodiments. If this technique is applied, the technique can be applied to those embodiments. Similarly, the effect of improving the S / N ratio of the pre-photographed image can be obtained. This technique may be applied to the first preliminary shooting (strobe flash shooting), and may or may not be applied to the second preliminary shooting (strobe non-flash shooting).
(Method 2) Method of performing noise reduction processing on a pre-photographed image By performing noise reduction processing as described in (a) to (c) below, the S / N ratio of the pre-photographed image can be improved.
(A) Processing for averaging a plurality of preliminary captured images A plurality of preliminary captured images are captured by performing preliminary imaging a plurality of times continuously, and the noise is reduced by averaging the plurality of preliminary captured images. A single image is created. Such an image is created by averaging the luminance components of a plurality of preliminary photographed images or individually averaging each component represented by RGB, Lab, or the like.

Such processing is performed in the first preliminary shooting in S102 and the second preliminary shooting in S103, respectively, and two noise-reduced images created by the execution are used for luminance comparison in S106. Like that.
(B) Processing for reducing the resolution of the pre-photographed image The noise of the pre-photographed image taken in the pre-photographing is reduced by a resolution conversion method using information on adjacent pixels such as bicubic or bilinear. Create a rendered image.

Such processing is performed in the first preliminary shooting in S102 and the second preliminary shooting in S103, respectively, and two noise-reduced images created by the execution are used for luminance comparison in S106. Like that.
(C) Processing for emitting a small amount of strobe light at the time of the second preliminary photographing In the second preliminary photographing at S103, a small amount of strobe light is emitted to capture a preliminary photographed image. Note that the light emission amount is much lower than the light emission amount in the first preliminary photographing in S102, and is at least a fraction of the light amount. The second pre-photographed image with low noise that has been photographed in this way is used for the luminance comparison in S106.

  In addition, the process of (a)-(c) mentioned above may be applied individually, respectively, and may be applied in combination of 2 or more.

  In addition to the processes (a) to (c) described above, various general noise reduction methods can be used for the noise reduction process for the pre-photographed image.

  In the foregoing, two methods for improving the S / N ratio of the preliminary photographed image have been described.

  Note that the method for improving the S / N ratio of the pre-photographed image is not limited to these two methods. For example, the exposure amount control similar to the method 1 may be performed by a combination of the imaging sensitivity and the aperture such as “lower the imaging sensitivity and open the aperture”.

  In the first to third embodiments, the first preliminary image (strobe emission image) and the second preliminary image (strobe non-emission image) are both shown as examples using through images. It is not limited to this. For example, the first and second preliminary captured images may be high-resolution images similar to the main captured image. The through image has a lower resolution than the actual captured image and can be read out from the image sensor in a short time. Therefore, when the through image is used as a preliminary image, noise reduction processing and the subsequent subject and background are separated. In the luminance comparison process for this purpose, it is possible to increase the speed due to the reduction in processing load. On the other hand, when a high-resolution image is used as the pre-photographed image, it is particularly effective in the noise reduction process (b) as compared with the case where a through image is used. Generally, when reducing the resolution of an image, the pixel value of the image after resolution reduction is determined based on a plurality of pixel values in the corresponding region of the image before resolution reduction. At this time, the noise reduction effect can be enhanced as the number of pixels in the image area before resolution reduction corresponding to the pixels of the image after resolution reduction increases. Here, considering the case where the resolution is reduced for both the through image and the high resolution image to create a low resolution image of the same resolution, the high resolution image has a higher resolution than the through image (that is, per unit image size). Therefore, the pixel value of the image after resolution reduction can be created based on a larger number of pixel values than when a through image is used. Therefore, an excellent noise reduction effect can be obtained when a high-resolution image is used.

  In the first to third embodiments, the exposure conditions for the second preliminary shooting (strobe non-flash shooting) are the exposure conditions for the first preliminary shooting (strobe flash shooting) and the flash emission conditions (flash emission). Although an example in which only whether or not) is different is shown, the present invention is not limited to this. For example, the second preliminary shooting may be performed under an exposure condition in which the exposure timing of the image sensor 14 is different from that of the first preliminary shooting. At this time, if the second preliminary shooting is performed under an exposure condition such that the exposure amount of the image sensor 14 by the ambient light is larger than the exposure condition of the first preliminary shooting, such as the automatic exposure mode, For the pre-photographed image, a decrease in the S / N ratio due to underexposure can be avoided.

  In the first to third embodiments, the example in which the subject / background separation information (subject separation information) is created based on the luminance comparison between the first preliminary photographed image and the second preliminary photographed image is shown. However, the present invention is not limited to this. For example, the subject separation information may be created based on a luminance comparison between a first preliminary image that is a flash emission image and a main image that is a non-flash image. In this case, since the second preliminary photographing in S103 can be omitted, the photographing process can be shortened. However, as in the first to third embodiments, when a through image is used as a pre-photographed image, the first pre-photographed image (through image) and the main photo image have different resolutions. After matching the resolutions of both images, the luminance comparison of the corresponding parts of both images is performed. For example, the image size of the actual captured image is reduced in accordance with the resolution of the first preliminary captured image, the luminance comparison is performed using the first preliminary captured image and the actual captured image of the reduced size, and the subject separation information is generated. .

  In the first to third embodiments, the example in which the luminance comparison for separating the subject and the background is performed based on the luminance ratio has been described. However, the present invention is not limited to this. For example, the luminance comparison may be performed based on the luminance difference. However, when the luminance comparison is performed based on the luminance difference, the exposure amount of the image sensor 14 due to the ambient light differs between the two images used for the luminance comparison (for example, the first preliminary shooting as described above). When shooting is performed with different exposure timings of the image sensor 14 in the second preliminary shooting, or when comparing the luminance between the first preliminary shooting image and the main shooting image), the subject and the background are separated. It may not be performed correctly. Therefore, in such a case, the luminance comparison is performed after gain adjustment is performed so that the background signal levels are equal in the two images used for the luminance comparison.

  Further, as described above, the pre-photographed image is not used for viewing unlike the main-photographed image. Therefore, the amount of strobe light irradiation during the pre-photographing need not be limited to an appropriate light amount. Rather, from the viewpoint of increasing the accuracy of subject separation, it is preferable to irradiate the strobe light with as much light as possible. This is because the larger the irradiation amount of strobe light, the larger the difference (Ss0−Ssb) between the exposure amount Ss0 due to the strobe light reflected from the subject and the exposure amount Ssb due to the strobe light reflected from the background. Therefore, the difference between the luminance ratio of the subject and the luminance ratio of the background in the first preliminary photographed image (strobe light emission image) and the second preliminary photographing image (strobe non-light emission image) represented by the above formula (3). This increases the accuracy of subject separation. However, when the strobe light is irradiated with a large amount of light, there is a possibility that saturation (hereinafter, this saturation of the image signal will be referred to as “out-of-white”) occurs in the image signal of the subject located near the camera. . Hereinafter, a method of subject separation when an overexposed region is present in the image will be described with reference to FIG. FIG. 9 shows a case where a subject of a person wearing clothes with a white portion at the chest is photographed by any of the electronic cameras of the first to third embodiments. In the first preliminary photographed image that is a strobe light emission image, the white part of the clothes is white due to high reflectance. Since the whiteout area has a saturation level of the image signal, the strobe light effect above the saturation level is not recorded in the area. Accordingly, in the luminance comparison between the first preliminary photographed image and the second preliminary photographed image, the luminance change amount of the white portion of the clothes is smaller than the actual luminance change amount due to whiteout. As a result, the white part of the clothes is not recognized as a subject, and the accuracy of subject separation is reduced. Therefore, when there is a whiteout area in the first preliminary image that is a flash emission image, the area may be preferentially determined as a subject. This determination makes it possible to correctly recognize the white part of the clothes, which is a whiteout area, as a subject, and to prevent a reduction in subject separation accuracy.

(Other)
It should be noted that either the first preliminary shooting in S102 or the second preliminary shooting in S103 may be performed first.

  Further, whichever comes first may be the shooting order of the preliminary shooting in S101 to S103 and the main shooting in S104 to S105.

  Further, the luminance comparison between the first preliminary captured image and the second preliminary captured image in S106 may be performed in units of pixels, or may be performed in units of areas composed of a plurality of pixels.

  Further, the subject / background separation information (subject separation information) created in S106 is recorded on the recording medium 23 together with the actual photographed image photographed in S105, and the processing from S107 onward is terminated without being executed. May be. In such a case, when the operation mode of the electronic camera is a non-shooting mode such as an image playback mode, the processing from S107 onward is executed using the subject separation information on the recording medium 23 and the actual shot image. Good.

  Further, some or all of the processes after S106 may be executed by an external processing device such as a computer instead of being executed by the electronic camera.

  In step S107, after separating the subject and the background, a process of displaying the separation state on an LCD monitor or the like and allowing the user to confirm may be executed. If the user instructs the separation process to be performed again when confirming the separation state, the threshold for extracting the subject area is changed to another value, and the subject separation process of S106 to S107 is executed again. You may make it do. Further, these series of processes may be repeatedly executed until a result desired by the user is obtained.

  In S108, as an example of image processing, an example in which blur processing is performed only on the background is shown. However, image processing is not limited to this, and other than blur processing, brightness correction processing, contrast enhancement processing, sharpening processing, and the like. Various processes can be applied. Further, the image processing may be performed only on the subject, may be performed only on the background, or may be performed on both the subject and the background. When image processing is performed on both the subject and the background, it is desirable to perform different image processing on the subject and the background. For example, the subject is sharpened, the background is blurred, the subject is weaker than the background, and the subject is subjected to contrast enhancement or sharpening that is stronger than the background. Is desirable.

  In S109, after the image processed image is displayed on an LCD monitor or the like, an instruction to redo the image processing may be received from the user. Then, when the user instructs to redo, the processing of S108 to S109 may be executed again by changing the conditions of the image processing (for example, in the case of blur processing, the strength of blur processing). . Further, these series of processes may be repeatedly executed until a result desired by the user is obtained.

  In the above description, the subject and the background are separated from the actual captured image. However, the subject and the foreground may be separated, or the subject, the background, and the foreground may be separated at the same time.

It is a block diagram of the electronic camera to which the imaging device of the present invention is applied. 3 is a flowchart showing an object separation operation of the electronic camera of the first embodiment. It is a figure explaining exposure control (control of shutter opening and closing operation) by automatic exposure mode. It is a figure explaining exposure control by the automatic exposure mode (control of shutter opening / closing operation, exposure of image sensor, strobe light emission in a bright scene). It is a figure explaining exposure control of the 1st preliminary photography of the electronic camera of a 1st embodiment. It is a figure explaining the 1st preliminary photography exposure control of the electronic camera of a 2nd embodiment. It is a figure explaining the 1st preliminary photography exposure control of the electronic camera of a 3rd embodiment. It is a figure explaining the method of improving the S / N ratio of the 1st preliminary photo image. It is a figure explaining the to-be-photographed object separation method in case a whiteout area | region exists in an image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Imaging lens, 12 ... Lens drive part, 13 ... Mechanical shutter, 14 ... Imaging element, 15 ... Signal processing part, 16 ... Timing generator (TG), 17 ... Buffer memory, 18 ... Data processing part, 19 ... Compression / Decoding section, 20 ... control section, 21 ... photometry section, 22 ... monitor, 23 ... recording medium, 24 ... light emitting section, 25 ... operation section, 26 ... bus

Claims (11)

Shooting with high strobe light intensity and shooting with low strobe light intensity are performed for the same scene, and the boundary between the subject area and the non-subject area in the actual captured image is determined based on the luminance comparison of the images acquired by the shooting. In the imaging device to detect,
When shooting with high strobe light intensity, the exposure of the image sensor is such that the period of the exposure period of the image sensor that overlaps the non-flash period of the strobe light is shorter than the overlap period in automatic exposure mode shooting. An image pickup apparatus comprising: control means for controlling the timing of light emission of the strobe light. The imaging apparatus according to claim 1,
The control device makes the time difference between the start of exposure and the start of light emission smaller than the time difference in automatic exposure mode photography. In the imaging device according to claim 1 or 2,
The image pickup apparatus, wherein the control unit makes a time difference between the end of the exposure and the end of the light emission smaller than the time difference in the automatic exposure mode photographing. In the imaging device according to any one of claims 1 to 3,
The image pickup apparatus, wherein the control unit matches a time from the start to the end of the exposure with a time from the start to the end of the light emission. In the imaging device according to any one of claims 1 to 4,
The image pickup apparatus, wherein the control means matches a timing at which an opening of a shutter for controlling an exposure period of the image pickup device is maximized with the light emission timing. In the imaging device according to any one of claims 1 to 5,
An imaging apparatus comprising: noise reduction means for performing noise reduction processing on an image other than the main photographed image used for the luminance comparison. In the imaging device according to any one of claims 1 to 6,
An image pickup apparatus, further comprising: a normalizing unit that normalizes an image used for the luminance comparison so that a signal level of a non-subject region between the images used for the luminance comparison is uniform. In the imaging device according to any one of claims 1 to 7,
An image pickup apparatus that emits the strobe light with a light emission amount larger than a light emission amount in automatic exposure mode photography in photographing with high strobe light intensity. The imaging device according to claim 8,
An image pickup apparatus, further comprising: a determination unit that preferentially regards a whiteout region in an image acquired by photographing with high strobe light intensity as the subject region regardless of the luminance comparison. In the imaging device according to any one of claims 1 to 9,
An imaging apparatus, further comprising: an image processing unit that performs different image processing on the subject area and the non-subject area in the main captured image. The imaging device according to claim 10.
The imaging apparatus, wherein the image processing means performs blur processing on the non-subject area.