JP2012095183A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow recording of animation of a subject under a low light intensity environment while utilizing the atmosphere in the field, with more precise focusing being allowed.SOLUTION: An image pickup device 130 repeats imaging operation for imaging a subject formed with an imaging lens 140, and imaging data is outputted in time series from an image pickup device interface circuit 132 corresponding to each of the imaging operations. The image data is sorted into first time series image data and second time series image data, in units of frame or field, by an image sorting process part 110. A light emission timing control part 120 controls an auxiliary light emission device 150 so that the imaging operation for outputting the image data corresponding to the first time series image data is synchronized with the timing of image pickup device 130, for light emission. An animation data generation process part 112 generates animation data based on the second time series image data.

Description

  The present invention relates to a photographing apparatus, and more particularly to a photographing apparatus capable of photographing a moving image while illuminating a subject using an auxiliary light source.

  An electronic camera capable of irradiating AF (automatic focus adjustment) auxiliary light is known for the purpose of increasing the accuracy of focus adjustment in a dark place and the speed of a focus adjustment operation. In addition, an apparatus called a movie light or a video light is known as a device that emits illumination light toward a subject during moving image shooting.

  Patent Document 1 discloses a moving image imaging apparatus that can adaptively adjust the amount of light emitted from a lighting device in accordance with a change in brightness of a subject in a low illumination environment. In this moving image pickup apparatus, the illumination light is pulse-emitted, and the light emission timing at that time is controlled to synchronize with the image pickup operation repeatedly performed by the image pickup device.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique that enables reduction in power consumption required for emitting AF auxiliary light when the subject is irradiated with AF auxiliary light. In order to reduce the power consumption required for the emission of the AF auxiliary light, the AF auxiliary light is emitted intermittently. More specifically, the lens movement for focus adjustment and the evaluation value corresponding to the contrast component of the subject are obtained alternately based on the image signal output from the imaging means, and at that time, the photographing lens AF auxiliary light is emitted each time a predetermined amount is moved.

  Patent Document 3 discloses a technique that facilitates confirmation of a subject composition before photographing in a dark background while suppressing power consumption in a photographing illumination device using an LED (light emitting diode) as a light source. Patent Document 3 discloses a digital camera for taking a still image, and pre-emission is emitted in synchronization with the imaging operation of the imaging device during the monitoring period before the exposure operation. At this time, in the first and second exposure periods TB1 and TB2 in which the CCD is exposed, pre-light emission L1 to L3, which are pulse lights, are emitted from the illumination device. Pre-light emission L1 to L3 is irradiated in synchronization with the first and second exposure periods TB1 and TB2.

JP 2005-354155 A Japanese Patent Laid-Open No. 2003-140027 JP 2005-128145 A

  When shooting moving images under low illumination conditions, the photographer does not necessarily want to use auxiliary light. For example, in a situation where a person is photographed in a room illuminated by candlelight, etc., when attempting to shoot a movie that takes advantage of the atmosphere of the place, the photographer must shoot without irradiating auxiliary light. May want.

  At the time of moving image shooting, in a situation where the relative distance between the subject and the photographer can change, it is necessary to continue the focus adjustment operation. Whether the AF method is so-called contrast AF or phase difference detection AF, the accumulation operation is performed by the image sensor or the AF sensor. At this time, if the field luminance is low, the accumulation operation may take time. Even if the accumulation operation takes time, if the subject brightness is low, the influence of noise may be relatively increased, leading to a decrease in AF speed and accuracy. In other words, by not irradiating the auxiliary light, an image having an atmosphere can be obtained, but it may be difficult to realize more accurate and faster focus adjustment.

  Also, when the photographer manually adjusts the focus (MF operation) while watching a live view image displayed on a liquid crystal display panel or EVF (electronic viewfinder) provided on the back of the photographing apparatus, the luminance of the field Is low, it is difficult to focus accurately. This is because it is difficult to grasp the focus adjustment position where the image is sharpest even if the image is dark and relatively noisy. Further, when the live view image is dark, it may be difficult to perform accurate framing.

  In any of Patent Documents 1 to 3, it is difficult to maintain the accuracy and speed of focus adjustment or to perform accurate framing in a situation where the photographer does not want to shoot using auxiliary light. There is no disclosure or suggestion, and naturally there is no disclosure or suggestion for solving the problem.

  The present invention has been made in view of the above-described problems, and enables more precise focus adjustment while making it possible to take a moving image by taking advantage of the atmosphere of a subject in a low-light environment. It aims to be.

(1) According to an aspect of the present invention, an imaging device includes:
An imaging unit capable of repeatedly performing an imaging operation of imaging a subject image formed by the imaging lens, and outputting image data in time series corresponding to each of the imaging operations;
A distribution processing unit that distributes the image data output in time series from the imaging unit to the first time-series image data and the second time-series image data in units of frames or fields;
An auxiliary light emitting device that emits illumination light toward a subject performs an imaging operation for outputting image data corresponding to the first time-series image data among the first and second time-series image data. A light emission timing control unit that controls to emit light in synchronization with the timing performed by the imaging unit;
A moving image data generation unit configured to generate moving image data based on the second time-series image data.

  According to the present invention, it is possible to achieve both obtaining a moving image that makes use of the atmosphere of the place where the subject is not irradiated with auxiliary light and performing highly accurate focus adjustment.

It is a block diagram which illustrates roughly the internal structure of an imaging device and an auxiliary light-emitting device. It is a block diagram explaining the principal part of an imaging device. It is a time chart explaining an imaging operation, an auxiliary light emission operation performed in synchronization therewith, and an image distribution operation. It is a figure which shows notionally a mode that the imaging | photography range is illuminated by the some light emission part with which an auxiliary light emission device is equipped, (a) is light emission of all the light emission parts, and the whole region of the imaging | photography range is illuminated. (B) shows a state in which only a region where the main subject recognized by the subject recognition processing unit exists is illuminated in the photographing range. It is a flowchart explaining the process sequence of imaging | photography operation control and auxiliary light emission control which are performed by a system controller when imaging | photography operation | movement is performed with an imaging device. It is a flowchart explaining the process sequence of the imaging | photography operation control and auxiliary light emission control which are performed by a system controller when imaging | photography operation | movement is performed with an imaging device, and is a flowchart explaining the process sequence following the flowchart of FIG. 5A. It is a flowchart explaining the process sequence of the imaging | photography operation control and auxiliary light emission control which are performed by a system controller when imaging | photography operation | movement is performed with an imaging device, and is a flowchart explaining the process sequence following the flowchart of FIG. 5A.

  FIG. 1 is a block diagram illustrating a schematic configuration of a photographing apparatus according to an embodiment of the present invention. The imaging apparatus includes a system controller 100, an image sensor 130, an image sensor interface circuit (denoted as “image sensor I / F circuit” in FIG. 1) 132, a shutter drive unit 134, and a display drive unit 170. , An image display unit 172, a display light emitting element 174, a flash memory 176, an SDRAM (Synchronous Dynamic Random Access Memory) 180, and an operation unit 190.

  The photographing apparatus also includes a photographing lens 140 and an auxiliary light emitting device 150. The photographing lens 140 may be incorporated in the photographing device or may be detachably attached to the photographing device as an interchangeable lens. The auxiliary light emitting device 150 may also be incorporated in the photographing apparatus or may be detachably attached to the photographing apparatus as an accessory.

  When the auxiliary light emitting device 150 is configured to be externally attached, the photographing device is provided with an accessory shoe or other fastening device, and the auxiliary light emitting device 150 is photographed via the accessory shoe or the fastening device. It can be attached to the device. Alternatively, when the control signal can be exchanged between the imaging device and the auxiliary light emitting device 150 in a wired or wireless manner, the auxiliary light emitting device 150 does not necessarily have to be directly attached to the imaging device.

  The system controller 100 includes a CPU 102, an image processing unit 104, a compression / decompression unit 106, a subject recognition processing unit 108, an image distribution processing unit 110, a moving image data generation processing unit 112, and a focus detection processing unit 114. , A focus adjustment control unit 116, a luminance detection processing unit 118, a light emission timing control unit 120, and an irradiation range control unit 122.

  In this embodiment, the system controller 100 includes a CPU 102 and a DSP (digital signal processor), and includes an image processing unit 104, a compression / decompression unit 106, a subject recognition processing unit 108, an image distribution processing unit 110, The moving image data generation processing unit 112 is realized as a function provided in the DSP. Also, the program code as firmware stored in the flash memory 176 is read into the SDRAM 180, and sequentially interpreted and executed by the CPU 102, whereby the focus detection processing unit 114, the focus adjustment control unit 116, the luminance detection processing unit 118, It is assumed that the light emission timing control unit 120 and the irradiation range control unit 122 are realized. Of course, these processing blocks may be configured by hardware logic or the like, or the entire system controller 100 may be one ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array). It may be constituted by.

  The operation unit 190 includes a power switch 190a, a mode setting switch 190b, a release switch 190c, a video shooting switch 190d, an up switch 190e, a down switch 190f, and an auxiliary light switch 190g (in FIG. The switch is labeled “Sw”).

  The photographing lens 140 includes an aperture driving unit 142, a focus adjustment driving unit 144, and a focusing lens 146. The auxiliary light emitting device 150 includes an auxiliary light control unit 152, an irradiation angle control unit 160, a light distribution lens 162, and an LED array 164.

  A memory card MC for recording image data generated by the photographing apparatus is detachably attached to the photographing apparatus.

  In the present specification, for the purpose of facilitating understanding of the moving direction of the components, the direction of the arrangement, and the like, the description will be made with the three-dimensional coordinate axes as follows. That is, it is assumed that the optical axis of the photographing lens 140 is parallel to the paper surface of FIG. 1 and extends in the left-right direction, and the X-axis is taken in a direction parallel to the optical axis. Further, the Y axis is taken in a direction (vertical direction in FIG. 1) parallel to the paper surface of FIG. 1 and perpendicular to the X axis, and the Z axis is taken in a direction perpendicular to the paper surface of FIG. At this time, the direction of the Z axis is determined so that the coordinate value increases in the direction from the front side to the back side in FIG. The X axis is oriented in the right direction along the paper surface of FIG. 1, and the Y axis is oriented in the upward direction along the paper surface of FIG.

  The image sensor 130 is a two-dimensional image sensor such as a CCD (charge coupled device) image sensor or a CMOS (complementary metal oxide semiconductor) image sensor. Even if the imaging device 130 is a so-called multi-plate type configured by including a spectral optical system such as a dichroic prism and a plurality of monochrome imaging devices, B (blue), G (green), R (red) A so-called single plate type filter in which an on-chip color filter such as the above is formed on the light receiving surface may be used. In this embodiment, the image sensor 130 is a single-plate CMOS image sensor in which B, G, and R on-chip color filters are formed on the light receiving surface, and includes a CDS (correlated double sampling) circuit, an amplifier, In the following description, it is assumed that an A / D converter or the like is provided so that a digital image signal can be output.

  By the way, the CMOS image sensor includes a so-called rolling shutter type and a global shutter type. A rolling shutter type CMOS image sensor selects each pixel by the XY address method, and performs a series of exposure operations such as clearing the charge, starting exposure, stopping exposure, and reading a signal for each pixel (line-sequentially). To). Therefore, there is no simultaneous exposure (accumulation). That is, exposure is not performed simultaneously within one screen. For that reason, it is impossible to acquire time-series image data obtained by an imaging operation involving irradiation of auxiliary light and time-series image data obtained by an imaging operation not accompanied by irradiation of auxiliary light, which will be described below. Therefore, the CMOS image sensor used in this embodiment has an exposure function in the global shutter system similar to that of the CCD image sensor.

  The image sensor interface circuit 132 performs control of an imaging operation by the image sensor 130 (imaging timing control, photoelectric conversion time control, image signal readout control such as all pixel readout and thinning readout, etc.).

  The shutter drive unit 134 includes a shutter mechanism such as a focal plane shutter or a lens shutter, and an actuator or driver for driving the shutter mechanism. The shutter driving unit 134 may include either a normally open type or a normally closed type shutter. In the present embodiment, the shutter driving unit is assumed to include a normally open type focal plane shutter. Based on a control signal output from the system controller 100, the shutter drive unit 134 performs still image shooting by opening and closing the shutter so that exposure for a predetermined length of time is given on the light receiving surface of the image sensor 130. Is possible. The shutter driving unit 134 is also configured to perform live view image display (monitor image display, through image display) and moving image shooting while keeping the shutter open. Note that the shutter drive unit 134 may be omitted depending on the specifications of the photographing apparatus.

  The CPU 102 accepts an operation by a photographer, and comprehensively controls operations of the photographing apparatus such as operation mode switching, photographing operation, and image data recording operation. The CPU 102 further functions as the focus detection processing unit 114, the focus adjustment control unit 116, the luminance detection processing unit 118, the light emission timing control unit 120, and the irradiation range control unit 122 as described above.

  The image processing unit 104 performs processing such as gamma correction, color conversion, demosaicing, noise reduction, and shading correction on the digital image signal input from the image sensor 130 via the image sensor interface circuit 132, and performs B, G, and R three processing. Generate image data consisting of two color planes. The image processing unit 104 also displays a live view image on the image display unit 172 (to be described later) when the photographing apparatus is in a photographing preparation state (a state in which the photographer points the subject at a subject and performs composition determination, focus adjustment, etc.). Processing for display is also performed as necessary.

  The compression / decompression unit 106 performs compression processing on the image data generated by the image processing unit 104 using an image compression technique such as JPEG, MPEG, or H264 as necessary. The compression / decompression unit 106 also performs decompression processing on the compressed image data included in the image file read from the memory card MC or the like, and generates image data for displaying a still image or a moving image on the image display unit 172. To process.

  The subject recognition processing unit 108 analyzes the image data generated by the image processing unit 104 and performs processing for detecting the position and size of the subject in the image. The processing performed by the subject recognition processing unit 108 can be as follows, for example.

  As a first example of processing performed by the subject recognition processing unit 108, there is processing for deriving an area in the image in which the main subject is seen. This process is a process of analyzing an image and deriving a region in which an object that is regarded as a main subject is shown by using a spatial frequency, saturation, or high brightness in the image as a criterion.

  The fact that the spatial frequency of a certain area is higher than the spatial frequency of other areas means that it is highly likely that the area is in focus or that a prominent subject is captured. Is possible. Therefore, it is possible to derive where the main subject appears in the screen by dividing the screen into several regions and deriving and comparing the spatial frequency components of the respective regions.

  If the saturation of a certain area is higher than the saturation of another area, it can be considered that there is a high possibility that a conspicuous subject such as a flower appears in the certain area. Therefore, by converting the image data into image data defined in a color space such as Lab or HSL, and deriving areas where the a, b and S values are relatively high, the main subject appears anywhere in the screen. It is possible to derive whether or not.

  For example, if there is an area with relatively high brightness at the top of the screen, the area has a bright background such as the sky, and it is highly likely that the main subject is in the area below that area. It is possible to think. Alternatively, if there is a region with high brightness on a relatively dark screen, it can be considered that the main subject is illuminated by artificial light outdoors at night or in a dark room. As described above, it is possible to derive the position where the main subject is captured by analyzing the position of the high brightness area in the screen or the brightness distribution.

  Regarding the method based on the spatial frequency, saturation, and brightness described above as a method for deriving the position of the main subject in the screen, even if any one of these methods is used, a plurality of methods are combined. May be used.

  As a second example of the process performed by the subject recognition processing unit 108, there is a so-called face recognition process for deriving a region where a face such as a person or an animal is captured in an image. This face recognition process uses a plurality of templates created in advance based on image features of the face, such as eyebrows, eyes, nose, mouth, etc. A process for deriving a high area can be performed. An area where an image having a high degree of matching with the template can be taken as an area where a face appears in the screen.

  As a third example of processing performed by the subject recognition processing unit 108, there is processing for deriving a position where a main subject is captured by a motion vector. In this process, images output from the image sensor 130 in time series and generated by the image processing unit 104 are sequentially processed to derive a motion vector. Based on the magnitude and pattern of the motion vector, it is possible to derive an area in which the subject of interest is shown in the screen.

  For example, if the photographer keeps panning the photographing device so that the moving child as the main subject is always located in the screen, the surrounding portion (in comparison with the motion vector of the portion where the child is shown in the obtained image) The motion vector of the peripheral part becomes large. For example, if the camera has a built-in sensor for detecting camera shake, whether the camera is almost stationary by detecting the output from the sensor or is panning as described above Can be discriminated. If there is a region with a relatively low motion vector in the screen under the situation where the photographing apparatus is panned, the subject in that region can be regarded as the main subject.

  Alternatively, if the photographing apparatus is in a substantially stationary state and there is a region having a relatively high motion vector in the screen, it is possible to conversely consider that the subject in that region is the main subject. In addition to the magnitude of the motion vector, it is also possible to estimate the position where the main subject appears from the difference in the pattern of change over time of the motion vector.

  The subject recognition processing unit 108 may perform subject recognition processing other than those described above. For example, after a photographer identifies a subject in a live view image displayed on the image display unit 172 by operating a touch panel or the like, template matching processing is performed using the subject image as a template, and a portion with a high degree of matching is selected as a main subject. It is also possible to set the position where It is also possible to derive the distance between the subject appearing in the image and the photographing device, and to set the position where the subject closest to the screen appears as the position where the main subject appears. In the subject recognition processing unit 108, any of the processing examples described above may be performed, or a plurality of processing may be performed in combination.

  The image distribution processing unit 110 outputs a series of image data (hereinafter referred to as an “image stream”) including a plurality of frames that are output in time series from the image sensor 130 and processed by the image processing unit 104. Is referred to as two time-series image data. In the present specification, these two time-series image data are referred to as first time-series image data and second time-series image data. The image data distribution process in the image distribution processing unit 110 can be the simplest process in which the image stream is distributed alternately for each frame.

  The image stream is the first image signal output from the image sensor 130, the second output image signal, the third output image signal,... Image data generated based on the output image signal is included. The image sensor 130 may be capable of interlaced scanning or progressive scanning, but in the present embodiment, imaging is performed by the progressive scanning method. In addition, in this specification, for convenience of explanation, image data generated from an image signal output oddly from the image sensor 130 after the start of the imaging operation is referred to as an odd frame (odd frame). Similarly, image data generated from the even-numbered image signal output from the image sensor 130 is referred to as an even frame. That is, the image distribution processing unit 110 performs a process of distributing a series of image streams into odd-numbered frame image data and even-numbered frame image data. In this specification, the odd frame image data distributed by the image distribution processing unit 110 is referred to as first time-series image data, and the even frame image data is referred to as second time-series image data.

  In the above description, the image distribution processing unit 110 has been described as performing the process of alternately distributing the image streams. For example, one frame of image data is distributed to the first time-series image data, and the one frame is followed. It is also possible to distribute the unequal distribution by repeatedly distributing the output n frames of image data to the second time-series image data. Here, n is an integer of 2 or more.

  When the image sensor 130 captures an image using the interlace scan method, the image distribution processing unit 110 outputs each image data generated based on the odd field and even field image signals alternately output from the image sensor 130. It is possible to sort. In this case, for example, the image data based on the image signal of the odd field can be allocated to the first time series image data, and the image data based on the image signal of the even field can be allocated to the second time series image data. Of course, the image data based on the image signal in the even field may be allocated to the first time series image data, and the image data based on the image signal in the odd field may be allocated to the second time series image data.

  By the way, the image distribution processing unit 110 may always perform the above-described image distribution processing, or may perform the image distribution processing only when necessary without performing the image distribution processing in a certain shooting situation. It may be. When image distribution processing is not performed by the image distribution processing unit 110, the moving image data generation processing unit 112 is based on an image stream (a series of image data that is not subjected to image distribution processing) generated by the image processing unit 104. It is also possible to generate moving image data.

  In addition, when the image sensor 130 is configured to be able to perform an imaging operation by switching to either of the progressive scan method and the interlace scan method, an image distribution process is performed by the image processing unit 110, or Corresponding to the case where it is not performed, the imaging operation method may be switched. That is, when image distribution processing is not performed by the image processing unit 110, the image sensor 130 performs an imaging operation by progressive scan, and when the image processing unit 110 performs distribution processing, the image sensor 130 captures images by interlace scanning. You may be comprised so that operation | movement may be performed. Then, each image generated by the interlace scan and corresponding to the odd field and the even field may be distributed to the first time series image data and the second time series image data. In this case, when the image sensor 130 operates in the interlace scan method, the vertical resolution is lowered, but it is also possible to perform an interpolation process so that the vertical resolution is maintained.

  The focus detection processing unit 114 analyzes the image data corresponding to a predetermined area (focusing area) in the screen in the image data generated and sequentially output by the image processing unit 104 (focus detection). Repeat the process. For example, by observing the change in contrast value while gradually moving the imaging position of the photographic lens 140, it is determined whether or not the imaging position of the photographic lens 140 has moved in a desirable direction (the image becomes sharper). I can know.

  The focus adjustment control unit 116 derives the moving direction and the moving amount of the imaging position of the photographing lens 140 based on the focus detection processing result by the focus detection processing unit 114, and outputs a control signal to the focus adjustment driving unit 144 described later. .

  The brightness detection processing unit 118 analyzes the image data generated and sequentially output by the image processing unit 104 to derive the brightness of the subject (or the field brightness).

  Here, the use of the first and second time-series image data will be described with reference to FIG. FIG. 2 is a block diagram showing a main part of the photographing apparatus according to the embodiment of the present invention. FIG. 2 shows a state in which a series of image streams output from the image processing unit 104 is distributed by the image distribution processing unit 110 into first time-series image data and second time-series image data. In FIG. 2, the flow of an image signal or image data is indicated by a broken line.

  When the image distribution processing unit 110 performs a process of distributing a series of image streams into the first time-series image data and the second time-series image data, the moving image data generation processing unit 112 converts the second time-series image data into the second time-series image data. Based on this, moving image data is generated. Similarly, when the image distribution processing unit performs the image data distribution processing as described above, the subject recognition processing unit 108 and the focus detection processing unit 114 perform the respective processes using the first time-series image data. At this time, a live view image can be displayed on the image display unit 172 as necessary. When displaying a live view image, the display driving unit 170 receives live view image display data based on any one of the first and second time-series image data from the image distribution processing unit 110. The live view image is displayed on the image display unit 172. Which of the first and second time-series image data is used for live view image display depends on the shooting situation. This will be described in detail later.

  The auxiliary light emitting device 150 will be described with reference to FIG. 1 again. The auxiliary light emitting device 150 is a device that emits auxiliary light toward the subject when the luminance of the subject is low. As described above, the auxiliary light emitting device 150 may be built in the photographing apparatus or may be detachably attached to the photographing apparatus as an accessory. That is, you may comprise externally. When the auxiliary light emitting device 150 is built in the photographing apparatus, power is supplied from the power supply of the photographing apparatus. When the auxiliary light emitting device 150 is configured to be externally attached, the auxiliary light emitting device 150 may have a built-in power source such as a battery, or may be supplied with power from the photographing device. .

  In the embodiment of the present invention, when shooting a moving image using a photographing device, the auxiliary light emitting device 150 is configured to be operable in one of two operation modes according to the photographing purpose. One is a mode in which an image of a subject illuminated by the auxiliary light emitted from the auxiliary light emitting device 150 is captured. This operation mode is referred to as A mode in this specification. In the A mode, auxiliary light may be continuously emitted from the auxiliary light emitting device 150 during moving image shooting, or intermittently emitted so as to be synchronized with the imaging operation (accumulation operation) of the image sensor 130. May be. Another operation mode is an operation mode in which light emission is performed in synchronization with the timing of an imaging operation (storage operation) for generating image data that is not recorded as moving image data during moving image shooting. This operation mode is referred to as B mode in this specification.

  When the auxiliary light emitting device 150 operates in the A mode to perform moving image shooting, the subject illuminated by the auxiliary light can be shot. On the other hand, when the auxiliary light emitting device 150 operates in the B mode and the moving image is shot, the subject in the recorded moving image data is not irradiated with the auxiliary light. It is possible to record an image that takes advantage of the atmosphere of the place in a lit situation.

  FIG. 3 conceptually shows a sequence when the auxiliary light emitting device 150 operates in the B mode and the image distribution processing unit 110 performs a process of distributing a series of image streams into first and second time-series image data. It is shown. During the imaging operation sequentially performed in time series by the imaging device 130, the auxiliary light emitting device 150 is turned on in synchronization with the exposure operation for generating the image data of the odd-numbered frames, and the auxiliary light is emitted. Then, the odd-frame image data is distributed to the first time-series image data, and the even-frame image data is distributed to the second time-series image data. The first time-series image data is input to the subject recognition processing unit 108 and the focus detection processing unit 114, and the second time-series image data is input to the moving image data generation processing unit 112.

  Accordingly, it is possible to perform subject recognition processing and focus detection processing with a relatively bright image illuminated by the auxiliary light emitted from the auxiliary light emitting device 150, and subject recognition processing and focus detection processing even in a relatively dark situation. It is possible to improve the accuracy of the.

  As shown in FIG. 1, the auxiliary light emitting device 150 operable as described above includes an auxiliary light control unit 152, an irradiation angle control unit 160, a light distribution lens 162, and an LED array 164 as an illumination light source. Is provided. The LED array 164 is configured by arranging a plurality of LEDs (light emitting diodes) L11... L15, L21... L25, L31. Hereinafter, when referring to these LEDs, they are described as “LEDs (L11 to L35)”. In the present embodiment, 15 LEDs are arranged.

  In FIG. 1, for convenience of illustration, the LEDs (L11 to L35) are drawn in an array along the XY plane, but in reality, the Y-axis direction is along a plane parallel to the YZ plane. They are arranged in a grid of 3 rows in the direction and 5 columns in the X-axis direction. Of course, these arrangement methods and the number of arrangements can be variously changed according to the application and specifications. The LEDs (L11 to L35) constituting the LED array 164 are not necessarily arranged on a plane, but are arranged along a curved surface such as a cylindrical surface, a spherical surface, an aspheric surface, or a step-like discontinuous surface. May be.

  Each LED (L11 to L35) is configured to be able to control the light emission amount during lighting, extinguishing, and lighting independently of each other. The light distribution lens 162 includes a Fresnel lens having a diffusing or condensing function for adjusting the irradiation angle of light emitted from the LED array 164 toward the subject, and its main surface is parallel to the YZ plane. It arrange | positions so that it may become. Note that the arrangement direction of the LED array 164 and the direction of the light distribution lens 162 are merely examples. By arranging optical elements for bending an optical path such as a mirror and a prism, the arrangement can be made in various arrangement directions and directions. Is possible.

  The irradiation angle control unit 160 changes the irradiation angle of light emitted from the LED arrays (L11 to L35) to the angle of view of the photographing lens 140 by changing the distance between the light distribution lens 162 and the LED array 164. A mechanism for changing the response is provided. For example, the distance between the light distribution lens 162 and the LED array 164 can be changed by a screw mechanism, a helicoid mechanism, a rack and pinion mechanism, or the like.

  The auxiliary light control unit 152 includes an auxiliary light controller 154, a current control unit 156, and a lighting control unit 158. The current control unit 156 includes current control circuits I11 to I15, I21 to I25, I31 to I35 that control currents supplied to the LEDs (L11 to L35). Hereinafter, when referring to these current control circuits, they are referred to as “current control circuits (I11 to I35)”. The lighting control unit 158 includes semiconductor switches S11 to S15, S21 to S25, S31 to S35 for controlling supply of current to each of the LEDs (L11 to L35). Hereinafter, when referring to these semiconductor switches, they are referred to as “semiconductor switches (S11 to S35)”.

  Each of the current control circuits (I11 to I35) described above is configured to be independently controllable by the auxiliary light controller 154. Each of the semiconductor switches (S11 to S35) is also configured to be independently controllable by the auxiliary light controller 154. The auxiliary light controller 154 controls the current control circuits (I11 to I35), the semiconductor switches (S11 to S35), and the irradiation angle control unit 160 based on a control signal output from the system controller 100. As a result, arbitrary LEDs in the LED array (L11 to L35) are selectively emitted to change the illuminance (light quantity), irradiation range, and irradiation angle, and assist light is applied only to some subjects in the photographing range. It is possible to irradiate.

  In addition, as a method for controlling the illuminance (light quantity) of the auxiliary light emitted from the LED arrays (L11 to L35), instead of the current control by the current control unit 156, the LED array while energizing the pulse at a relatively high frequency. A method of supplying power to any LED in (L11 to L35) and changing the duty ratio at that time may be used. Here, the relatively high frequency means that one cycle of pulse energization is sufficiently short with respect to the time length when the accumulation operation for one frame is performed in the image sensor 130 (for example, the accumulation operation for one frame). The frequency at which a plurality of pulse energizations are performed). In addition, when any LED in the LED array (L11 to L35) emits light in synchronization with the accumulation operation in the image sensor 130, light is emitted at a constant illuminance without controlling the illuminance, and the lighting time at that time is set. By changing it, the integrated light quantity may be equivalently changed. Furthermore, you may use together adjustment of illumination intensity, and control of the said lighting time.

  The example in which the auxiliary light emitting device 150 includes the LED array 164 as a light source has been described above, but other light sources may be provided as long as light can be emitted intermittently. For example, an organic EL (electroluminescence) light-emitting element in which a plurality of independent light-emitting portions are arranged in a matrix can be used.

  Further, as a configuration that enables the irradiation angle to be changed, the light distribution lens 162 may be fixed, and the LED array 164 may be configured to be movable with respect to the light distribution lens 162. Alternatively, the light distribution lens 162 and the LED array 164 may be fixed, and a reflector provided around the LED array 164 may be configured to be movable. Furthermore, the light distribution lens 162 and the reflector may be fixed, and the LED array 164 may be configured to be movable.

  The light emission timing control unit 120 and the irradiation range control unit 122 will be described. The light emission timing control unit 120 is irradiated with pulsed auxiliary light from the auxiliary light emitting device 150 in synchronization with the exposure operation (imaging operation and accumulation operation) performed by the image sensor 130 (the auxiliary light is intermittently emitted). The control signal is output to the auxiliary light controller 154.

  The irradiation range control unit 122 outputs control signals related to the auxiliary light irradiation range and the auxiliary light irradiation angle within the shooting range based on the angle of view of the shooting lens 140 and the position of the main subject derived by the subject recognition processing unit 108. To do. At this time, the irradiation range control unit 122 may output a control signal related to the light emission illuminance of the auxiliary light emitting device 150. In this case, the auxiliary light control unit 152 controls the current value when supplying current to the LEDs selected in the LED arrays (L11 to L35) based on the received control signal.

  FIG. 4 is a diagram conceptually illustrating a state in which a subject within the photographing range is illuminated by the LED array (L11 to L35), and shows a state in which the subject side is viewed from the photographing device side. Therefore, the YZ plane exists on a plane parallel to the paper surface of FIG. 4, and the X axis extends in a direction perpendicular to the paper surface. At this time, the X-axis has a direction in which the coordinate value increases from the back side to the front side. Further, the Y-axis is on the upper side along the plane of the drawing, and the Z-axis is on the right side of the plane of FIG.

  FIG. 4A shows the respective irradiation ranges of the LED arrays (L11 to L35) within the imaging range. In FIG. 4B, only four LEDs (L22, L23, L32, L33) in the LED array (L11 to L35) emit light corresponding to the position of the main subject derived by the subject recognition processing unit 108. In this manner, the main subject that is controlled and recognized is irradiated. At this time, the auxiliary light emitting device 150 is operating in the B mode. That is, it operates in an operation mode that emits light in synchronization with the timing of an imaging operation (storage operation) for generating image data (first time-series image data) that is not recorded as moving image data during moving image shooting. .

  When the auxiliary light emitting device 150 is operating in the B mode, it is possible to suppress power consumption accompanying auxiliary light emission by irradiating only the necessary range as described above. As described with reference to FIG. 3, in the embodiment of the present invention, the first time-series image data is also used for subject recognition processing. When the movement of the subject is intense or the photographing apparatus is panned suddenly, the main subject may be out of the auxiliary light irradiation range. In such a case, the irradiation range control unit 122 controls the auxiliary light control unit 152 to gradually increase the number of LEDs emitting light in the LED arrays (L11 to L35) or to temporarily emit all LEDs. It is desirable to emit a signal. Alternatively, when the auxiliary light emitting device 150 is operating in the B mode, all the LEDs in the LED array (L11 to L35) are turned on, and the amount of light emitted from the selected LED at this time is Thus, it is possible to perform control such as reducing the amount of light emitted from the unselected LED.

  Referring to FIG. 1 again, the image display unit 172 includes a TFT (thin film transistor) color liquid crystal display element and a backlight, and is configured to display a color image. Alternatively, a self-luminous organic EL display device may be provided. The image display unit 172 is configured to be able to display a live view image in addition to playback and display of still images and moving images. The image display unit 172 may have a form as a so-called monitor display apparatus provided on the back surface of the photographing apparatus or the like, or may have a form of EVF (electronic viewfinder). Alternatively, both may be provided.

  The display driving unit 170 displays an image based on the display image data generated by the image processing unit 104 and temporarily stored in a VRAM (video random access memory) area in the SDRAM 180 on the image display unit 172. .

  The display light emitting element 174 is an indicator configured to include a light emitting unit such as one or a plurality of LEDs. The display light emitting element 174 can display power on / off of the photographing apparatus, a moving image photographing mode, a still image photographing mode, an operation mode such as a reproduction mode, an access state to the memory card MC, and the like.

  The flash memory 176 is a rewritable nonvolatile memory and stores firmware (program code) executed by the CPU 102. The flash memory 176 also has control parameters that need to be maintained even when no power is supplied to the photographing device, adjustment parameters recorded for each photographing in the photographing device manufacturing process, and selection by the photographer. The set operation settings and operation status information are stored. Further, the image data may be stored in the flash memory 176 as necessary.

  The SDRAM 180 is a memory having a relatively high access speed, and is used as a work area when the CPU 102 executes the program code stored in the flash memory 176. The SDRAM 180 is also configured to be accessible from the image processing unit 104, the subject recognition processing unit 108, and the like. The work area and the image display unit 172 when the image processing unit 104, the subject recognition processing unit 108, etc. perform image processing are also provided. It is also used as a VRAM area when displaying an image.

  The operation unit 190 detects a photographer's operation on the power switch 190a, the mode setting switch 190b, the release switch 190c, the moving image shooting switch 190d, the up switch 190e, the down switch 190f, and the auxiliary light switch 190g, and the detection result is displayed on the system controller 100. Output to. These switches 190 a to 190 g include a touch panel, a push switch, a slide switch, a dial switch, and a rotation direction and a rotation of a ring-shaped operation member (operation ring) provided on the lens barrel of the photographing lens 140 by imitating a helicoid ring. Any one of the switches capable of detecting the amount may be used. It is also possible to use a sensor that can detect the direction, size, pattern, and the like of swing or vibration applied to the photographing apparatus as a switch. Further, the operation unit 190 may receive a command from a remote controller operated by the photographer.

  The power switch 190a is a switch for turning on / off the power of the photographing apparatus. The mode setting switch 190b is a switch for setting the operation mode of the photographing apparatus. The release switch 190c is a switch operated when shooting a still image. The moving image shooting switch 190d is a switch operated when starting / stopping moving image shooting.

  The up switch 190e and the down switch 190f may be switches that are operated when changing settings such as the shutter speed and the aperture value. However, in the embodiment of the present invention, the focus adjustment is performed manually. It is assumed that the switch is operated by In other words, when the photographing apparatus is set to a mode in which the photographer can operate and adjust the focus, the photographer operates the up switch 190e and the down switch 190f to connect the photographing lens 140. It is assumed that the image position can be adjusted.

  The above configuration is referred to as power focus or the like. With this configuration, the imaging position is in the front or rear direction with respect to the light receiving surface of the image sensor 130 (the direction in which the imaging position approaches the subject) as the photographer operates the up switch 190e and the down switch 190f. Alternatively, the focus adjustment optical element in the photographic lens 140 is driven so as to move in a direction away from the subject. The photographer can manually adjust the focus by operating the up switch 190e and the down switch 190f while viewing the live view image displayed on the image display unit 172.

  Incidentally, in the case where the photographing apparatus is configured so that the photographing lens can be exchanged, a photographing lens manufactured for a photographing apparatus different from the photographing apparatus may be attached via a mount adapter. In addition, since the lens mount is compatible, it can be directly attached to the photographing apparatus. However, since it is a photographing lens dedicated to manual focus, the above-described power focus operation may not be performed. In such a case, since the focus cannot be adjusted by operating the up switch 190e and the down switch 190f, the photographer operates the helicoid ring of the lens so that the live view image displayed on the image display unit 172 is sharp.

  The auxiliary light emitting device 150 can automatically start light emission when the subject falls below a predetermined luminance value. However, the user may determine to start light emission. The auxiliary light switch 190g is a switch operated when it is determined that the auxiliary light emitting device 150 needs to emit light.

  The photographing lens 140 includes an aperture drive unit 142, a focus adjustment drive unit 144, and a focusing lens 146 as a focus adjustment optical element. In FIG. 1, the focusing lens 146 is depicted as a single convex lens, but may be a concave lens. The focusing lens 146 may also include a plurality of lens elements. In that case, the focusing lens 146 may have a positive condensing function (power) or a negative condensing function as a whole. Further, the focusing lens 146 may be arranged at any position among all the lenses constituting the optical system of the photographing lens 140. Alternatively, focusing may be performed by moving the entire photographing optical system along the optical axis.

  The diaphragm driving unit 142 is based on a diaphragm mechanism for adjusting the amount of light transmitted through the photographing lens 140, a diaphragm actuator for driving the luminous diaphragm mechanism, and a diaphragm command value from the system controller 100. A diaphragm control unit that drives the diaphragm actuator and changes the aperture diameter of the iris diaphragm mechanism.

  The focus adjustment driving unit 144 includes a focusing actuator for driving the focusing lens 146, a focusing control unit that drives the focusing actuator based on a focusing command value from the system controller 100, and moves the imaging surface of the photographing lens 140. Is provided.

  The taking lens 140 may be a single focus lens or a variable focus lens. When the photographic lens 140 is a variable focus lens, the photographic lens 140 includes a manual magnification mechanism that allows the photographer to change the focal length by operating the operation ring for changing the focal length. Alternatively, a zooming motor and a lens driving mechanism are built in, and the focal length is electrically changed in response to the photographer operating a switch for changing the focal length provided in the operation unit 190. It may be configured.

  When the focal length of the photographing lens 140 is changed by any of the methods described above, it is desirable that information related to the set focal length is sent from the photographing lens 140 to the system controller 100. The irradiation range control unit 122 acquires information related to the focal length and outputs a signal for controlling the irradiation angle of the auxiliary light emitted from the auxiliary light emitting device 150 to the auxiliary light control unit 152.

  FIG. 5 is a flowchart schematically illustrating a processing procedure executed by the system controller 100 when moving image shooting or still image shooting is performed by the shooting apparatus. The processing shown in FIG. 5 is started when the photographing apparatus is turned on and the operation mode is switched to the photographing mode. Further, the premise when the process shown in FIG. 5 is executed by the system controller 100 is as follows. The photographing device is set to a mode for photographing a subject that is not illuminated by the auxiliary light emitting device 150, that is, an auxiliary light off mode. Further, the operation mode of the auxiliary light emitting device 150 is set to the B mode.

  Hereinafter, description will be made with reference to FIGS. 1 and 2 as well as FIG. In S500, the system controller 100 starts a live view operation. At this time, the image distribution processing unit 110 has not started the image distribution processing. Accordingly, a series of image streams generated and output by the image processing unit 104 is output to the subject recognition processing unit 108, the focus detection processing unit 114, and the display driving unit 170 without being distributed. Note that at the time when the process of S500 is executed, the distinction of the shooting mode (still image shooting mode or moving image shooting mode) has not been determined, so that no image data is output to the moving image data generation processing unit 112. .

  In S502, the system controller 100 determines whether or not the still image shooting mode is selected. In S504, which is a branch destination when the determination in S502 is negative, the system controller 100 determines whether the moving image shooting mode is selected. If the determination in S504 is negative, the process returns to S502, and the determination process in S502 and S504 is repeated until one of the still image shooting mode and the moving image shooting mode is selected.

  In S506, which is a branch destination when the determination in S504 is affirmative, that is, when it is determined that the moving image shooting mode is selected, the system controller 100 determines whether or not the photographer has performed a moving image shooting start operation. While the determination at S506 is negative, the determination process at S506 is repeated. If the determination in S506 is affirmed, the process proceeds to S508.

  In step S <b> 508, the system controller 100 permits the operation of the image distribution processing unit 110. As a result, the image distribution processing unit 110 starts a process of sequentially distributing a series of image streams received from the image processing unit 104 to the first and second time-series image data.

  In S510, the system controller 100 permits the operation of the moving image data generation processing unit 112. As a result, the moving image data generation processing unit 112 starts a process of generating moving image data based on the second time-series image data received from the image distribution processing unit 110.

  In S512, the system controller 100 performs setting so that an image based on the second time-series image data is displayed in a live view. In response to this setting, live view display image data based on the second time-series image data is output from the image distribution processing unit 110 to the display driving unit 170. As a result, the image display unit 172 starts displaying a live view image based on the second time-series image data. The photographer can know what kind of moving image is recorded by taking a moving image while viewing the live view image.

  In step S <b> 514, the system controller 100 permits the subject recognition processing unit 108 to start subject recognition processing. The subject recognition processing unit 108 starts to perform subject recognition processing using the first time-series image data received from the image distribution processing unit 110. The subject recognition process can be any one or a combination of a plurality of processing contents described above.

  In S516, the system controller 100 permits the focus detection processing unit 114 to start focus detection processing. The focus detection processing unit 114 starts to perform focus detection processing using the first time-series image data received from the image distribution processing unit 110. In the focus detection process, it is possible to sequentially repeat the calculation of the contrast value by analyzing the image data corresponding to the preset partial area in the screen, and to derive the change of the contrast value over time. It is.

  In S518, the system controller 100 determines whether or not the automatic focus adjustment mode is selected. If this determination is affirmative, the process proceeds to S520 (FIG. 5B), and if negative, the process branches to S560. The process starting from S520 is an autofocus process, and the process starting from S560 is a manual focus (power focus) process.

  In S520, the system controller 100 determines based on the signal received from the luminance detection processing unit 118 whether or not the current subject luminance is a luminance that requires the emission of auxiliary light. If the determination in S520 is affirmed, the process proceeds to S540. If the determination is negative, the process proceeds to S522.

  In S522, which is a branch destination when it is determined in S520 that the auxiliary light emission is not necessary, the system controller 100 determines whether or not the auxiliary light emitting device 150 is in a light emitting operation, and this determination is affirmed. Then, the process proceeds to S524, and if the result is negative, the process proceeds to S526.

  If the system controller 100 determines in S522 that the auxiliary light emitting device 150 is in a light emitting operation under the situation where the field luminance is relatively high (bright) and the auxiliary light emission is unnecessary, the auxiliary controller in S524 The light emitting operation of the light emitting device 150 is stopped, and the process proceeds to S526. On the other hand, if it is determined in S522 that the auxiliary light emitting device 150 is not emitting light, the system controller 100 skips the processing of S524 and proceeds to S526.

  In S540, which is a branch destination when it is determined in S520 that the auxiliary light emission is necessary, the system controller 100 determines whether or not the auxiliary light emitting device 150 is in a light emitting operation, and this determination is affirmed. Then, the process proceeds to S526, and if the result is negative, the process proceeds to S542.

  If the system controller 100 determines in S540 that the auxiliary light emitting device 150 is not in a light emitting operation under a situation where the field luminance is relatively low (dark) and the auxiliary light needs to be emitted, the auxiliary light is obtained in S542. The light emitting operation of the light emitting device 150 is permitted, and the process proceeds to S526. On the other hand, if it is determined in S540 that the auxiliary light emitting device 150 is already in a light emitting operation, the system controller 100 skips the process of S542 and proceeds to S526.

  When the processing of S542 described above is performed, timing control is performed so that the auxiliary light emitting device 150 emits auxiliary light in synchronization with the accumulation operation for acquiring the first time-series image data. That is, the light emission operation in the B mode is started.

  In step S526, the system controller 100 acquires subject position information as information generated by subject recognition processing performed by the subject recognition processing unit 108. This subject position information is information that can specify where the main subject is shown in the screen.

  In step S <b> 528, the system controller 100 outputs, to the auxiliary light control unit 152, information specifying an irradiation range when auxiliary light is emitted from the auxiliary light emitting device 150 based on the subject position information acquired in S <b> 526. By the way, when the auxiliary light emitting device 150 is built in the photographing apparatus, the system controller 100 can grasp the number of arrays in the row direction and column direction of the LEDs constituting the LED array 164. In that case, the system controller 100 may output information specifying an LED to emit light as information for specifying the irradiation range. Alternatively, when communication is performed in advance between the auxiliary light control unit 152 and the system controller 100 and it is known whether the auxiliary light emitting device 150 includes each LED (light emitting unit) in the row and column directions. In addition, it is possible to output information specifying the LED to emit light.

  In step S530, the system controller 100 performs focus adjustment state detection processing. That is, the result of the focus detection process performed in the latest past is received from the focus detection processing unit 114. Then, based on the moving direction of the imaging plane by the focus adjustment operation performed in the past and the change tendency (increase / decrease tendency) of the contrast value, the direction and movement when moving the imaging plane next time Processing to determine the quantity. Here, the moving direction of the imaging surface means a direction in which the imaging surface of the photographic lens 140 approaches the subject and a direction away from the subject.

  In S532, the system controller 100 performs focus adjustment control. That is, a process of outputting a control signal for focus adjustment from the focus adjustment control unit 116 to the focus adjustment drive unit 144 of the photographing lens 140 is performed.

  In step S534, the system controller 100 determines whether the photographer has performed an operation for ending moving image shooting. While the determination in S534 is negative, a series of processes from S520 to S532 and S540 and S542 are continuously performed. On the other hand, if the determination in S534 is affirmative, the system controller 100 proceeds to S536 and performs a moving image shooting end process. Along with this processing, processing for stopping the operation of the circuit related to moving image shooting is performed. In addition, processing for closing the moving image file recorded in the memory card MC is performed, and a series of shooting operations is completed. At this time, the system controller 100 outputs a control signal for turning off the auxiliary light emitting device 150 to the auxiliary light control unit 152 when the auxiliary light emitting device 150 is on.

  Next, processing in the case where the determination in S518 of FIG. 5A is negative, that is, in the case where it is determined that the photographing apparatus is currently set in the moving image photographing mode by manual focus will be described with reference to FIG. 5C.

  In S560, the system controller 100 performs setting so that an image based on the first time-series image data is displayed in live view. In response to this setting, live view display image data based on the first time-series image data is output from the image distribution processing unit 110 to the display driving unit 170. As a result, the image display unit 172 starts displaying a live view image based on the first time-series image data.

  In step S <b> 562, the system controller 100 determines whether or not the current subject brightness is a brightness that requires the emission of auxiliary light based on the signal received from the brightness detection processing unit 118. If the determination in S562 is affirmed, the process proceeds to S580. If the determination is negative, the process proceeds to S564.

  In S564, which is a branch destination when it is determined in S562 that the auxiliary light emission is not necessary, the system controller 100 determines whether or not the auxiliary light emitting device 150 is in a light emitting operation, and this determination is affirmed. If so, the process proceeds to S566. If denied, the process proceeds to S568. In other words, when the system controller 100 determines in S564 that the auxiliary light emitting device 150 is in a light emitting operation under the situation where the field luminance is relatively high (bright) and the auxiliary light emission is unnecessary, the process proceeds to S566. In step S568, the light emission operation of the auxiliary light emitting device 150 is stopped, and the process proceeds to S568. On the other hand, if it is determined in S564 that the auxiliary light emitting device 150 is not emitting light, the system controller 100 skips the process of S566 and proceeds to S568.

  In S580, which is a branch destination when it is determined in S562 that the auxiliary light emission is necessary, the system controller 100 determines whether or not the auxiliary light emitting device 150 is in a light emitting operation, and this determination is affirmed. If so, the process proceeds to S568, and if negative, the process proceeds to S582. That is, if the system controller 100 determines in S580 that the auxiliary light emitting device 150 is not in a light emitting operation under a situation where the field luminance is relatively low (dark) and the auxiliary light needs to be emitted, the process proceeds to S582. Then, the light emitting operation of the auxiliary light emitting device 150 is permitted, and the process proceeds to S568. On the other hand, if it is determined in S580 that the auxiliary light emitting device 150 is already in the light emitting operation, the system controller 100 skips the process of S582 and proceeds to S568.

  When the processing of S582 described above is performed, timing control is performed so that the auxiliary light emitting device 150 emits auxiliary light in synchronization with the accumulation operation for acquiring the first time-series image data. That is, the light emission operation in the B mode is started. In S560, since the image based on the first time-series image data is switched so as to be displayed in live view, the subject in the image displayed in live view is in a state illuminated by the auxiliary light emitting device 150. .

  In S568, the system controller 100 determines whether or not an operation of the up switch 190e by the photographer has been detected. If the determination in S568 is affirmed, the process proceeds to S590, and if the determination is negative, the process proceeds to S570.

  In S590, which is a branch destination when the determination in S568 is affirmative, the system controller 100 sets the focusing lens 146 in the direction in which the imaging surface of the photographing lens 140 approaches the subject (the direction in which the subject at a closer position is in focus). A control signal is output to the focus adjustment driving unit 144 so as to move in a fixed amount. At this time, a control signal is output from the focus adjustment control unit 116 to the focus adjustment drive unit 144. By the way, when the photographing lens 140 performs focus adjustment by a so-called inner focus method, the ratio between the amount of movement of the focusing lens 146 along the optical axis direction and the amount of movement of the imaging plane associated therewith varies. Therefore, the predetermined amount is also various. As an example of the predetermined amount, it is possible to use an amount necessary for the image plane to move by 20 μm, 50 μm, or 100 μm. After the process of S590, the process proceeds to S572.

  In S570, which is a branch destination when the determination in S568 is negative, the system controller 100 determines whether or not an operation of the down switch 190f by the photographer has been detected. If the determination in S570 is affirmed, the process proceeds to S592, and if the determination is negative, the process proceeds to S572.

  In S592, which is a branch destination when the determination in S570 is affirmative, the system controller 100 places the focusing lens 146 in a direction in which the imaging plane of the photographing lens 140 moves away from the subject (a direction in which the subject at a farther position is in focus). A control signal is output to the focus adjustment driving unit 144 so as to move in a fixed amount. At this time, a control signal is output from the focus adjustment control unit 116 to the focus adjustment drive unit 144. The predetermined amount here may be the same as described above. After the process of S592, the process proceeds to S572.

  In step S572, the system controller 100 determines whether the photographer has performed an operation for ending the moving image shooting. While the determination in S572 is negative, a series of processes from S560 to S570 and S580, S582, S590, and S592 are continuously performed. On the other hand, if the determination in S572 is affirmed, the system controller 100 proceeds to S536 (FIG. 5B), and performs the above-described moving image shooting end operation.

  With reference to FIG. 5A, the still image shooting process executed when the determination in S502 is affirmed will be described. In step S550, the system controller 100 performs a series of still image shooting processing. In this series of still image shooting processing, live view display operation, operation to detect the release operation of the photographer, focus adjustment operation, exposure determination operation, operation to adjust the aperture of the taking lens 140, operation to open and close the shutter, For example, an operation of reading an image signal from the image sensor 130 and performing image processing to generate a still image file is included. When the shutter is open, the auxiliary light emitting device 150 can be operated as needed to illuminate the subject.

  In step S552, the system controller 100 determines whether the photographer has performed an operation to end the still image shooting operation. While the determination in S552 is negative, the system controller 100 continues the still image shooting process in S550. If the determination in S552 is affirmative, the system controller 100 proceeds to S554 and performs still image shooting end processing. Along with this processing, processing for stopping the operation of the circuit related to still image shooting is performed. Then, a series of shooting operations is completed.

  As described above, according to the embodiment of the present invention, the recorded moving image can have an atmosphere by utilizing the light in the shooting scene, while the focus detection accuracy and subject recognition processing are performed. It is possible to improve the accuracy of such as.

  In addition, when the photographer performs manual focus operation, an image obtained by irradiating the subject with auxiliary light is displayed as a live view image, so that focus adjustment and framing can be performed even in a relatively dark shooting scene. It becomes easy.

  In the above description, the example in which the auxiliary light emitting device 150 is always turned on in the B mode while it is determined that the auxiliary light emission is necessary during moving image shooting has been described. The auxiliary light emitting device 150 may emit light in the B mode only while the adjustment operation is being performed. At this time, when the photographing apparatus is operating in the manual focus mode, the light emitting operation is performed in the B mode while the photographer operates the up switch 190e or the down switch 190f and for a predetermined time after the operation is completed. May be. Further, when the manual focus lens is attached to the photographing apparatus, the photographer performs the focus adjustment operation while operating the auxiliary light switch 190g, and emits light in the B mode while the auxiliary light switch 190g is operated. You may make it operate | move. By controlling the light emitting operation in this manner, it is possible to suppress power consumption accompanying light emission of the auxiliary light emitting device 150.

  Regarding the emission of the auxiliary light, the system controller 100 may determine whether to emit auxiliary light or not based on the luminance of the field as described above, and may control so-called low-brightness automatic light emission. Light may be emitted only when light emission is required (no light is emitted when the photographer does not select auxiliary light emission). In this case, the system controller 100 operates the auxiliary light emitting device 150 in the B mode when it is detected that the photographer has operated the auxiliary light switch 190g and the field luminance is lower than a predetermined luminance. It is possible to perform a process of shooting a moving image.

  In the embodiment of the present invention, the example in which the focus detection processing unit 114 performs the focus detection process by detecting the contrast value has been described. However, the focus detection process may be performed by the phase difference detection method. The phase difference detection is performed based on a phase difference between images formed on one or more pairs of AF (automatic focus detection) sensor arrays that are transmitted through different pupil positions in the photographing lens 140 and provided in the photographing apparatus. In this method, the focus adjustment state of the lens 140 is detected.

  In order to continue the focus adjustment during the moving image shooting operation, it is desirable that the focus detection can be performed during the shooting operation even when the focus detection process is performed by the phase difference detection method. As a method for this, the light transmitted through the photographing lens 140 is divided into light traveling along a plurality of optical paths using optical path dividing means such as a half mirror, and one is used for the image sensor 130 and the other is used for phase difference detection. It is possible to lead to an AF sensor array of Alternatively, by embedding a phase difference detecting element (pixel) at an appropriate location on the light receiving surface of the image sensor 130 and detecting the phase difference with this embedded type phase difference detecting element, the above-described optical path dividing means is unnecessary. Become.

  At the same time that the photoelectric conversion operation is performed by the phase difference detection element embedded in the image sensor 130 or a separate phase difference detection element, the image sensor 130 may perform an image capturing operation for obtaining first time-series image data. it can. Then, at the timing when the image pickup operation for obtaining image data for moving image recording (second time-series image data) is performed by the image pickup device 130, the auxiliary light is not irradiated and the image pickup device 130 is embedded in the image pickup device 130, or It is possible to irradiate the auxiliary light in synchronization with the accumulation operation performed by a separate phase difference detection element.

  In the above description of the embodiment, the imaging frame rate, the moving image recording frame rate, and the like when the image distribution processing unit 110 performs image distribution processing are not mentioned, but it is possible to capture and record moving images by various methods. It is.

  For example, when the image sensor 130 continues the imaging operation at 60 fps (frames / second) and the image distribution processing unit 110 operates, as described above, the odd frame image is converted into the first time-series image data as the even frame. Can be distributed to the second time-series image data. Of course, the distinction between the odd frames and the even frames is merely for convenience of explanation, so that the images of the even frames may be allocated to the first time-series image data and the odd frames may be allocated to the second time-series image data.

  Further, the image sensor 130 can perform an imaging operation at, for example, 30 fps when the image distribution processing unit 110 is not operated, and can increase the imaging frame rate (for example, 60 fps) when the image distribution processing unit 110 is operated. is there.

  Alternatively, the imaging element 130 may operate at a constant imaging frame rate regardless of whether the image distribution processing unit 110 operates or not, and the distribution process may be performed as follows. When the distribution process is performed by the image distribution processing unit 110, an image obtained at the imaging timing of the image distributed to the first time-series image data is lost in the second time-series image data. The missing frame may be generated by interpolation processing using image data of a frame that is temporally before and after the missing frame in the second time-series image data. This is the same regardless of whether the imaging operation is performed by a progressive scan method or an interlace scan method. By the way, when moving image shooting is performed, the number of image updates per second is referred to as a frame rate or a field rate depending on whether progressive scan or interlace scan is performed. In this specification, these frame rate and field rate are collectively referred to as an imaging rate.

  As described above, the distribution process performed by the image distribution processing unit 110 can distribute the image streams alternately to the first and second time-series image data, or can distribute them unevenly.

  Regarding the spectral characteristics of the light emitted from the auxiliary light emitting device 150, various characteristics (color temperature) similar to so-called white light, A light source, B light source, or the like can be used. Further, when the auxiliary light emitting device 150 is used exclusively for focus detection, so-called monochromatic light may be emitted. At this time, the image sensor 130 and the AF sensor for detecting the phase difference, although outside the visible range, emit light having a wavelength that can be detected (for example, near-infrared light). The auxiliary light emitting device 150 can be used even in a situation where it is beaten.

100 ... System controller 102 ... CPU
DESCRIPTION OF SYMBOLS 104 ... Image processing part 108 ... Compression / decompression part 110 ... Image distribution processing part 112 ... Moving image data generation processing part 114 ... Focus detection processing part 116 ... Focus adjustment control part 118 ... Luminance detection processing part 120 ... Light emission timing control part 130 Image pickup device 132 Image pickup device interface circuit 140 Shooting lens 144 Focus adjustment drive unit 146 Focusing lens 150 Auxiliary light emitting device 152 Auxiliary light control unit 154 Auxiliary light controller 156 Current control unit 158 Lighting control unit 160 ... Irradiation angle control unit 162 ... Light distribution lens 164 ... Auxiliary light emitting device 170 ... Display drive unit 172 ... Image display unit 176 ... Flash memory 180 ... SDRAM
190… Operation unit

Claims (11)

An imaging unit capable of repeatedly performing an imaging operation of imaging a subject image formed by the imaging lens, and outputting image data in time series corresponding to each of the imaging operations;
A distribution processing unit that distributes the image data output in time series from the imaging unit to the first time-series image data and the second time-series image data in units of frames or fields;
An auxiliary light emitting device that emits illumination light toward a subject performs an imaging operation for outputting image data corresponding to the first time-series image data among the first and second time-series image data. A light emission timing control unit that controls to emit light in synchronization with the timing performed by the imaging unit;
An imaging apparatus comprising: a moving image data generation unit configured to generate moving image data based on the second time-series image data. A first focus detection processing unit that sequentially analyzes the first time-series image data to derive a contrast value and detects a focus adjustment state of the photographing lens;
Based on a result of the detection by the first focus detection processing unit, an operation related to focus adjustment of the photographing lens is controlled so that the subject image formed on the light receiving surface of the imaging unit becomes sharper. The imaging apparatus according to claim 1, further comprising a first focus adjustment control unit. Signals are read from one or more pairs of sensor arrays capable of performing a photoelectric conversion operation in synchronization with lighting of the auxiliary light, and the one or more pairs are obtained by subject light incident through different pupil positions in the photographing lens. A second focus detection processing unit for detecting a focus adjustment state of the photographing lens based on a phase difference of an image formed on the sensor array;
Based on the detection result of the focus adjustment state of the photographing lens by the second focus detection processing unit, the focus adjustment of the photographing lens is performed so that the subject image formed on the light receiving surface of the imaging unit becomes sharper. The photographing apparatus according to claim 1, further comprising a second focus adjustment control unit that controls an operation according to the above. A focus adjustment operation detection unit that detects a focus adjustment operation that is an operation of the focus adjustment operation unit by the photographer;
In response to detection of the focus adjustment operation by the focus adjustment operation detection unit, one or a plurality of focus adjustment optical elements provided in the imaging lens are driven to display the imaging surface of the imaging lens. The imaging apparatus according to claim 1, further comprising a third focus adjustment control unit that performs control for moving the light receiving surface forward or backward. An auxiliary light emission operation detection unit that detects an auxiliary light emission operation that is an operation of the auxiliary light emission operation unit by the photographer;
The light emission timing control unit performs an imaging operation for outputting image data corresponding to the first time-series image data when the auxiliary light emission operation detection unit detects the auxiliary light emission operation. 2. The photographing apparatus according to claim 1, wherein control is performed so that the auxiliary light emitting device emits light in synchronization with a timing performed by a unit. An image display unit capable of displaying an image based on the first time-series image data or the second time-series image data as a live view image;
The auto-focus mode that automatically adjusts the focus, and the manual focus mode that the photographer manually adjusts the focus.
When the automatic focus adjustment mode is selected, an image based on the second time-series image data is displayed, and when the manual focus adjustment mode is selected, an image based on the first time-series image data is displayed. The photographing apparatus according to claim 4, wherein the photographing apparatus is configured to display as a view image. The imaging unit includes a plurality of types including a first imaging rate and a second imaging rate that is an imaging rate higher than the first imaging rate as an imaging rate that is a frame rate or a field rate during an imaging operation. The imaging operation can be performed at any imaging rate among the imaging rates of
The imaging unit performs the imaging operation at the first imaging rate when the auxiliary light emitting device does not emit light, while the imaging unit performs the imaging operation at the second imaging rate when the auxiliary light emitting device emits light. Perform the action
The distribution processing unit converts the first time-series image data and the second time-series image into image data output in time series when the imaging operation is performed at the second imaging rate by the imaging unit. The photographing apparatus according to claim 1, wherein the data is alternately allocated to data.   The moving image data generation unit further generates image data corresponding to a frame or a field allocated to the first time-series image data in the distribution processing unit by interpolation processing, and the generated image data and the The imaging apparatus according to any one of claims 1 to 6, wherein the moving image data is generated from second time-series image data. The auxiliary light emitting device includes a plurality of light emitting units, and is configured to be able to change a light irradiation range by selectively causing some of the light emitting units to emit light. ,
A subject recognition processing unit for processing image data output from the imaging unit and deriving a position where a main subject appears in a shooting range;
2. An irradiation range control unit that controls an irradiation range of the light of the auxiliary light emitting device so as to irradiate a position where the main subject derived by the subject recognition processing unit is reflected. Or the imaging device of 2. The auxiliary light emitting device includes a plurality of light emitting units, and is configured to be able to change a light irradiation range by selectively causing some of the light emitting units to emit light. ,
A face recognition processing unit for processing image data output from the imaging unit and deriving a position where the face appears in the shooting range;
The irradiation range control unit that controls the irradiation range of the light of the auxiliary light emitting device so as to irradiate the position where the face derived by the face recognition processing unit is reflected. The imaging device according to 2. The auxiliary light emitting device includes a plurality of light emitting units, and is configured to be able to change a light irradiation range by selectively causing some of the light emitting units to emit light. ,
A motion vector derivation processing unit that sequentially processes image data output in time series from the imaging unit to derive a motion vector, analyzes the motion vector, and derives a position where a main subject appears in the imaging range; ,
An irradiation range control unit that controls an irradiation range of the light of the auxiliary light emitting device so as to irradiate a position where the main subject derived by the motion vector derivation processing unit is reflected. The imaging apparatus according to 1 or 2.

Images