JP6643371B2 - Imaging device, control method, and program - Google Patents

Description

  The present invention relates to an imaging device, a control method, and a program.

  2. Description of the Related Art Some imaging devices such as digital cameras have a continuous shooting (continuous shooting) function of continuously shooting still images while performing automatic focus detection (imaging surface phase difference AF) of an imaging surface phase difference method on a main subject. Some are provided. During the execution of the continuous shooting function, the imaging apparatus reads an image to be displayed as a live view (LV) image on a rear monitor or the like and a recorded image to be recorded as a still image, and performs display and recording in real time.

  For example, there is known a technique for improving the followability to a main subject when displaying an image acquired from an imaging element as a live view image (LV image) on a display device while performing focus detection during continuous shooting. Patent Literature 1 proposes a technique for switching between sequentially displaying images having different resolutions or displaying only high-resolution images on a display device. According to Patent Literature 1, even in continuous shooting with a low frame rate, the display time interval of the LV image can be shortened, and the followability to the main subject during framing can be improved.

JP 2015-144346 A

  However, in the technique proposed in Patent Document 1, when images having different resolutions are sequentially displayed on a display device, a difference in image quality depending on a difference in exposure between images having different resolutions, a difference in signal reading conditions, and the like. Did not mention. If a difference occurs in the image quality between images due to the above-described causes, images having different brightness, resolution, contrast, etc., are displayed alternately. Display. In this case, there is a problem that the discomfort given to the user is increased by alternately displaying the images.

  The present invention proposes a technique for reducing discomfort given to a user when acquiring and displaying different types of images during continuous shooting.

An imaging device that captures an image of a subject through an imaging optical system according to an example of the present invention includes an imaging device that acquires an image signal from light formed by the imaging optical system, and a first device that controls an exposure parameter of the imaging device. A control unit, a first image corresponding to a pixel unit having a first pixel number among pixel units of the image sensor, and a pixel unit having a second pixel number smaller than the first pixel number; A second control unit that performs a process of switching and outputting the second image. The second control unit makes a focus determination based on at least one of the first image and the second image, and the second control unit outputs an image signal obtained by pupil division by the image sensor. read and perform focus detection, controls the focusing of the imaging optical system, said first control means, by controlling the exposure parameters for acquiring the first image and the second image Reducing the amount of change in the exposure parameter between the first image and the second image acquired continuously , and wherein the first control means acquires the second image based on the focus determination. By controlling the exposure parameter, the amount of change in the exposure parameter between the first image and the second image acquired successively is reduced .

  According to the example of the present invention, it is possible to reduce discomfort given to a user when acquiring and displaying different types of images during continuous shooting.

FIG. 2 is a schematic structural diagram of an image sensor according to an embodiment of the present invention. FIG. 2 is a pixel array diagram of an image sensor according to an embodiment of the present invention. 1 is a schematic diagram of an imaging system according to an embodiment of the present invention. FIG. 4 is a diagram illustrating a focus detection signal according to the embodiment of the present invention. FIG. 6 is a diagram illustrating a focus determination table according to the first embodiment. 5 is a flowchart illustrating a process according to an embodiment of the present invention. FIG. 7 is a flowchart illustrating processing subsequent to FIG. 6. FIG. 5 is a timing chart of alternate display in the embodiment of the present invention. FIG. 7 is a timing chart in a case where alternate display is not performed in the embodiment of the present invention. FIG. 11 is a diagram illustrating a focus determination table according to a second embodiment. FIG. 11 is a diagram illustrating a focus determination table according to a second embodiment. It is a schematic diagram of an imaging system in a third example. It is a pixel array figure of an image sensor in a 3rd example. It is a flow chart of the display processing in a 3rd example. 13 is a flowchart of an LV image generation / display process in a third embodiment. It is a flowchart of an imaging subroutine process in the third embodiment. It is a flowchart of a display determination process in the third embodiment. 13 is a flowchart of still image generation / display processing in the third embodiment. 5 is a timing chart when an LV image and a still image are displayed. It is a timing chart at the time of displaying an LV image and a still image alternately. It is a timing chart at the time of displaying an LV image and a still image alternately. It is a figure of the alternate display method in 4th Example. It is a figure of the alternate display method in 4th Example. 15 is a flowchart of LV image generation / display processing in a fourth embodiment. 15 is a flowchart of still image generation / display processing in a fourth embodiment.

  Hereinafter, examples of the present invention will be described in detail. In each of the embodiments, an imaging apparatus including an imaging element having pupil-divided focus detection pixels and performing continuous shooting (AF continuous shooting) while focusing on a subject will be described. In continuous shooting, even if the aperture value changes during still image shooting, aperture control is performed so that the difference from the aperture value during live view shooting is reduced, thereby maintaining the display quality and framing during framing. A decrease in followability can be suppressed.

[First embodiment]
FIG. 1 is a schematic block diagram illustrating a configuration example of an image sensor according to a first embodiment of the present invention. The imaging element 306 has a configuration in which many pixel units 101 are arranged in a two-dimensional array. A vertical output line 102, a transfer signal line 103, a reset signal line 104, and a row selection signal line 105 are connected to each pixel portion 101, respectively. The column ADC block 111 outputs an A (analog) / D (digital) converted signal with respect to a signal output from the vertical output line 102 connected to the pixel unit 101. The row scanning circuit 112 is connected to the pixel portion 101 by a transfer signal line 103, a reset signal line 104, and a row selection signal line 105. The plurality of column scanning circuits 113 are connected to the plurality of column ADC blocks 111 by horizontal signal lines 115-a and 115-b. The timing control circuit 114 performs control by outputting a timing control signal to each of the column ADC block 111 and the column scanning circuit 113.

  The switching unit 116 switches between signals on the horizontal signal lines 115-a and 115-b and outputs the signals to a parallel / serial conversion unit (hereinafter, referred to as a P / S conversion unit) 117. The P / S conversion unit 117 acquires the output of the switching unit 116 and performs parallel / serial conversion. The P / S converter 117 outputs the converted signal to the outside.

  In the image sensor 306 of this embodiment, the plurality of pixel units 101 are connected to the transfer signal line 103, the reset signal line 104, and the row selection signal line 105 in the horizontal direction (row direction), and are vertically arranged in the vertical direction (column direction). It is connected to the output line 102. Each of the vertical output lines 102 has a different connection destination depending on the readout row unit. A signal read from the pixel unit 101 is output from the horizontal signal line 115-a and the horizontal signal line 115-b for each channel via the column ADC block 111, and sent to the switching unit 116. The image signal selected by the switching unit 116 is subjected to parallel / serial conversion by the P / S conversion unit 117 in accordance with the timing of the timing control circuit 114, and is output to the outside of the image sensor 306.

  As a method for reading out pixel signals, a method for reading out all pixels, a method for reading out by thinning out pixels in the vertical direction, a method for adding and reading out pixels in the horizontal direction, a vertical thinning-out horizontal addition method, and the like can be appropriately selected. . The vertical thinning-out horizontal addition method is a method in which thinning-out reading is performed in the vertical direction, and pixels are added and read out in the horizontal direction. In this embodiment, a method of reading out all pixels is adopted as a still image reading method. In the first reading mode, pixel signals are read from all the pixel units of the image sensor as the pixel units having the first number of pixels. Since the number of pixels to be read is large, only a limited number of images can be acquired within a predetermined time. Therefore, when a moving subject is moving at high speed, it is difficult to accurately capture the subject within the angle of view. On the other hand, a vertical thinning-out horizontal addition method is adopted as a live view reading method. In the second reading mode, a pixel signal is read from a pixel portion having a second number of pixels smaller than the first number of pixels. Since the number of readout pixels is small, it is suitable for high-speed processing.

  FIG. 2 schematically illustrates a pixel array of the image sensor 306 used in the image plane phase difference AF according to the present embodiment. The imaging element 306 has a configuration in which a plurality of photoelectric conversion units corresponding to the on-chip microlens 201 are arranged. A Bayer array is applied to the color filters, and red (RED) and green (GREEN) color filters are provided alternately from left to right in pixels in the odd-numbered rows indicated by R / G rows. In addition, the pixels of the even-numbered rows indicated by the G / B rows are provided with green (GREEN) and blue (BLUE) color filters alternately from the left. The on-chip micro lens 201 is configured on a color filter. A plurality of photoelectric conversion units arranged inside the on-chip micro lens 201 are indicated by a plurality of rectangles. The photoelectric conversion units forming a pair in the pupil division type imaging device are referred to as an A pixel 101a and a B pixel 101b, respectively. An A image that is a first viewpoint image is generated by an output from the A pixel 101a included in the first pixel group. A B image that is a second viewpoint image is generated by an output from the B pixel 101b included in the second pixel group. The calculation unit detects a relative image shift amount between the A image and the B image by a correlation operation, and calculates a defocus amount of a predetermined area. The focus lens moves based on the defocus amount, and the focus adjustment operation of the lens unit 301 is performed. In addition, the signal of the A image and the signal of the B image are subjected to addition processing in the image sensor, and image signals for display and recording are generated. Note that a configuration in which a part of the light receiving surface of the photoelectric conversion unit is shielded from light may be employed instead of a pixel configuration in which a plurality of photoelectric conversion units are provided as illustrated in FIG.

  FIG. 3 is a block diagram illustrating an outline of an imaging system using the imaging element 306. The imaging system of the present embodiment is a digital camera that acquires a moving image or a still image as an example, but is not limited to this. For example, the present invention is also applicable to mobile devices such as surveillance cameras and smartphones, and mobile objects such as in-vehicle cameras.

  The lens unit 301 includes a plurality of lenses forming an imaging optical system. The lens unit 301 is an interchangeable lens that can be mounted on the camera body, and a lens unit integrated with the camera body. The lens driving unit 302 drives a movable lens (a zoom lens, a focus lens, and the like) included in the imaging optical system. A mechanical shutter (denoted as a mechanical shutter in the figure) 303 is used for controlling an exposure time, and an aperture 304 is used for controlling an exposure amount. A mechanical shutter / aperture drive unit (denoted as a shutter / aperture drive unit) 305 in the figure controls the exposure state of the image sensor 306 by driving the mechanical shutter 303 and the aperture 304. Note that an optical filter such as an ND filter may be provided as a means for controlling the exposure amount.

  The imaging element 306 receives light from a subject formed through an imaging optical system, performs photoelectric conversion, and outputs an electric signal. In the case of a stacked-type imaging device, it has an imaging layer and a circuit layer. The imaging signal processing circuit 307 processes the output signal of the imaging element 306 and outputs the processed image signal. A first memory unit (denoted as a memory unit I in the figure) 308 stores an image signal and the like processed by the imaging signal processing circuit 307.

  The overall control calculation unit 309 is a central unit that controls the entire imaging system, and includes a CPU (central processing unit). The CPU controls the operation of each unit by executing a program read from a second memory unit (denoted as a memory unit II in the figure) 314. The second memory unit 314 also stores calculation results of the CPU, camera information set in the imaging device by a user operation, and the like. The recording medium control interface (I / F) unit 310 performs a process of recording an image signal or the like on the recording medium 312 and reading information from the recording medium 312 according to a control command of the overall control calculation unit 309. The recording medium 312 is detachable from the main body of the imaging device. The display unit 311 includes a display device such as a liquid crystal display panel, and displays image data and the like on a screen according to a control command of the overall control calculation unit 309. The external interface (I / F) unit 313 is a communication processing unit that exchanges information with an external device such as a computer. In the present embodiment, the imaging device includes the display unit 311. However, in the case of an imaging device without a display unit, image information and related information are output to an external display device via the external I / F unit 313. For example, a process of alternately outputting a still image and a live view image during AF continuous shooting and a process of outputting only the live view image are executed. The operation unit 315 includes an input device such as a switch or a touch panel, receives an operation instruction from a user, and outputs an operation instruction signal to the overall control calculation unit 309. The overall control calculation unit 309 controls the entire imaging system based on information on the imaging mode, the exposure condition, and the like of the imaging system set by the user using the operation unit 315.

  The light from the subject that has passed through the lens unit 301 is adjusted to an appropriate light amount by the stop 304, and is formed on the imaging surface of the imaging element 306. A photoelectric conversion unit included in the pixel unit 101 of the imaging element 306 performs photoelectric conversion on an optical image of a subject and outputs an electric signal. The electric signal is further subjected to gain control, is converted from an analog signal to a digital signal by A / D conversion, is captured as R, Gr, Gb, and B signals, and is sent to the imaging signal processing circuit 307.

  The imaging signal processing circuit 307 performs a predetermined calculation process using the captured image data. The overall control calculation unit 309 performs exposure control and distance measurement control based on the obtained calculation results. As a result, TTL (through the lens) AE (automatic exposure) processing and EF (flash automatic light emission control) processing are performed. Further, the imaging signal processing circuit 307 performs predetermined arithmetic processing using the captured image data, and performs TTL AWB (auto white balance) processing based on the obtained arithmetic result. In addition, the imaging signal processing circuit 307 performs various signal processing such as low-pass filter processing for reducing noise, shading correction processing, and white balance processing, and performs various corrections and compression of image signals.

  The lens unit 301 during shooting is controlled by a lens driving unit 302 such as zoom driving and focus driving. The mechanical shutter 303 and the diaphragm 304 are respectively driven by a mechanical shutter / aperture driving unit 305 according to a control command of the overall control calculation unit 309. The first memory unit 308 temporarily stores the image signal after imaging. The recording medium control I / F unit 310 performs a process of recording an image signal on the recording medium 312 and a process of reading a signal from the recording medium 312. The display unit 311 displays a captured image on a screen.

  Next, with reference to FIG. 4, focus determination in the present embodiment will be described. FIG. 4 is a diagram illustrating a focus detection signal when each output signal of the A pixel and the B pixel in the present embodiment is sent to the imaging signal processing circuit 307. In FIG. 4, the horizontal axis X indicates the position of the connected signal in the pixel arrangement direction, and the vertical axis Y indicates the signal intensity. The focus detection signal 401a corresponds to the signal of the A image, and the focus detection signal 401b corresponds to the signal of the B image. These signals are signals formed as an A image signal and a B image signal, which are image signals for focus detection, respectively. In the example illustrated in FIG. 4, since the lens unit 301 is in a defocused state with respect to the imaging element 306, the focus detection signal 401a is shifted to the left and the focus detection signal 401b is shifted to the right. The defocus amount is calculated from the image shift amount by the imaging signal processing circuit 307 calculating the shift amount of the signal between the A image and the B image. That is, it is possible to detect how much the lens unit 301 is out of focus.

ただし、ａ_ｊはｊ番目のＡ画素の信号であり、ｂ_ｊはＢ画素の信号である。（１）式の量Ｕは合焦時に最小値となる。 Specifically, the imaging signal processing circuit 307 performs a correlation operation for calculating a phase difference between the A image signal and the B image signal, and performs a distance measurement operation for calculating a subject distance from the phase difference. The subject distance is a distance from the imaging device to the subject, and is calculated as an amount corresponding to the defocus amount. Equation (1) can be used as the quantity U representing the degree of coincidence between the A image and the B image.

Here, a _j is the signal of the j-th A pixel, and b _j is the signal of the B pixel. The amount U in the equation (1) becomes a minimum value when focusing.

合焦時のシフト量においてＰは最小値となる。このシフト量に基づいて、どの程度合焦位置からずれているかを示すデフォーカス量が算出される。 Further, a process of shifting the A image relative to the B image by p pixels and calculating the value of U is performed. The amount calculated by equation (2) is denoted by P.

P becomes the minimum value in the shift amount at the time of focusing. Based on this shift amount, a defocus amount indicating how much the camera is deviated from the in-focus position is calculated.

  When the defocus amount becomes zero, that is, when the image is in focus, the A image and the B image should match, but if there is a difference between the A image and the B image, the accuracy of the imaging surface phase difference AF decreases. Resulting in. Therefore, for example, a process is performed in which the area of the common region of the two images is determined, and the reliability is determined according to the determined area. The reliability of focusing is determined to be higher as the area is larger, and the reliability of focusing is determined to be lower as the area is smaller.

（３）式において、Ｅ値が大きな値である場合、合焦の信頼性が高いと判断され、Ｅ値が小さな値である場合には合焦の信頼性が低いと判断される。 Next, the imaging signal processing circuit 307 calculates the steepness of the edges of the focus detection signals 401a and 401b in FIG. The steepness of the edge (denoted by E) is calculated using equation (3).

In equation (3), when the E value is a large value, it is determined that the reliability of focusing is high, and when the E value is a small value, it is determined that the reliability of focusing is low.

  FIG. 5 is a diagram illustrating a focus determination table according to the present embodiment. The focus determination table has the following four levels of state values A to D according to the reliability of the correlation calculation result and the reliability based on the edge steepness E. Regarding the reliability of the correlation operation result, there are three levels of “large”, “medium”, and “small”, and the smaller the correlation operation result (U value), the higher the reliability. The reliability based on the edge steepness E is set to three levels of “high”, “medium”, and “low”, and the higher the E value, the higher the reliability.

A: In-focus state.
B: Out of focus. Focusing is possible by slightly changing the shooting conditions.
C: Out of focus. If the photographing condition is changed more than B, focusing can be performed.
D: State in which focus is not achieved at all. If the photographing conditions are not significantly changed, focusing will not be achieved.
For example, when the correlation calculation result is “small” and the edge steepness E is “high”, the result is A, which is a state value at the time of focusing. In the present embodiment, focus determination in four stages will be described for convenience of description, but it is needless to say that focus determination in less than four stages or five or more stages is also possible.

With reference to FIG. 6 and FIG. 7, an example of processing at the time of AF continuous shooting in the present embodiment will be described. FIGS. 6 and 7 are flowcharts illustrating AF continuous shooting using the focus determination table.
At the start of continuous shooting, the overall control calculation unit 309 determines the aperture F value for still image shooting (S601). Next, exposure of the imaging element 306 for still image shooting starts (S602). After the exposure, an image signal of a still image is read from the image sensor 306 (S603). Next, the overall control calculation unit 309 determines the result of the focus determination according to the focus determination table of FIG. It is determined whether or not the state value A is satisfied (S604). If it corresponds to A (if it is in focus), the process proceeds to S605; otherwise, the process proceeds to S606.

  In step S605, the overall control calculation unit 309 sets the aperture F value for live view (LV) shooting to the same F value as for still image shooting, and proceeds to step S610. In step S606, the overall control calculation unit 309 determines whether the result of the focus determination corresponds to the state value B. If it corresponds to B, the process proceeds to S607; otherwise, the process proceeds to S608. In step S <b> 607, the overall control calculation unit 309 changes the aperture F value for live view shooting from the still image shooting by one stop to the open side, and proceeds to S <b> 610.

  In step S608, the overall control calculation unit 309 determines whether the result of the focus determination corresponds to the state value C. In the case of C, the process proceeds to S609, and in the case of not C, that is, in the case of D, the process proceeds to S611. In step S609, the overall control calculation unit 309 changes the aperture F value for live view shooting from the still image shooting to an open side by two stops, and proceeds to step S610.

  If the focus determination result corresponds to any one of the state values A, B, and C, in S610, a process of displaying the captured still image on the screen of the display unit 311 is performed, and then, in S612 of FIG. Proceed to. On the other hand, when the focus determination result corresponds to the state value D, in S611, the overall control calculation unit 309 opens the aperture F value for live view shooting. Then, the process proceeds to S612 in FIG.

  In step S612, the overall control calculation unit 309 starts exposure of the image sensor 306 for live view shooting. Then, the signal of the live view image is read from the image sensor 306 (S613), and the live view image is displayed on the screen of the display unit 311 (S614). The overall control calculation unit 309 performs AF control (S615). After that, the overall control calculation unit 309 determines whether or not to end the continuous shooting based on the operation instruction given by the user using the operation unit 315 (S616). If it is determined that the continuous shooting is to be continued, the process returns to S601. If it is determined in step S616 that the continuous shooting has been completed, a series of processing ends.

  In this embodiment, when the focus determination result corresponds to the state value B or C, the aperture value during live view shooting changes. Accordingly, it is necessary to control the exposure time and the ISO sensitivity in order to adjust the brightness of the image to the still image. At this time, the overall control calculation unit 309 determines which of the exposure time and the ISO sensitivity is preferentially changed according to the determination result of the edge steepness. For example, when the value of the edge steepness is low, there is a possibility that the subject blur has occurred. In this case, the overall control calculation unit 309 sets the exposure time to be short, and sets the ISO sensitivity to the same value as in the case of a still image. Conversely, when the value of the edge steepness is high, the overall control calculation unit 309 sets the exposure time to the same time as in the case of a still image, and sets the ISO sensitivity to be lower. By doing so, it is possible to increase the focus determination accuracy of the phase difference AF during live view shooting.

  FIG. 8 shows a timing chart when the still image display and the live view display are performed alternately. Each of the periods T1 to T4 indicates an exposure period. Periods S1 and S3 respectively show the still image readout period of S603 in FIG. 6, and periods S2 and S4 show the live view (LV) readout period of S613 of FIG. 7, respectively. The timing chart shows the open / closed state of the shutter. During the display period, the still image display in S610 in FIG. 6 and the live view display in S614 in FIG. 7 are performed alternately. The periods D1 and D3 are the still image display periods in S610 in FIG. 6, and the periods D2 and D4 are the live view display periods in S614 in FIG. The timing of focus determination, aperture control, and AF control are indicated by black rectangle symbols, respectively. The timings of the still image focus determination and the live view focus determination are after the end of the readout periods S1 to S4, respectively. The aperture control is performed after the focus determination of the still image, and the aperture control and the AF control are performed after the focus determination of the live view.

  After the end of the exposure period T1, a signal of a still image is obtained in a still image reading period S1. Then, the focus determination processing of the still image is performed, and when the focus determination result corresponds to any of the state values A, B, and C in FIG. 5, the still image is displayed in the period D1. Then, aperture control is performed. After the end of the exposure period T2, a signal of a live view image is read out in a live view readout period S2. Live view display is performed in the period D2. After the focus determination of the live view, the aperture control and the AF control are performed.

  FIG. 9 shows a timing chart when the still image display and the live view display are not alternately displayed. Each item is the same as in FIG. 8, but no still image is displayed. After the end of the exposure period T1, a signal is read in the still image read period S1. Then, focus determination of the still image is performed. When the focus determination result corresponds to the state value D in FIG. 5, the still image display is not performed. Then, the aperture is set to open, and after the exposure period T2 ends, a signal of a live view image is read in a live view readout period S2. Live view display is performed in the period D2. After the focus determination of the live view, the aperture control and the AF control are performed.

  In the present embodiment, aperture control during live view shooting is determined based on the focus determination table in FIG. 5 and the aperture state during still image shooting. When an image at the time of shooting a still image and an image at the time of live view shooting are alternately displayed, control for reducing the difference in exposure condition (aperture value) is performed. Thus, even when the aperture is stopped down during still image shooting during AF continuous shooting, it is possible to maintain both display quality and followability during framing.

  In the first embodiment, the control for reducing the difference between the aperture values is shown as the exposure condition. However, the exposure condition is controlled so as to reduce the difference between the exposure time and the sensitivity (ISO sensitivity) in addition to the aperture value. You may. That is, the program diagram at the time of still image shooting and the program diagram at the time of live view shooting may have the same configuration.

  When the focus is determined both at the time of still image shooting and at the time of live view shooting, weighting may be performed for each. This is because the resolution at the time of live view shooting is lower than the resolution of a still image, and thus the focus determination accuracy may be low. Therefore, the weight of the focus determination result at the time of live view shooting is set low. Is preferred.

[Second embodiment]
Next, a second embodiment of the present invention will be described. In the present embodiment, a configuration will be described in which the aperture during live view shooting is controlled in accordance with differences in the focal length and subject distance of the lens unit 301 in addition to the state of the aperture during still image shooting. In the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, detailed description thereof will be omitted, and differences will be mainly described.

  The way the image looks when the aperture value is changed greatly differs depending on the difference in the focal length of the lens unit 301. For example, when the aperture value is changed by two steps at a wide angle with a short focal length, the change in the appearance of the image is small, but when the aperture value is changed by two steps at a long focal length, the change in the image appearance is small. growing. Therefore, in the present embodiment, the following control is performed according to the focal length of the lens unit 301.

-When the focal length is on the wide angle side (for example, 50 mm or less)
When it is determined that the focus determination result in FIG. 5 is the state value B, the overall control calculation unit 309 opens the aperture value in the aperture control of the live view in S607 in FIG. Change to the side. When the in-focus determination result in FIG. 5 is determined to be the state value C, the overall control calculation unit 309 opens the aperture value in the live view aperture control in S609 in FIG. Change to the side.

In the case of the telephoto side where the focal length is long, the overall control calculation unit 309 performs the same processing as in the first embodiment by controlling the aperture of the live view.

  Next, the difference in subject distance will be described. Depending on the distance from the imaging device to the subject, the appearance of the image when the aperture value is changed greatly differs. For example, when the aperture value is changed by two steps when the subject distance is large, the change in the appearance of the image is small, but when the aperture value is changed by two steps when the subject distance is small, the change in the appearance of the image is small. growing. Therefore, in the present embodiment, the following control is performed according to the subject distance.

-When the subject distance is large (for example, 3 m or more)
When determining that the focus determination result in FIG. 5 is the state value B, the overall control calculation unit 309 sets the aperture value in the live view aperture control in S607 in FIG. Change to When the overall control calculation unit 309 determines that the focus determination result in FIG. 5 is the state value C, the aperture value in the live view aperture control in S609 in FIG. Change to the side.

When the subject distance is small The overall control calculation unit 309 performs the same processing as that of the first embodiment by controlling the aperture of the live view.

In the present embodiment, the following control is performed according to the focal length of the lens unit 301 and the subject distance.
When the focal length is longer than the threshold value and the subject distance is larger than the threshold value When the overall control calculation unit 309 determines that the focus determination result in FIG. The aperture value in the aperture control is changed to a two-step opening side compared to the case of a still image. When the overall control calculation unit 309 determines that the focus determination result in FIG. 5 is the state value C, the aperture value in the live view aperture control in S609 in FIG. Change to the side.

When the focal length is longer than the threshold value and the subject distance is equal to or less than the threshold value, the overall control calculation unit 309 performs the same processing as that of the first embodiment in the live view aperture control.

  Next, focus determination according to the mode will be described with reference to FIG. FIG. 10A illustrates a focus determination table in the first focus determination mode in which priority is given to tracking during framing. The first focus determination mode is a mode in which display updating is prioritized over AF focusing accuracy. FIG. 10B illustrates a focus determination table in the second focus determination mode in which the priority is given to the AF accuracy during framing. The second focus determination mode is a mode in which the focus accuracy of AF is given priority over display update. The overall control calculation unit 309 switches the focus determination table for each mode. The reliability levels and the state values A to D are as described in FIG.

  In the case of the first focus determination mode shown in FIG. 10A, in order to alternately display a still image and a live view image as much as possible, the state value D is compared with the focus determination table in FIG. The number is low. That is, the case where the correlation calculation result is “large” and the steepness of the edge is “medium”, and the case where the correlation calculation result is “medium” and the steepness of the edge is “low”. Is determined as the state value C. Therefore, the frequency at which the still image and the live view image are alternately displayed becomes relatively high.

  In the case of the second focus determination mode shown in FIG. 10B, in order to give priority to AF accuracy, the number of state values D is larger in the focus determination table than in FIG. That is, when the correlation calculation result is “large” and the steepness of the edge is “high”, when the correlation calculation result and the steepness of the edge are “medium”, and when the correlation calculation result is “small”. , And the steepness of the edge is “low”, it is determined to be the state value D. Therefore, the frequency at which the live view image is displayed becomes relatively high.

  In the present embodiment, the exposure state of the image sensor 306 is changed by aperture control according to the focal length of the lens unit 301 and the subject distance. Thus, even when the aperture is stopped down during still image shooting during AF continuous shooting, it is possible to maintain both display quality and follow-up during framing in accordance with shooting conditions and shooting modes.

[Third embodiment]
FIG. 11 is a block diagram showing a configuration of an image pickup apparatus having a form of a camera system including a camera main body and a photographing lens which can exchange a plurality of photographing lenses according to a third embodiment of the present invention. The camera system (imaging apparatus) of the present embodiment is a single-lens reflex camera with interchangeable lenses, and includes a lens unit 1100 including a taking lens and a camera body 1120. The lens unit 1100 is mounted on the camera body 1120 via a mount M indicated by a dotted line in the center of the figure.

  The lens unit 1100 has an imaging optical system (first lens group 1101, aperture 1102, second lens group 1103, third lens group (focus lens 1104)), and a drive / control system. The first lens group 1101 is arranged at the tip of the lens unit 1100, and is held movably in the optical axis direction OA. The aperture 1102 has a function as a mechanical shutter for controlling the exposure time when photographing a still image, in addition to a function of adjusting the amount of light during photographing. The diaphragm 1102 and the second lens group 1103 are integrally movable in the optical axis direction OA, and realize a zoom function by moving in conjunction with the first lens group 1101. The focus lens 1104 is also movable in the optical axis direction OA, and the subject distance (focusing distance) at which the lens unit 1100 focuses changes according to the position. By controlling the position of the focus lens 1104 in the optical axis direction OA, focus adjustment for adjusting the focal distance of the lens unit 1100 can be performed.

  The drive / control system includes a zoom actuator 1111, an aperture actuator 1112, a focus actuator 1113, a zoom drive circuit 1114, an aperture drive circuit 1115, a focus drive circuit 1116, a lens MPU 1117 (processor), and a lens memory 1118.

  The zoom drive circuit 1114 drives the first lens group 1101 and the second lens group 1103 in the optical axis direction OA using the zoom actuator 1111 and controls the angle of view of the optical system of the lens unit 1100. An aperture driving circuit 1115 drives the aperture 1102 using an aperture actuator 1112, and controls the opening diameter and opening / closing operation of the aperture 1102. The focus drive circuit 1116 drives the focus lens 1104 in the optical axis direction OA using the focus actuator 1113, and controls the focusing distance of the optical system of the lens unit 1100. The focus drive circuit 1116 detects the current position of the focus lens 1104 using the focus actuator 1113.

  The lens MPU 1117 performs overall calculations and controls related to the lens unit 1100, and controls the zoom drive circuit 1114, the aperture drive circuit 1115, and the focus drive circuit 1116. The lens MPU 1117 is connected to the camera MPU 1125 via the mount M, and communicates commands and data. For example, the lens MPU 1117 detects the position of the focus lens 1104, and notifies lens position information in response to a request from the camera MPU 1125. The lens position information includes the position of the focus lens 1104 in the optical axis direction OA, the position and diameter of the exit pupil in the optical axis direction OA when the optical system is not moving, and the optical axis of the lens frame that restricts the light flux of the exit pupil. Includes information such as position and diameter in direction OA. The lens MPU 1117 controls the zoom drive circuit 1114, the aperture drive circuit 1115, and the focus drive circuit 1116 in response to a request from the camera MPU 1125. Optical information necessary for the imaging surface phase difference AF is stored in the lens memory 1118 in advance. The lens MPU 1117 controls the operation of the lens unit 1100, for example, by executing a program stored in a built-in nonvolatile memory or the lens memory 1118.

  The camera body 1120 has an optical system (optical low-pass filter 1121 and image sensor 1122) and a drive / control system. The optical low-pass filter 1121 reduces false colors and moire of a captured image. The image sensor 1122 is composed of a CMOS image sensor and peripheral circuits, and has m pixels in the horizontal direction and n pixels in the vertical direction (n and m are integers of 2 or more). The image sensor 1122 of the present embodiment has a pupil division function, and is capable of performing an imaging plane phase difference AF using image data having parallax.

  The drive / control system includes an imaging element driving circuit 1123, an image processing circuit 1124, a camera MPU 1125 (processor), a display unit 1126, an operation switch group 1127, a memory 1128, and an imaging plane phase difference focus detection unit 1129. The image processing circuit 1124 generates a focus detection signal and image data for display / recording from the image data output from the image sensor 1122.

  The image sensor driving circuit 1123 controls the operation of the image sensor 1122, A / D converts the acquired image data, and transmits the data to the camera MPU 1125. The image processing circuit 1124 performs general image processing performed by a digital camera, such as γ conversion, white balance adjustment processing, color interpolation processing, and compression encoding processing, on the image data acquired by the image sensor 1122.

  The camera MPU 1125 performs overall calculations and controls related to the camera body 1120, and controls the image sensor driving circuit 1123, the image processing circuit 1124, the display unit 1126, the operation switch group 1127, the memory 1128, and the imaging plane phase difference focus detection unit 1129. Control. The camera MPU 1125 is connected to the lens MPU 1117 via the mount M, and communicates commands and data with the lens MPU 1117. The camera MPU 1125 issues, to the lens MPU 1117, a request for obtaining a lens position, a request for driving an aperture, a focus lens, and a zoom with a predetermined drive amount, a request for obtaining optical information unique to the lens unit 1100, and the like. The camera MPU 1125 includes a ROM 1125a storing a program for controlling the operation of the camera body 1120, a RAM 1125b storing variables, and an EEPROM 1125c storing various parameters.

  The display unit 1126 is configured with an LCD or the like, and displays information on a shooting mode of the camera body 1120, a preview image before shooting, a confirmation image after shooting, an in-focus state image at the time of focus detection, and the like. The operation switch group 1127 includes a power switch, a release (imaging trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. The memory 1128 is a detachable flash memory and records a photographed image.

  The imaging plane phase difference focus detection unit 1129 performs focus detection processing by the imaging plane phase difference AF based on the imaging plane phase difference AF signal obtained by the image processing circuit 1124. Specifically, the image processing circuit 1124 generates a pair of image data formed by a light beam passing through a pair of pupil regions of the imaging optical system as an imaging plane phase difference AF signal, and the imaging plane phase difference focus detection unit 1129 generates The amount of defocus (the amount of defocus) is detected based on the amount of deviation between the pair of image data. As described above, the imaging plane phase difference focus detection unit 1129 of the present embodiment performs the imaging plane phase difference AF based on the output of the imaging element 1122 without using a dedicated AF sensor.

  The operation of the imaging plane phase difference focus detection unit 1129 will be described in detail with reference to FIG. FIG. 12 is a diagram illustrating a pixel array of the image sensor 1122 according to the present embodiment. The state observed from the 1100 side is shown. The image sensor 1122 is provided with a Bayer array color filter. In the Bayer array according to the present embodiment, for example, green (G) and red (R) color filters are alternately arranged in the X direction in order from the left for pixels in the odd rows, and pixels for the even rows are arranged in the X direction from the left. In this order, blue (B) and green (G) color filters are alternately arranged in the X direction. In the pixel 1211, a circle indicated by a reference numeral 1211i represents an on-chip microlens, and a plurality of rectangles (reference numerals 1211a and 1211b) arranged inside the on-chip microlens 1211i each represent a photoelectric conversion unit.

  The image sensor 1122 of this embodiment has first and second focus detection pixels that receive light beams that pass through different pupil partial regions of the imaging optical system. In this embodiment, the photoelectric conversion units of all the pixels in the image sensor 1122 are divided into two in the X direction, and the photoelectric conversion signal of one of the divided areas and the sum of the two photoelectric conversion signals are independent. And read it out. However, the configuration may be divided into a plurality of parts in the X direction and the Y direction. Then, regarding the signal read out independently, by taking the difference between the sum of the two photoelectric conversion signals and the photoelectric conversion signal in one of the divided areas, the signal corresponds to the signal obtained in the other photoelectric conversion area. A signal can be obtained. The photoelectric conversion signals of these divided areas are used for phase difference focus detection by a method described later, and a 3D (3-Dimensional) image composed of a plurality of images having parallax information can also be generated. . On the other hand, the sum of the two photoelectric conversion signals is used as a normal captured image.

  Here, the imaging plane phase difference AF signal will be described. In the present embodiment, the on-chip micro lens 1211i of FIG. 12 and the divided photoelectric conversion units 1211a and 1211b pupil-divide the emitted light beam of the imaging optical system. Then, for a plurality of pixels 1211 within a predetermined range arranged in the same pixel row, an image formed by connecting the outputs of the photoelectric conversion units 1211a to form an A image for the imaging surface phase difference AF (A image for AF). . A knitted image obtained by connecting the outputs of the photoelectric conversion units 1211b is referred to as a B image for the imaging surface phase difference AF (B image for AF). As the outputs of the photoelectric conversion units 1211a and 1211b, pseudo luminance (Y) signals calculated by adding the green, red, blue, and green outputs included in the unit array of the color filters are used. However, the A image for AF and the B image for AF may be formed for each of the colors red, blue, and green. By detecting the relative image shift amount between the AF A image and the AF B image generated by the correlation calculation, it is possible to detect the defocus amount (defocus amount) in a predetermined area. In this embodiment, the sum of the output of one photoelectric conversion unit and the output of all photoelectric conversion units is read from the image sensor 1122. For example, when the output of the photoelectric conversion unit 1211a and the sum of the outputs of the photoelectric conversion units 1211a and 1211b are read, the output of the photoelectric conversion unit 1211b is obtained by subtracting the output of the photoelectric conversion unit 1211a from the sum. Thereby, both the A image for AF and the B image for AF can be obtained, and the imaging surface phase difference AF can be realized. Such an image sensor is known as disclosed in, for example, Japanese Patent Application Laid-Open No. 2004-134867, and therefore, further detailed description will be omitted.

  Here, the configuration in which the exit pupil is divided into two in the horizontal direction has been described as an example, but the configuration may be such that the exit pupil is divided into two in the vertical direction for some pixels in the image sensor 1122. Further, the exit pupil may be divided in both the horizontal and vertical directions. By providing a pixel that divides the exit pupil in the vertical direction, it is possible to perform the imaging surface phase difference AF corresponding to the contrast of the subject not only in the horizontal direction but also in the vertical direction.

  As described above, the image sensor 1122 has not only a function of imaging but also a function as a focus detection device. In addition, as a focus detection method, since a focus detection pixel that receives a light beam obtained by dividing the exit pupil is provided, it is possible to perform the imaging surface phase difference AF.

  FIG. 13 is a flowchart illustrating focus detection processing, imaging processing, and display processing performed by the camera body 1120 according to the third embodiment of the present invention. In this flowchart, the processing of each step is realized by the camera MPU 1125 controlling each unit of the camera main body 1120 and communicating with the lens MPU 1117 according to a program stored in the ROM 1125a, unless otherwise specified. In the following description, an operation will be described in which the imaging apparatus performs LV imaging in which imaging is performed from a live view (LV) state in which image data for display is sequentially displayed on the display unit 1126.

  In step S701, the camera MPU 1125 controls the driving of the image sensor driving circuit 1123 and the image sensor 1122, and acquires image data by imaging the subject. Here, since the driving operation is for photographing a moving image for LV display, photographing using a so-called electronic shutter is performed in which charge accumulation and readout are performed for a time corresponding to the frame rate for LV display. The main purpose of the LV display is to allow the photographer (user) to confirm the imaging range and the imaging conditions, for example, 30 frames / second (imaging interval of 33.3 ms) and 60 frames / second (imaging interval of 16 frames). .6 ms). In the process of S701, as an initial value setting operation, an after-mentioned alternate display flag is initialized to 0.

  Next, in step S702, the camera MPU 1125 obtains focus detection data obtained from the first focus detection pixel and the second focus detection pixel included in the focus detection area of the image sensor out of the imaging data obtained in step S701. Further, the camera MPU 1125 generates an imaging signal, and acquires image data obtained by applying a color interpolation process or the like in the image processing circuit 1124. In this way, image data (image signal) and focus detection data (focus detection signal) can be acquired by one image capturing. When the image pickup pixel, the first focus detection pixel, and the second focus detection pixel are configured as individual pixels, image data is obtained by performing a process of complementing the focus detection pixels and the like. Note that the camera MPU 1125 performs flicker detection for detecting whether or not flicker has occurred under set shooting conditions, using the image signal (second image signal) obtained in the process of S702. In the present embodiment, a flicker detection result executed when the power of the camera body 1120 is turned on or in response to a manual operation by a user is recorded in the memory 1128. Is determined. Note that any known method of flicker detection may be employed. For example, in the present embodiment, flicker detection is performed by detecting a flicker light amount change cycle and a position of a feature point (see, for example, Japanese Patent Application Laid-Open No. 2015-220673), but is not limited thereto. . For example, flicker detection is performed by extracting a brightness difference occurring over the entire screen corresponding to the image signal, and comparing the extracted information on the brightness difference with flicker information stored in advance in the camera body 1120. Any configuration may be used as long as it is executed.

  Next, in S800, the camera MPU 1125 controls the image processing circuit 1124 to generate an image for LV display (LV image) based on the image signal (second image signal) obtained in the process of S702. , On the display unit 1126. The image for LV display is, for example, a reduced image corresponding to the resolution of the display unit 1126, and the image processing circuit 1124 performs a reduction process when generating image data in the process of S702. Then, the camera MPU 1125 causes the display unit 1126 to display the LV image (second image) after the reduction processing. As described above, during LV display, imaging of a subject and sequential display are performed at a predetermined frame rate, so that the user can adjust the composition at the time of shooting, exposure conditions, and the like through the display unit 1126.

  In step S800, the camera MPU 1125 generates an LV image (second image) from the image signal (second image signal) acquired in step S702, and generates a still image for alternate display (third image) described later. ) And different processes are executed depending on whether or not to perform the alternate display. When performing the alternate display, a process is performed to reduce the difference between the second image and the third image to generate a second image signal. Accordingly, when performing display using the third image and the second image acquired under different exposure conditions and signal readout conditions, it is possible to perform display with less discomfort. Details of the processing in S800 will be described later. In the following description, a still image for recording (first image) and a still image for alternate display (third image) are generated based on a first image signal described later, and a second image signal is generated. , An LV image (second image) is generated.

  Next, in step S703, the camera MPU 1125 obtains a defocus amount and a direction for each focus detection area using the focus detection data corresponding to the focus detection area acquired in step S702. In the present embodiment, the camera MPU 1125 performs processing for generating an image signal for focus detection, calculating a shift amount (phase difference) of the focus detection signal, and obtaining a defocus amount and a direction from the calculated shift amount. When generating an image signal for focus detection, the camera MPU 1125 performs processing (eg, gain adjustment, low-pass processing, etc.) different from signal processing performed when generating the second image signal.

  The second image signal, which is the original data of the second image used for display, is required to be able to confirm the composition, color, brightness, blur state, and the like of the imaging range on the display unit 1126. , Some signals may be saturated. On the other hand, when the image signal for focus detection does not include a saturation signal, highly accurate focus detection can be performed. In addition, since the second image is subjected to image processing corresponding to a still image used for alternate display, which will be described later, sufficient contrast cannot be secured to perform focus detection, or a signal of a high-frequency component cannot be obtained by low-pass processing or the like. May not be available. Therefore, in the process of S703, the camera MPU 1125 performs a process different from the process of generating the second image on the second image signal, thereby enabling acquisition of a signal of a contrast or a high-frequency component, which is suitable for focus detection. Then, an image signal for focus detection is generated.

  Next, in step S704, the camera MPU 1125 detects ON / OFF of SW1, which is a switch indicating the start of shooting preparation. A release (imaging trigger) switch, which is one of the operation switches 1127, can detect two stages of on / off in accordance with the amount of depression of the release switch. The ON / OFF of SW1 described above corresponds to the first ON / OFF of a release (imaging trigger) switch, and indicates the start of shooting preparation.

  Next, in step S704, if the switch SW1 is not detected to be on (or the switch is detected to be off), the camera MPU 1125 advances the process to step S709, and determines whether the main switch included in the operation switch group 1127 has been turned off. I do. On the other hand, if SW1 is detected to be on in S704, the camera MPU 1125 advances the process to S705, and sets (selects) a focus detection area to be focused. Here, the focus detection area designated by the user may be set, or information such as the information on the defocus amounts of the plurality of focus detection areas obtained in the processing of S704 and the distance from the center of the imaging range of the focus detection area. May be automatically set by the camera MPU 1125 based on the. In general, a subject that the user intends to photograph has a high probability of being located at a position where the photographing distance is short, and has a high probability of being located near the center of the imaging range. For example, when there are a plurality of subjects, the camera MPU 1125 automatically sets a focus detection area that is considered appropriate in consideration of these conditions. In addition, for example, when it is possible to determine a subject from which a main subject can be determined and a background, the camera MPU 1125 sets an area where the main subject exists as a focus area.

  Next, in step S706, the camera MPU 1125 drives the lens based on the defocus amount detected in the selected focus detection area. When the detected defocus amount is smaller than the predetermined threshold, if the lens drive is not always performed, it is possible to suppress the lens drive that is not required frequently.

  Next, in step S707, the camera MPU 1125 detects ON / OFF of SW2, which is a switch indicating an imaging start instruction. As described above, the release (imaging trigger) switch can detect two stages of on / off in accordance with the amount of depression. The above-described SW2 corresponds to the second-stage ON / OFF of the release (imaging trigger) switch, and indicates the start of imaging. If the switch M2 is not detected in step S707, the process returns to step S702, and the camera MPU 1125 repeats the above-described LV display and focus detection.

  If SW2 is detected to be on in S707, the process proceeds to S900, and the camera MPU 1125 executes a shooting subroutine. Details of the shooting subroutine will be described later. When the shooting subroutine is executed in S900, the process proceeds to S1000.

  Next, in S1000, display determination of the third image (still image for alternate display) is performed, and then, in S708, it is determined whether to display the third image on the display unit 1126. The third image is a still image for alternate display corresponding to the still image for recording (first image) obtained in S900.

  Next, if it is determined in step S708 that the third image is to be displayed on the display unit 1126, the process advances to step S1100, where the camera MPU 1125 generates a third image and causes the display unit 1126 to display the third image. . On the other hand, if it is determined in S708 that the third image is not to be displayed, the process returns to S702 without generating the third image. Details of the third image display determination performed in the process of S1000 will be described later.

  In S1100, the camera MPU 1125 generates a still image for alternate display (third image) based on the still image for recording (first image) and the LV image (second image) obtained in S900. I do. As described above, since the LV image is generated in advance so as to reduce the difference from the still image for alternate display, when generating the still image (third image) for alternate display, the minimum required Is performed. It should be noted that a configuration may be adopted in which priority is given to facilitation of the imaging operation, and the alternately displayed still image (third image) is generated in association with the LV image (second image). Details of the processing performed in S1100 will be described later.

  Upon completion of the process in S1100, the process proceeds to S709. In step S709, the camera MPU 1125 ends a series of shooting operations (including focus detection and display control) when the main switch is detected to be off. On the other hand, if it is determined in step S709 that the main switch has not been turned off, the process returns to step S702 to repeat the above-described various processes. For example, if the main switch is not turned off in the processing of S709, SW1 is detected in the processing of S704, and SW2 is continuously detected in the processing of S707, a series of photographing operations is repeatedly performed. Photographing (continuous shooting) is performed. The camera body 1120 according to the present embodiment acquires a recording still image (first image) a plurality of times at predetermined intervals when performing continuous shooting regardless of the presence or absence of the LV display. Then, when continuous shooting is instructed during LV display, at least the LV image can be displayed on the display unit 1126 while a still image for recording is being acquired (that is, during continuous shooting).

  Next, the generation and display processing of the LV image (second image) shown in S800 will be described in detail with reference to FIG. FIG. 14 is a flowchart illustrating a process of generating and displaying an LV image according to the third embodiment of the present invention. Note that the flowchart shown in FIG. 14 illustrates processing executed in the second and subsequent imaging operations in continuous imaging, and the second and subsequent imaging processing will be described as the n-th (n is a natural number). In the following description, the number of times of the corresponding imaging operation (for example, the n-th time) is shown in parentheses.

  First, in step S801, the camera MPU 1125 sets the shooting conditions for obtaining the image data (n) obtained in step S702 and the third image (n-1) that is a still image for alternate display obtained in step S900. Get shooting conditions.

  Here, as the photographing conditions, exposure conditions regarding exposure parameters such as an exposure time (charge accumulation time), an aperture value (F value) related to an aperture diameter of the aperture, and gain information applied to a signal by ISO sensitivity adjustment and the like are applicable. . The shooting conditions include subject blur information output from a gyro sensor (not shown) and blur information obtained from contrast information of a main subject portion of an image.

  Next, in S802, the camera MPU 1125 determines whether or not the alternate display flag = 1 indicating information on whether or not to alternately display the second image and the third image based on the information acquired in the processing in S801. Is determined. The setting of the alternate display flag will be described later with reference to FIG.

  If it is determined in step S802 that the alternate display flag is 1, the process proceeds to step S803, where the camera MPU 1125 performs editing settings for the second image (n), which is an LV image used for alternate display. Specifically, when the alternate display is performed, the difference between the third image (n−1) and the second image (n) acquired (generated) immediately before using the imaging conditions obtained in S801. The editing setting of the LV image (the second image (n)) is performed so as to reduce the size. As this editing setting, gain adjustment, sharpness (blurring) adjustment, low-pass processing (spatial frequency filtering), and the like are performed on the image data acquired in the processing of S702. With this configuration, it is possible to reduce a sense of discomfort given to the user between the images displayed alternately.

  Here, the gain adjustment in S803 will be specifically described. When the gain and the exposure time are different between the third image (n−1) and the image data (n) acquired in the processing of S702, the camera MPU 1125 converts the LV image (n) into the third image (n). The editing setting is such that the gain is adjusted according to -1). For example, it is assumed that the exposure time for the third image (n-1) is 1/30 second and the gain is twice. In this case, if the exposure time for the image data (n) acquired in the process of S702 is 1/15 second and the gain is four times, the second image (n) as the LV image is acquired in the process of S702. The editing setting is such that the gain becomes 1/4 times that of the image data (n).

  Note that when the gain of the first image (still image for recording) and the gain of the second image (LV image) are different due to the setting of the exposure simulation described later, the third image (still image for display) is obtained. As the editing setting, a gain adjustment amount according to the second image may be set.

  Regarding the sharpness processing and the low-pass processing, the focus state and the blur state of the third image (n-1) and the image data (n) acquired in the processing of S702 may vary depending on the acquisition period even if the exposure conditions are adjusted. It is difficult to match both images. Therefore, in the present embodiment, the blur state near the main subject is analyzed to reduce the difference between the third image (n-1) and the second image (n). For example, if it can be determined that the third image (n-1) is more blurred than the image (n) acquired in S702, the third image (n) is subjected to low-pass processing and the second image The blur state of (n) is aligned with the third image (n-1). Therefore, in the processing of S803, the filter setting of the low-pass processing is performed. The strength of the low-pass processing or the sharpness processing may be set by acquiring contrast information or spatial frequency component information near the main subject. Alternatively, the setting may be made using camera shake information at the time of exposure. Note that the processing executed in S803 may be only one of image processing and gain adjustment.

  As described above, in the present embodiment, when the third image and the second image are alternately displayed, the second image displayed next to the third image is added to the third image. This is a configuration that is generated so that the difference is small. With this configuration, when both the LV image and the still image are alternately displayed, it is possible to reduce a sense of discomfort given to the user due to a difference between the two images.

  On the other hand, if the alternate display flag is not 1 in S802 (that is, the alternate display flag is 0), the process proceeds to S804, and the camera MPU 1125 performs editing settings for displaying only the second image on the display unit 1126. In this case, since there is no need to reduce the difference between the third image (n-1) and the second image (n), the editing settings may be set so as to provide appropriate brightness and contrast for display.

  When various editing settings are completed in the processing of S803 and S804, the camera MPU 1125 applies the set editing settings to the image (n) acquired in the processing of S702 in S805 and performs the n-th imaging operation. A corresponding LV image (second image) is generated. When the generation of the second image (n) is completed in S805, the process proceeds to S806, where the camera MPU 1125 adjusts the gain between the third image (n-1) and the second image (n), It is determined whether there is a difference that cannot be adjusted even when the sharpness processing and the low-pass processing are performed. For example, when the F value is greatly different or the focus state is significantly different between the two images described above, the difference between the two may not be fully adjusted by the sharpness processing or the low-pass processing.

  If it is determined in step S806 that there is an unadjustable difference, in step S807, the camera MPU 1125 combines the third image (n-1) and the second image (n) and newly creates a second image. (N) is generated. The combination of the two images uses, for example, an average value of the two images (for example, an output value or a luminance value of an image signal). Note that the same processing is performed when a third image (n) to be described later is generated. Therefore, moving average processing is performed on the two images in time series. Thereby, the difference between the two images can be reduced. On the other hand, if the averaging process of the two images is performed while the movement of the subject or the panning operation is performed, an unnatural display in which the moving portions in the images are shifted and overlapped with each other. In such a case, the synthesizing process may not be performed.

  Next, in step S808, the camera MPU 1125 displays the second image (n) on the display unit 1126. Note that each of the above-described processes may be executed using an image signal before being converted into image data, instead of being applied to each image output as image data. For example, in the processing of S807, the image signal corresponding to the third image (n-1) and the image signal corresponding to the second image (n) may be combined.

  Next, the details of the imaging subroutine process shown in S900 will be described with reference to FIG. FIG. 15 is a flowchart illustrating an imaging subroutine process according to the third embodiment of the present invention. For example, in the imaging subroutine performed after the processing related to the generation and display of the LV image related to the n-th imaging operation in the flowchart illustrated in FIG. 14, a recording still image (first image) corresponding to the n-th imaging operation Is generated.

  In step S901, the camera MPU 1125 starts exposure control processing and sets shooting conditions (exposure time, F value, gain (photographing sensitivity)). This exposure control processing can be performed by the camera MPU 1125 based on the luminance information of the image data, and any known technique may be used as the method. In the present embodiment, for example, in the n-th imaging operation, the above-described luminance information is acquired based on the still image for recording acquired in the (n-1) -th imaging operation. Alternatively, the configuration may be such that luminance information is obtained based on the LV image obtained in the n-th imaging operation.

  In step S901, the camera MPU 1125 controls the aperture actuator 1112 based on the set F value and the exposure time, and controls the operation of the shutter 1102 that also functions as the aperture. Further, the camera MPU 1125 controls the driving of the image sensor driving circuit 1123 based on the set exposure time to execute charge accumulation.

  When the charge accumulation is completed, in step S902, the camera MPU 1125 controls the operation of the image sensor driving circuit 1123 to acquire the image data (first image signal) corresponding to the still image by reading out the accumulated charge. . Then, the camera MPU 1125 acquires a recording still image (first image) based on the first image signal. Note that reading of the first image signal is so-called reading of all pixels, but a reading line of the image sensor 1122 may be set so that high-pixel image data can be obtained as a still image for recording. Further, the camera body 1120 of the present embodiment may generate a still image for display (third image) based on a still image for recording (first image), or generate a still image for recording. It may be generated based on the first image signal simultaneously with the image. In this case, a still image for recording and a still image for display are respectively generated based on the first image signal. Note that the LV image and the still image used in the present embodiment are different types of images in which the number of pixels of the image sensor that performs reading is different as described above.

  Next, in step S903, the camera MPU 1125 performs a defective pixel correction process on the read image data via the image processing circuit 1124. In step S904, the camera MPU 1125 performs demosaic (color interpolation) processing, white balance processing, γ correction (gradation correction) processing, color conversion processing, and edge enhancement processing on the image data after the defective pixel correction via the image processing circuit 1124. And other image processing, encoding processing, and the like. In step S <b> 905, the camera MPU 1125 records a data file corresponding to a recording still image (first image) in the memory 1128.

  Next, in S906, the camera MPU 1125 records the characteristic information of the camera body corresponding to the still image for recording recorded in the processing of S905 in the memory 1128 and the memory provided in the camera MPU 1125. Here, the characteristic information of the camera body includes exposure conditions, image processing information, information on the light receiving sensitivity distribution of the imaging pixels and focus detection pixels of the image sensor 1122, vignetting information of the imaging light flux, information on flange back, and manufacturing error information. And so on. Since the light receiving sensitivity distribution information of the image sensor 1122 is information dependent on the on-chip microlens ML and the photoelectric conversion unit PD, information on these members may be recorded.

  Next, in step S907, the camera MPU 1125 records the characteristic information of the lens unit corresponding to the recording still image recorded in the processing in step S905 in the memory 1128 and the memory provided in the camera MPU 1125. Here, the characteristic information of the lens unit includes information on the exit pupil, information on the focal length and F-number at the time of imaging, aberration information on the optical system, manufacturing error information, and the like.

  Next, in step S908, the camera MPU 1125 records the image-related information relating to the previously acquired still image for recording in the memory 1128 and the memory provided in the camera MPU 1125. Examples of the image-related information include information on a focus detection operation before imaging, subject movement information, information on the accuracy of the focus detection operation, and the like.

  Next, details of the third image display determination processing shown in S1000 will be described with reference to FIG. FIG. 16 is a flowchart illustrating a display determination process of a still image for display according to the third embodiment of the present invention. For example, in the third image display determination process performed after the generation and display process of the recording still image (first image) related to the n-th imaging operation in the previous imaging subroutine, the n-th imaging operation Are generated (third image (n)) for display corresponding to.

  In step S1001, the camera MPU 1125 determines whether the alternate display setting has been turned off by the user. As the alternate display setting, for example, it is possible to turn on / off the alternate display setting displayed on the menu screen by manually operating the operation member by the user. If the alternate display setting is off, the process proceeds to S1009, where the camera MPU 1125 sets the alternate display flag = 0 as information indicating that alternate display is not to be performed.

  Next, if the alternate display is set to ON in step S1001, the process proceeds to step S1002, where the camera MPU 1125 directs a light-emitting device (not shown) toward a subject when capturing a still image (first image) for recording. It is determined whether light emission (illumination) is performed. When the subject is illuminated by the light emitting device, the subject is illuminated in the recording still image (first image), but is not illuminated in the LV image (second image). There is a large difference in brightness between the two images. Therefore, in this case, the alternate display is not performed in order to prevent the user from feeling uncomfortable due to the alternate display of images having significantly different brightness. If the amount of light emitted from the light emitting device is small, the difference in brightness between the two images can be reduced by adjusting the gain or the like. In such a case, the display may be alternately displayed.

  Next, in step S1003, the camera MPU 1125 determines whether or not each exposure condition has a difference equal to or more than a predetermined value between the first image (n) and the second image (n). As described above, the difference between the exposure conditions can be reduced by adjusting the gain or combining the two images. However, when the difference between the two images is large, it is difficult to reduce the difference. For example, it is assumed that the first image is acquired under the condition that the aperture opening is made small (small aperture) in long-time exposure using a so-called slow shutter. In this case, the second image and the first image may be acquired under the same exposure condition. However, for the LV image, there is a problem that the display update time on the display unit 1126 is prolonged and the focus detection accuracy is deteriorated. Become. Therefore, in the process of S1003, when it is determined that there is a difference between the exposure conditions equal to or more than the predetermined value, the setting is made such that the alternate display is not performed.

  Next, in step S1004, the camera MPU 1125 determines whether or not to emit auxiliary light for assisting the focus detection process when acquiring the LV image (second image). Generally, the auxiliary light is light projection for assisting focus detection, and the illumination range by the light projection is narrower than the imaging range, or the color is biased only in a specific range. Is not suitable. Therefore, when the above-described auxiliary light is emitted when the LV image is obtained, the difference in brightness between the still image and the LV image is large, and it is difficult to compensate for the difference. Therefore, in the n-th imaging operation, when the auxiliary light for assisting the focus detection is emitted at the time of acquiring the LV image (the second image), the setting is made such that the alternate display is not performed.

  Next, in S1005, the camera MPU 1125 sets the ISO sensitivity (gain) of the recording still image (first image (n)) and the Lv image (second image (n)) out of the predetermined range. It is determined whether or not the value corresponds to. As described above, there is a difference in the signal value reading method between the still image and the LV image. For example, an LV image is read out from a still image by adding and compressing signals of the same color in the horizontal direction of the image sensor by so-called thinning-out reading, so that the hue and the like may be different between the two images. In this case, a difference (for example, a difference in noise amount) between the still image and the Lv image greatly differs depending on the set value of the ISO sensitivity. Therefore, the camera body 1120 of the present embodiment provides a range (threshold) in which the alternate display can be performed, and when the ISO sensitivity of the still image and the LV image is included in the range, the difference between the two images is small. Judge and perform alternate display. On the other hand, if any one of the ISO sensitivities of the still image and the LV image falls outside the range, the difference between the two images is determined to be large, and the alternate display is not performed.

  In the present embodiment, a configuration has been described in which the ISO sensitivities of the still image (for recording) and the LV image are compared with a predetermined range (threshold) related to the ISO sensitivities, but the present invention is not limited to this. . For example, it is determined whether or not the difference (absolute value) of the ISO sensitivity between the continuously acquired LV image (n) and the recording still image (n) is equal to or greater than a predetermined threshold, and the ISO sensitivity of both images is determined. When the difference is smaller than a predetermined threshold, the display may be alternately displayed.

Next, in step S1006, the camera MPU 1125 determines whether the deviation amount (phase difference) calculated in step S703 is a value outside a predetermined range. When it is determined that the deviation amount falls outside the predetermined range, the deviation amount is determined to be large, and the alternate display is not performed.

  Next, in step S1007, the camera MPU 1125 determines whether flicker is detected in step S702. If there is flicker, shooting is performed under an exposure condition in which flicker does not occur. Therefore, it is determined that there is an exposure condition difference equal to or greater than a predetermined value, and alternate display is not performed. In addition, when the shooting mode for reducing the influence of flicker is set, the alternate display flag may be set to 0 and the alternate display may not be performed.

  When each of the determinations in S1001 to S1007 described above is a NO determination, in S1008, the camera MPU 1125 sets a flag indicating that the alternate display is to be performed (alternate display flag = 1) as the alternate display flag. Note that a configuration may be adopted in which only one of the above-described determinations in S1001 to S1007 is executed or excluded, and the presence or absence of alternate display is determined based on the determination result. For example, the configuration may be such that the alternate display is executed when the alternate display setting is ON regardless of other conditions.

  Next, details of the generation and display processing of the display still image (third image) shown in S1100 described above will be described with reference to FIG. FIG. 17 is a flowchart illustrating a process of generating and displaying a still image for display according to the third embodiment of the present invention. Note that the flowchart illustrated in FIG. 17 is also described assuming the n-th (n is a natural number) imaging operation in the second and subsequent imaging operations in continuous imaging.

  First, in step S1101, the camera MPU 1125 captures a capturing condition for acquiring the image data (n) acquired in step S702 and captures a third image (n-1), which is a still image for alternate display, acquired in step S900. Get conditions. This processing is the same as in S801 described above.

  In step S1102, the camera MPU 1125 performs editing settings for the third image used for alternate display. Specifically, in the case of performing the alternate display, the display conditions are set such that the difference between the third image (n-1) and the second image (n) is reduced using the imaging conditions obtained in S1101. Edit setting of the still image (third image). In the processing of S803 described above, the point that the difference between the display still image (n-1) and the LV image (n) is reduced to suppress a sense of discomfort given to the user has been described. On the other hand, in the process of S1102, in order to reduce the difference between the LV image (n) and the still image (n-1) for display and to suppress the sense of discomfort given to the user, the display is performed in accordance with the LV image (n). To obtain a still image (n). With this configuration, even when the still image for display (n-1), the LV image (n), and the still image for display (n) are alternately displayed in this order, the difference between the images to be displayed continuously is determined. It is possible to suppress the discomfort given to the user by reducing the amount. Note that the editing settings in the processing of S1102 are substantially the same as the processing in S803 described above, and thus description thereof will be omitted.

  Next, in S1103, the camera MPU 1125 applies a third image (still image for display (n)) to the first image (still image (n) for recording) based on the editing settings determined in S1102. ). Then, in step S1104, the camera MPU 1125 determines whether there is a difference between the second image (n) and the third image (n) that cannot be adjusted even if the gain adjustment, the sharpness processing, and the low-pass processing are performed. Determine whether or not. This determination is substantially the same as S806 described above, and a description thereof will be omitted.

  If it is determined in S1104 that there is an unadjustable difference, the camera MPU 1125 combines the second image (LV image (n)) and the third image (still image (n) for display) in S1105. Then, a third image (n) is newly generated. The method of synthesizing the two images is substantially the same as that in S807 described above, and a description thereof will not be repeated.

  Next, in step S1106, the camera MPU 1125 displays the third image (still image (n) for display) on the display unit 1126. Note that, as described above, each process illustrated in FIG. 17 is not performed on each image output as image data, but is performed using an image signal before being converted into image data. Is also good.

  Next, the above-described alternate display operation of the LV image and the still image will be described with reference to FIGS. FIG. 18 is a diagram exemplarily illustrating a timing chart in a case where an alternating display is performed and an LV image and a still image are displayed immediately after image generation. FIG. 19 is a diagram exemplarily illustrating a timing chart according to the first method in a case where an LV image and a still image are alternately displayed in consideration of a predetermined display delay time in the case of performing the alternate display. . FIG. 20 is a diagram exemplarily illustrating a timing chart according to a second method in a case where an LV image and a still image are alternately displayed in consideration of a predetermined display delay time in the case of performing the alternate display. It is. It is assumed that various operations and processes in the following description are executed by the camera MPU 1125.

  18 to 20, hatched blocks indicate processing related to still images, and white blocks indicate processing related to LV images. Tr1 and Tr2 indicate the time of exposure, readout, and image acquisition (time required from the start of exposure to the completion of acquisition) of the still image and the LV image, respectively. tr1 and tr1 'indicate the exposure start time of the still image, and tr2 indicates the exposure start time of the LV image. td1 and td1 'indicate the display start time of the still image, and td2 indicates the display start time of the LV image.

  18 to 20, Td1_0 indicates a display time of a still image, and Td2_0 indicates a display time of an LV image. In addition, ΔTd1 in FIGS. 19 and 20 indicates the display delay time of the still image (the time from the completion of acquisition of the still image to the start of display), and ΔTd2 indicates the display delay time of the LV image (from the completion of acquisition of the LV image. (Time until display starts). In the examples of FIGS. 19 and 20, ΔTd1 = 0, but the present embodiment is not limited to ΔTd1 = 0. T11 to t18 and t21 to t27 in FIGS. 19 and 20 represent times in the respective drawings. Although the times are shown in the figure up to t18 and t27, if it is determined that continuous shooting is to be continued, the continuous shooting process is continued after the illustrated time.

  First, a specific example in the case of performing alternate display without display delay will be described with reference to FIG. In this case, first, a still image (for recording / display) is generated at a time Tr1 from the timing of tr1. Then, a still image for display is displayed at time Td1_0 from the timing shown by td1 without display delay.

  Next, an LV image is generated at a time Tr2 from the timing of tr2. Then, the generated LV image is displayed at time Td2_0 from timing td2 without display delay. Thereafter, the display still image and the LV image are alternately displayed in the same procedure.

  Next, with reference to FIG. 19, a description will be given of a specific example in the case of performing alternate display in consideration of a predetermined display delay time based on the first method. In the first method, the display delay time is set so that the time from the acquisition timing of the still image to the display start timing is substantially equal to the time from the acquisition timing of the LV image to the display start timing td2. That is, the first display delay time ΔTd1 and the second display delay time ΔTd2 are set such that the time from the timing tr1 to the timing td1 is substantially equal to the time from the timing tr2 to the timing td2. Then, the display time Td1 of the still image for display and the display time Td2 of the LV image are calculated from the calculated first display delay time ΔTd1 and second display delay time ΔTd2.

Specifically, the camera MPU 1125 acquires the time Tr1 and Tr2 from the start of exposure to the completion of acquisition of each of the still image and the LV image based on the imaging conditions set in S801. Then, the camera MPU 1125 determines the first display delay time ΔTd1 for a still image to be 0 (may be other than 0 as described above). Then, the camera MPU 1125 calculates the second display delay time ΔTd2 for the LV image according to the following equation.
Tr2 + ΔTd2 = Tr1 + ΔTd1
ΔTd2 = Tr1 + ΔTd1-Tr2 = Tr1-Tr2

Next, the camera MPU 1125 calculates the display times Td1 and Td2 of the still image and the LV image. Specifically, the camera MPU 1125 acquires the display times Td1_0 and Td2_0 (see FIG. 18) when the display delay times ΔTd1 and ΔTd2 do not exist, based on the imaging conditions set in S801 and S1101. Then, the camera MPU 1125 calculates the display times Td1 and Td2 according to the following equation.
Td1 = Td1_0 + (ΔTd2-ΔTd1) = Td1_0 + ΔTd2
Td2 = Td2_0 + (ΔTd1-ΔTd2) = Td1_0−ΔTd2

  Next, based on the calculated first display delay time ΔTd1 and display time Td1, the camera MPU 1125 sets a display start time (timing) and a display time of a still image for display, and instructs display. Specifically, the still image whose acquisition has been started from time tr1 starts displaying a still image for display at time t12 and displays it until time t16. It should be noted that substantially the same processing is performed on the LV image, and a description thereof will be omitted. With the configuration described above, the time from the still image acquisition time tr1 to the still image display start time td1 is substantially equal to the time from the LV image acquisition time tr2 to the LV image display start time td2. It is possible to display an image that does not exist.

  Next, with reference to FIG. 20, a specific example in the case of performing alternate display in consideration of a predetermined display delay time based on the second method will be described. In the second method according to the present embodiment, the first display delay time ΔTd1 and the LV image are set such that the display time Td1 of the still image and the display time Td2 of the LV image are substantially equal to each other. The second display delay time ΔTd2 is set. Specifically, when the second method is used, a display delay is set only for an image requiring a short time to acquire, without providing a display delay for an image requiring a long time to acquire. That is, the first display delay time ΔTd1 = 0, and the display time of each image is controlled so that the display time Td1 of the display still image and the display time Td2 of the LV image are substantially equal. That is, the second method differs from the first method in that the display times Td1 and Td2 of the still image and the LV image are different from the display start time (timing) td2 of the LV image.

  The second display delay time ΔTd2 for the LV image is calculated using the following equation 4 together with the above equation.

  Next, the camera MPU 1125 calculates the display times Td1 and Td2 of the still image and the LV image. Specifically, the camera MPU 1125 acquires the display times Td1_0 and Td2_0 (see FIG. 18) when the display delay times ΔTd1 and ΔTd2 do not exist, based on the imaging conditions set in S801 and S1101. Then, based on the first display delay time ΔTd1 = 0 and the second display delay time ΔTd2 calculated by Expression 4 described above, the camera MPU 1125 calculates display times Td1 and Td2 according to the following Expression 5.

  Next, using the calculated first display delay time ΔTd1 and display time Td1, the camera MPU 1125 sets the display start time (timing) and display time of the still image for display, and instructs display. Specifically, the still image whose acquisition has been started from time tr1 starts displaying a still image for display at time t22 and displays it until time t25. It should be noted that substantially the same processing is performed on the LV image, and a description thereof will be omitted. With the configuration described above, the display time Td1 of the still image for display and the display time Td2 of the LV image become substantially equal, so that it is possible to display an image without a sense of discomfort.

  In the case of performing the alternate display, the imaging apparatus according to the present embodiment may be configured to use any one of the various methods illustrated in the timing charts illustrated in FIGS. In particular, it is preferable that by performing the alternate display using the first and second methods described above, it is preferable to adjust the display delay of the still image and the LV image, thereby suppressing a sense of discomfort given to the user. Note that a configuration in which the first method and the second method are combined to perform the alternate display may be employed.

  As described above, according to the present embodiment, the camera body 1120 sequentially acquires a plurality of images and sequentially displays the images when it is considered that the difference between the images used for the alternate display is large based on various conditions. Is not executed. In other words, the camera body 1120 is configured to execute a process of sequentially acquiring and displaying a plurality of images (alternate display) when the difference between the images used for the alternate display is considered to be small based on various conditions. is there. Therefore, since the camera body 1120 of the present embodiment can determine whether or not to perform the alternating display based on the difference between the images used for the alternating display, the unpleasant feeling given to the user by performing the alternating display increases. Can be suppressed.

  Note that, in the above-described embodiment, an example in which still images and LV images are acquired one by one alternately has been described, but the number of images acquired as each image is not limited to this. The imaging device according to the present invention may be configured to acquire the second image (such as an LV image) at least once during acquisition of still images at predetermined intervals. For example, a configuration in which a plurality of LV images are acquired while acquiring a still image may be employed. In this case, the configuration may be such that after the display still image is displayed on the display unit 1126, the generation and display of the LV image is repeated (updated) a plurality of times, and then a new display still image is displayed. When the exposure time for acquiring an LV image with respect to a still image is short, the above-described plurality of LV images are added (frame addition), and for example, the exposure time is set to be substantially the same as the still image. You may. When the above-described frame addition is performed, the frame addition can be realized without impairing the display update interval by performing the moving average process between the LV images.

[Fourth embodiment]
Next, a fourth embodiment of the present invention will be described with reference to FIGS. In the fourth embodiment, a method of alternate display according to the setting regarding the exposure simulation function will be described. Specifically, in this embodiment, when performing the alternate display, the camera body 1120 for reducing the difference in brightness between successive images used for the alternate display regardless of whether the exposure simulation setting is on or off. Will be described.

  Here, the exposure simulation function simulates an image obtained under exposure conditions (for example, exposure parameters such as an exposure time, an aperture value, and a gain) set by a user before actually capturing a subject. Function. Therefore, by turning on the exposure simulation setting, the user can confirm in advance a still image that can be acquired by the main imaging.

  Further, there is a case where an image is captured with an inappropriate brightness for a subject depending on a user's intention. In this case, if the exposure simulation setting is turned on, the LV display is performed in accordance with the still image that can be acquired under the set exposure condition. For example, when the user acquires a still image under the under exposure condition In addition, the LV image becomes dark, and the visibility deteriorates. In such a case, by turning off the exposure simulation setting, it is possible to display an image having a brightness suitable for visual recognition by the user.

  Hereinafter, the focus detection processing, the imaging processing, and the display processing according to the present embodiment will be described with reference to FIG. The description of the same configuration as that of the third embodiment is omitted, and the same reference numeral is given. FIG. 21 is a diagram illustratively illustrating an alternate display method based on an exposure simulation setting and an alternate display setting according to the fourth embodiment of the present invention. Among them, FIG. 21A is a diagram exemplarily illustrating an image displayed on the display unit 1126 according to the alternate display setting and the exposure simulation setting according to the fourth embodiment of the present invention. Note that FIG. 21 assumes a case in which an LV image displayed on the display unit 1126 and a still image for display are alternately displayed during LV shooting. In FIG. 21, an image surrounded by a bold frame is a still image (third image) for display, and the other images are LV images.

  First, the case where the alternate display setting is ON and the exposure simulation setting is ON will be described. In this case, the brightness of the image used for alternate display differs depending on the exposure condition set by the user. For example, when the exposure condition set by the user is an appropriate brightness (exposure) for the subject, an image having a brightness corresponding to the appropriate exposure is alternately displayed as (11) shown in FIG. 21A. Used for Further, for example, when the exposure condition set by the user is the brightness (exposure) on the underside of the subject, as shown in (21) in FIG. 21A, the brightness corresponding to the underside exposure is obtained. Use the images alternately. When the exposure simulation is turned on, the brightness of the LV image is adjusted to match the brightness of the still image.

  Next, a case where the alternate display setting is on and the exposure simulation setting is off will be described. In this case, since the exposure simulation setting is turned off, it is not necessary to adjust the brightness of the image used for the alternate display according to the exposure condition set by the user. Therefore, in this case, as shown in (12) shown in FIG. 21A or (22) shown in FIG. 21A, it corresponds to an appropriate brightness (exposure) that prioritizes the visibility of the user regardless of the exposure condition. Are used for alternate display. In this case, the brightness of the display still image is adjusted to match the brightness of the LV image.

  Next, the case where the alternate display setting is off and the exposure simulation setting is on will be described. In this case, since the alternate display setting is turned off, the display still image (third image) is not displayed on the display unit 1126 (in FIG. 21A, it is indicated by a dotted line). Since the exposure simulation setting is turned on, the image processing performed on the LV image differs depending on the exposure condition. For example, when an exposure condition corresponding to an appropriate exposure for the brightness of the subject is set, as shown in (13) illustrated in FIG. 21A, an LV having a brightness corresponding to the appropriate exposure is set. Only the image is displayed on the display unit 1126. When an exposure condition corresponding to the underside exposure with respect to the brightness of the subject is set, as shown by (23) in FIG. 21A, the LV image corresponding to the underside exposure is set. Is displayed on the display unit 1126.

  Next, a case where the alternate display setting is off and the exposure simulation setting is off will be described. In this case, since the alternate display setting is turned off, a still image (third image) for display is not displayed on the display unit 1126 as shown in (14) shown in FIG. 21A (note that in FIG. 21A, This is shown by the dotted line). Then, since the exposure simulation setting is turned off, as shown in (24) shown in FIG. 21A, the LV image corresponding to the appropriate brightness (exposure) giving priority to the visibility of the user regardless of the exposure condition. Is displayed on the display unit 1126.

  Here, processing when the alternate display setting is on and the exposure simulation setting is off will be described with reference to FIG. 21B. FIG. 21B is a diagram exemplarily illustrating the alternate display when the alternate display setting is on and the exposure simulation setting is off. In this setting, usually, when a still image for display is generated, the gain is adjusted in accordance with the LV image acquired immediately before. However, as shown in (22 ') of FIG. 21B, when the exposure condition set by the user is under exposure, the still image has brightness equivalent to the under exposure, and the LV image has low visibility. The brightness is equivalent to the appropriate priority exposure. Therefore, when these images are alternately displayed, images having different brightness (no continuity) are alternately displayed, so that a sense of discomfort given to the user is large. Therefore, in the present embodiment, when the alternate display setting is on and the exposure simulation setting is off, priority is given to the alternate display, and the brightness of the image for the alternate display is adjusted in the same manner as when the exposure simulation setting is on. It may be configured (such as (21) in FIG. 21B). Conversely, in order to give priority to the exposure simulation setting, a configuration may be used in which an image used for alternate display is selected in the same manner as when the alternate display setting is turned off ((24) in FIG. 21B).

  Next, details of the generation and display processing of the LV image (second image) according to the present embodiment will be described with reference to FIG. FIG. 22 is a flowchart illustrating an LV image generation / display process according to the fourth embodiment of the present invention. Hereinafter, the description will be made assuming an n-th (n is a natural number) imaging operation corresponding to the second and subsequent imaging operations in continuous imaging. Further, the basic processing flow in this embodiment is substantially the same as that of the third embodiment (FIG. 13), and the processing relating to the second image generation / display (FIG. 22) and the third Only the processing (FIG. 23) related to image generation / display differs from the third embodiment described above. Hereinafter, points different from the third embodiment will be described.

  Steps S1801 to S1802 are the same as steps S801 to S802 in the third embodiment, and a description thereof will be omitted. If it is determined in S1802 that the alternate display flag = 1, the process proceeds to S1803, and if it is determined in S1802 that the alternate display flag is ≠ 1 (that is, the alternate display flag = 0), the process proceeds to S1806.

  In step S1803, the camera MPU 1125 determines whether the exposure simulation setting is on. Note that the camera body 1120 of the present embodiment can arbitrarily switch on / off the exposure simulation setting by operating the operation switches 1127 and the like by the user. If it is determined in step S1803 that the exposure simulation setting is ON, the flow advances to step S1804, and the camera MPU 1125 performs editing settings for the second image (n), which is an LV image used for alternate display. The process of S1804 aims to generate an LV image according to the exposure condition set by the user and use the LV image for alternate display. Therefore, the camera MPU 1125 uses the exposure condition obtained in S1801 so that the difference between the third image (n−1) and the second image (n) acquired (generated) immediately before is reduced. Edit setting of the image (second image (n)) is performed. The details of the editing settings are the same as in the third embodiment described above, and a description thereof will be omitted.

  If it is determined in step S1803 that the exposure simulation setting is OFF, the process advances to step S1805, and the camera MPU 1125 advances to step S1804. The camera MPU 1125 edits the second image (n), which is an LV image used for alternate display. I do. In the processing of S1805, since the exposure simulation setting is turned off, the editing setting of the LV image is performed in consideration of the visibility of the user so that the brightness becomes equivalent to the appropriate exposure for the brightness of the subject. In the case where the display is alternately displayed and the exposure simulation setting is turned off, the brightness of the display still image is adjusted in accordance with the LV image in a third image generation / display process described later. Therefore, the editing setting at this timing may be any setting for adjusting the brightness to an LV image in consideration of the user's visibility. Also, it is assumed that the information regarding the proper exposure is stored in advance in the memory 1128 or the like.

  Next, a case where it is determined that the alternate display flag is # 1 in the process of S1802 will be described. Note that the processing in S1806 is substantially the same as the processing in S1803, and thus description thereof is omitted. If it is determined in step S1806 that the exposure simulation setting is ON, the process advances to step S1807. In step S1807, the camera MPU 1125 edits the display LV image (second image (n)) when a still image is not displayed on the display unit 1126. I do. In the process of S1807, since the third image (the still image (n-1) for recording) acquired immediately before is not used for display, only the LV image (n) is used according to the preset exposure condition. Edit settings to make the image brighter. For example, at the time of continuous shooting, editing settings for an LV image are performed in accordance with the exposure conditions when a still image for recording (first image) is obtained. In this embodiment, in particular, the LV image is set to a desired brightness by adjusting the gain according to the exposure condition, but the brightness of the LV image according to the exposure condition is realized by adjusting other parameters. It may be a configuration.

  If the exposure simulation setting is off in step S1806, the process advances to step S1808, and the camera MPU 1125 edits the display LV image (second image (n)) when the still image is not displayed on the display unit 1126. Do. In this case, unlike the processing of S1807 described above, it is not necessary to adjust the LV image to a brightness according to the exposure condition (for example, a still image for recording), and the brightness of the LV image is displayed on the display unit 1126. Edit settings to make the brightness suitable for Subsequent processing in S1809 to S1812 is substantially the same as the processing in S805 to S808 in the third embodiment described above, and a description thereof will be omitted.

  Next, generation and display processing of a display still image (third image) according to the present embodiment will be described in detail with reference to FIG. FIG. 23 is a flowchart illustrating a process of generating and displaying a still image for display according to the fourth embodiment of the present invention. In this embodiment, as described above, the editing setting of the LV image is performed so as to approach the brightness of the still image for recording. A case will be described in which the edit setting of the display still image (n) is performed so as to match.

  The process of S1901 is the same as the process of S701 of the third embodiment described above, and thus the description is omitted. Next, in step S1902, the camera MPU 1125 determines whether the exposure simulation setting has been turned on by the user. If it is determined in step S1902 that the exposure simulation setting is ON, the flow advances to step S1903, and the camera MPU 1125 performs editing settings for a display still image (third image (n)) used for alternate display.

  In the process of S1903, in order to execute an exposure simulation, a display still image (n) is acquired so as to have a brightness corresponding to the exposure condition of the previously acquired recording still image (n). A still image for display is used for alternate display. Therefore, the camera MPU 1125 displays the still image for display (third image) such that the brightness is in accordance with the exposure condition obtained in S1901 and the difference from the previously acquired (generated) LV image (n) is small. Image (n)) is edited. When the exposure simulation setting is turned on, the brightness of the LV image is adjusted according to the still image. Therefore, the editing setting for the still image is set by the user regardless of the brightness of the immediately preceding LV image. Edit settings may be based on only the set exposure conditions.

  If it is determined in step S1902 that the exposure simulation setting is off, the flow advances to step S1904, and the camera MPU 1125 performs editing settings for a display still image (third image (n)) used for alternate display. In the processing of S1904, since the exposure simulation is not performed, a still image for display (third image (n)) is acquired in accordance with the exposure condition set by the user, and the still image for display is used for alternate display. The purpose is to: Therefore, the camera MPU 1125 sets the display still image (third image (n)) so as to reduce the difference from the previously acquired (generated) LV image (n) regardless of the exposure condition acquired in S1901. ) Edit settings. As this editing setting, for example, a gain adjustment for acquiring a display still image is performed in accordance with the brightness of the LV image. That is, when performing the alternating display and not performing the exposure simulation, the brightness of the still image is adjusted in accordance with the brightness of the LV image acquired (or displayed) immediately before. The processing of S1905 to S1908 after the photographing is substantially the same as the processing of S1103 to S1106 of the third embodiment, and a description thereof will be omitted.

  As described above, the camera main body 1120 according to the present embodiment is configured to acquire each image such that the difference between the images used for the alternate display becomes small regardless of the presence or absence of the exposure simulation. For example, when the exposure simulation setting is turned on, the brightness of the LV image is adjusted based on the exposure condition set by the user. When the exposure simulation setting is turned off, the brightness of the still image is adjusted in accordance with the brightness of the LV image, giving priority to the visibility of the user. With this configuration, it is possible to reduce continuous display of images having different brightness in alternate display. Therefore, the camera body 1120 according to the present embodiment suppresses the image displayed on the display unit 1126 from becoming unnatural according to the setting related to the alternate display and the exposure simulation setting, and reduces a sense of discomfort given to the user. I can do it.

  In the above-described embodiment, the point that the brightness of the still image is adjusted in accordance with the LV image when the alternate display setting is on and the exposure simulation setting is off has been described. It is not something to be done. For example, to perform brightness adjustment (for example, gain adjustment or image processing) on a high-resolution image with a small pixel thinning amount such as a still image, the calculation load is large. In this case, the camera MPU 1125 performs alternate display. The configuration may be such that control is not performed.

  When adjusting a still image in accordance with an LV image, the field of view may be adjusted in addition to the brightness. In LV display, electronic image stabilization is often performed, so it may be narrower than the view range of the still image.In such a case, adjust the still image according to the view range of the LV image Is preferred.

  Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these, and various modifications and changes can be made within the scope of the gist. For example, in the above-described embodiment, an example in which focus detection is performed using an LV image has been described. However, the second image is not limited to the LV image. For example, when separately obtaining an image used for focus detection and an LV image, both the image for focus detection and the LV image correspond to the second image. Further, in a case where the LV display is not performed before the start of capturing of a still image, a configuration in which a focus detection image and a still image that are not used as an LV image are alternately displayed when alternate display is performed during continuous shooting of a subject. It may be.

  In the above-described embodiment, a so-called interchangeable lens type imaging apparatus in which a lens unit can be attached to and detached from the camera body 1120 has been described. However, the present invention is not limited to this. For example, the configuration of the present invention described above can be applied to a case where a lens fixed type imaging device in which a camera body and a lens unit are integrally formed is adopted.

  Further, in the above-described embodiment, the case where the digital camera is employed as an example of the imaging apparatus for implementing the present invention has been described, but the present invention is not limited to this. For example, a configuration may be adopted in which a portable device such as a digital video camera or a smartphone, or an imaging device other than a digital camera such as a security camera is used.

[Other Examples]
The present invention supplies a program for realizing one or more functions of the above embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This process can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

  As described above, the preferred embodiments of the present invention have been described, but the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist.

301: Lens unit 302: Lens drive unit 304: Aperture 305: Shutter / aperture drive unit 306: Image sensor 307: Image signal processing circuit 309: Overall control operation unit

Claims (13)

An imaging device for imaging a subject through an imaging optical system,
An image sensor that acquires an image signal from light formed by the imaging optical system;
First control means for controlling an exposure parameter of the image sensor;
A first image corresponding to a pixel portion having a first pixel number and a second image corresponding to a pixel portion having a second pixel number smaller than the first pixel number among the pixel portions of the image sensor. And a second control means for performing a process of switching and outputting
The second control unit performs a focus determination based on at least one of the first image and the second image,
The second control means reads out an image signal pupil-divided by the image sensor, performs focus detection, controls focus adjustment of the imaging optical system,
Said first control means, the first is an image with controlling exposure parameters for acquiring the second image, the exposure parameters of the first image and the second image to be acquired continuously Reduce the amount of change ,
The first control means controls an exposure parameter for acquiring the second image based on the focus determination, thereby controlling an exposure parameter between the first image and the second image acquired successively. Reduce the amount of change in
Imaging device. The exposure parameter comprises at least one of an aperture value, sensitivity, and exposure time,
The imaging device according to claim 1 . Display means for displaying the first image and the second image on a display unit,
The display means reduces a difference in an exposure parameter between the first image and the second image when the first image and the second image are alternately displayed on the display unit. Generate and display a third image ,
The imaging device according to claim 1 . The difference between the first image and the second image includes an exposure condition related to the exposure parameter, or blur information related to subject blur information and contrast information.
The imaging device according to claim 3 . The display means generates the third image by combining the first image and the second image,
The imaging device according to claim 3 . A determination unit configured to determine whether to alternately display the first image and the second image on the display unit,
The determination unit determines whether or not to continuously display the first image and the second image on the display unit based on predetermined information regarding the first image or the second image. Do
The imaging device according to claim 3 . The predetermined information includes a light emission amount of the light emitting device when acquiring the first image, a difference in an exposure parameter between the first image and the second image, and a time when acquiring the second image. Focus detection results, information on the presence / absence of emission of auxiliary light related to focus detection when acquiring the second image, information on gains related to the first image and the second image, and presence / absence of flicker detection of a light source Is at least one of
The imaging device according to claim 6 . The first image is a still image for recording, and the second image is a live view image used for a live view.
The imaging device according to claim 1 . The first control means sets an exposure parameter such that a condition for acquiring the first image has a higher priority.
An imaging device according to claim 8 . The second control means switches between a first focus determination mode in which priority is given to followability during framing and a second focus determination mode in which priority is given to accuracy of focus adjustment, so that the first image and the Performing a first process of alternately outputting a second image or a second process of outputting the second image without outputting the first image;
An imaging device according to claim 8 . The imaging device includes a plurality of microlenses, and a plurality of photoelectric conversion units corresponding to each microlens,
The imaging device according to claim 1 . A control method of an imaging device that images a subject through an imaging optical system,
Using an imaging device, an acquisition step of acquiring an image signal from light imaged by the imaging optical system,
A first control step of controlling an exposure parameter of the image sensor;
A first image corresponding to a pixel portion having a first pixel number and a second image corresponding to a pixel portion having a second pixel number smaller than the first pixel number among the pixel portions of the image sensor. And a second control step of performing a process of switching and outputting.
The second control step performs a focus determination based on at least one of the first image and the second image,
The second control step is to read out an image signal subjected to pupil division by the image sensor, perform focus detection, control focus adjustment of the imaging optical system,
Said first control step, by controlling the exposure parameters for acquiring the first image and the second image, the first image and the exposure parameters of the second image to be acquired continuously Reduce the amount of change ,
The first control step controls an exposure parameter for acquiring the second image based on the focus determination, so that an exposure parameter for the first image and the second image acquired successively is obtained. Reduce the amount of change in
Control method. A program for causing a computer to execute the control method according to claim 12 .

Images