JP2020046615A - Control device, imaging apparatus, control method, program, and storage medium - Google Patents

Abstract

To provide a control device capable of performing precise image stabilization in panning shooting.SOLUTION: A control device (110) includes: detection means (110a) that detects motion information of a subject; acquisition means (110b) that acquires a piece of distance information on the basis of a focus signal from a first sensor (302) or a second sensor (303) of phase difference detection type arranged in different directions; and correction means (110c) that performs image stabilization on the basis of the motion information and the distance information. The correction means performs the image stabilization using the distance information from the selected first sensor or second sensor on the basis of the panning direction.SELECTED DRAWING: Figure 7

Description

  The present invention relates to an imaging apparatus capable of performing image blur correction to assist panning shooting.

  As a photographing technique for expressing a sense of speed of a moving subject, panning photographing is known. The panning shooting is intended for a user to pan the imaging apparatus in accordance with the movement of the subject, thereby stopping the moving subject and flowing the background. Normally, panning shooting is performed by setting the shutter speed to be low in accordance with the moving speed of the subject and setting the exposure time longer than usual. However, when the shutter speed is set to a low speed, camera shake and subject shake are likely to occur. Further, since the imaging apparatus is panned in accordance with the movement of the subject, if the subject speed does not match the panning speed, main subject blur occurs.

  Patent Literature 1 discloses that a motion vector is detected from a plurality of frames continuously captured by an imaging device based on a moving amount of each region in a screen, and a main subject and a background are identified using a defocus amount. An image processing apparatus that performs panning shot shooting is disclosed.

JP-A-2006-171460

  However, in the image processing device disclosed in Patent Literature 1, the motion vector detection region and the defocus amount calculation region need to match. On the other hand, when performing panning shooting using a finder, since the optical systems of the photometric sensor and the distance measuring sensor are different from each other, the motion vector detection area and the defocus area calculation area are not necessarily the same. In addition, since the distance measurement sensor determines information of a specific point by combining a plurality of sensors, a plurality of defocus amounts are generated for a motion vector detection area, and it is determined which defocus amount should be used. May not be determined. As a result, high-precision image blur correction cannot be performed in panning shooting.

  Therefore, an object of the present invention is to provide a control device, an imaging device, a control method, a program, and a storage medium that can perform high-precision image blur correction in panning shooting.

  A control device according to one aspect of the present invention includes: a detection unit configured to detect movement information of a subject; and a distance information based on a focus signal from a first sensor or a second sensor of a phase difference detection method arranged in different directions. Acquiring means for acquiring, and correcting means for performing image blur correction based on the motion information and the distance information, wherein the correcting means selects the first sensor or the second sensor selected based on a panning direction. The image blur correction is performed using the distance information from the sensor.

  An imaging device according to another aspect of the present invention includes an imaging device that photoelectrically converts an optical image formed via an imaging optical system, and the control device.

  A control method according to another aspect of the present invention includes a step of detecting motion information of a subject, and a step of detecting distance information based on a focus signal from a first sensor or a second sensor of a phase difference detection method arranged in different directions. Acquiring, and performing an image blur correction based on the motion information and the distance information, wherein in the step of performing the image blur correction, the first sensor or the first sensor selected based on a panning direction. The image blur correction is performed using the distance information from the second sensor.

  A program according to another aspect of the present invention causes a computer to execute the control method.

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the present invention will be described in the following embodiments.

  According to the present invention, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium that can perform high-precision image blur correction in panning shooting.

It is a schematic sectional view of the imaging device in this embodiment. It is a block diagram of an imaging device in this embodiment. FIG. 5 is a diagram illustrating a relationship between an autofocus unit and a focus plate according to the embodiment. It is an example of an output of a photometry unit in this embodiment. FIG. 5 is a diagram illustrating a relationship among a vector detection frame, vector information, and a focus plate according to the embodiment. FIG. 4 is a diagram illustrating a positional relationship between a photometric unit and an autofocus unit on a focus plate according to the embodiment. 6 is a flowchart illustrating a procedure for creating a defocus map according to the embodiment. It is an example of a defocus map in the present embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, the structure of the imaging device 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a schematic cross-sectional view of the imaging apparatus 100. FIG. 1A shows a state before shooting, and FIG. 1B shows a state during shooting. The imaging device 100 includes an imaging device main body (camera main body) 1 and a lens device (interchangeable lens) 2 that is detachable from the imaging device main body 1. However, the present invention is not limited to this, and is also applicable to an imaging device in which an imaging device main body and a lens device are integrally configured.

  As shown in FIG. 1A, the user can observe the light that has entered the lens device 2 from the subject before photographing with the eyepiece unit 105, and can confirm the photographable scene. As shown in FIG. 1B, during shooting, the shutter 107 and the movable mirror unit 109 operate, so that light from a subject can be emitted to the image sensor 108.

  The lens device 2 is mounted on the front surface of the imaging device main body 1 and is connected to the imaging device main body 1 via the mount contact 112. The camera control unit 110 and the lens control unit 113 are electrically connected to each other via the mount contact 112. The lens control unit 113 adjusts the position of the lens group 115 and controls the stop 114. The camera controller 110 controls the position of the lens group 115 and controls the aperture 114 via the lens controller 113. In the present embodiment, the lens group 115 and the stop 114 form an imaging optical system.

  Light from the subject is condensed by the lens group 115 of the lens device 2 and is radiated on the focus plate 101 by the movable mirror unit 109 including a main mirror and a half mirror. The focus plate 101 is arranged at a position where an image is formed as a photographable scene of the image pickup apparatus main body 1. The pentaprism 102 reflects a photographable scene formed on the focus plate 101 to the eyepiece 104. By observing the eyepiece 104 through the eyepiece 105, the user can observe a photographable scene formed on the focus plate 101.

  The photometric unit 103 converts the light flux formed on the focus plate 101 into an electric signal and outputs it as a digital value. Further, the photometry unit 103 converts a scene constrained by a plurality of pixels and formed on the focus plate 101 into an image signal. The photometric unit 103 is electrically connected to the camera control unit 110 (in FIG. 1A, a connection line between the photometric unit 103 and the camera control unit 110 is omitted). The camera control unit 110 can receive the signal output from the photometry unit 103 and measure the light flux condensed by the lens device 2. Further, the camera control unit 110 can measure a change in a subject based on an image signal output from the photometry unit 103.

  The light collected by the half mirror of the movable mirror unit 109 is applied to the auto focus unit 111. The auto focus unit 111 has a phase difference detection type sensor (focus detection unit), and performs focus detection in a plurality of directions such as a horizontal direction and a vertical direction with respect to one ranging point (focus detection point). The focus detection result (measured value) is transmitted to the camera control unit 110.

The camera control unit 110 receives the focus detection result (measured value) from the autofocus unit 111, selects information advantageous for focal length adjustment based on a plurality of focus detection results for one ranging point, and performs ranging. Perform the operation. Then, the camera control unit 110 calculates a movement amount for adjusting the focusing lens included in the lens group 115 to the focal position. At the same time as the distance measurement calculation, the camera control unit 110 calculates a defocus amount representing distance information (relative distance information from the focal position) and its reliability (defocus reliability). Then, the camera control unit 110 creates a defocus map (defocus information) representing a defocus amount for each ranging point (focus detection point) based on the reliability.
The camera control unit 110 adjusts the focal length by adjusting the position of the focusing lens based on the calculated movement amount (performs focus control). Further, the camera control unit 110 performs photometric correction based on the defocus map.

  As shown in FIG. 1A, before photographing, the movable mirror unit 109, the shutter 107, and the image sensor 108 are arranged on the optical path OP of the lens device 2. On the other hand, as shown in FIG. 1B, at the time of photographing, the movable mirror unit 109 moves so as to be off the optical path OP of the lens device 2, and the light beam of the lens device 2 is irradiated on the image sensor 108. The imaging element 108 has a CMOS image sensor or a CCD image sensor, and photoelectrically converts an optical image (subject image) formed via the imaging optical system of the lens device 2. The display unit 106 displays the image signal output from the image sensor 108 as a shooting result.

  Next, the configuration of the imaging device 100 will be described with reference to FIG. FIG. 2 is a block diagram of the imaging device 100. In FIG. 2, portions common to FIG. 1 are denoted by the same reference numerals, and detailed description is omitted. The lens control unit 113 is electrically connected to the aperture drive unit 201 and the lens drive unit 202, respectively, and adjusts the positions of the aperture 114 and the lens group 115 in response to a request from the camera control unit 110. The movable mirror controller 203 controls the state of the movable mirror 109 according to a request from the camera controller 110. The shutter control unit 204 controls the state of the shutter 107 in response to a request from the camera control unit 110.

  The signal processing unit 205 converts an electric signal output from the image sensor 108 into an image signal, and performs signal processing on the image signal. The signal processing unit 205 performs processing such as rotation, expansion, and reduction on the image signal. The display unit 106 displays the image signal output from the signal processing unit 205 as a shooting result. The recording unit 206 records the image signal output from the signal processing unit 205 with the photographing on a recording medium. In the present embodiment, the storage medium is a rewritable nonvolatile memory, but is not limited to this.

  The gyro 207 is an angular velocity sensor (panning detection unit) that detects the movement (angular velocity) of the imaging device main body 1. In the present embodiment, the gyro 207 detects the movement of the imaging apparatus body 1 when the user captures the main subject on the optical path of the lens device 2 and acquires panning information such as a panning direction (a panning direction). be able to.

  The camera control unit 110 includes, for example, a microcomputer (CPU), and has a ROM and a RAM (not shown). The camera control unit 110 controls the operation of the imaging device main body 1 using the RAM as a working memory according to programs and data recorded in the ROM in advance. Further, the camera control unit 110 has a detecting unit 110a, an acquiring unit 110b, a correcting unit 110c, and a determining unit 110d. The detection unit 110a detects motion information (motion vector information) of the subject based on the image acquired by the photometry unit 103. The acquisition unit 110b performs distance information (such as a defocus map) based on a focus signal from a first sensor (vertical sensor 302) or a second sensor (horizontal sensor 303) of a phase difference detection method arranged in different directions. Defocus information). The correction unit 110c performs image blur correction based on the motion information and the distance information. The determination unit 110d determines the panning direction based on a signal from the gyro 207 that detects the panning direction. In the present embodiment, the correction unit 110c performs image blur correction using distance information from the first sensor or the second sensor selected based on the panning direction.

  The image sensor 108 has, for example, a CMOS image sensor or a CCD image sensor, and accumulates electric charge under the control of the camera control unit 110. The value of the charge stored in the image sensor 108 or the charge converted into a digital signal is read and output to the signal processing unit 205.

  Next, with reference to FIGS. 3 to 6, image blur correction in panning shooting using the imaging apparatus 100 will be described. FIG. 3A shows a detection area of a phase difference detection type sensor mounted on the autofocus unit 111. As shown in FIG. 3A, the autofocus unit 111 includes a vertical direction sensor (first sensor) 302 and a horizontal direction sensor (second sensor) that are phase difference detection sensors with respect to one ranging point 301. Distance measurement (focus detection) is performed by using 303. FIG. 3B is a diagram illustrating distance measuring points detectable by the autofocus unit 111 on the focus plate 101. The camera control unit 110 obtains the reliability (defocus reliability) simultaneously with the output of the phase difference detection type sensor in order to determine the information of the ranging point, thereby determining which of the vertical direction sensor 302 and the horizontal direction sensor 303. Whether to use the information can be uniquely determined. Further, the camera control unit 110 can obtain relative distance information from the current focus position for each ranging point. FIG. 3C is an example of a defocus map (relative distance information) of each ranging point created when performing autofocus.

  FIG. 4A is a diagram in which image signal information 401 acquired by the photometric unit 103 is represented on the focus plate 101. From the photometric unit 103, continuous image signals can be obtained as shown in FIG. FIG. 5A shows a vector detection frame 501 set for measuring the amount of change in a continuous image acquired by the photometry unit 103. FIG. 5B shows the result of detecting the movement amount (vector information 502) for each vector detection frame 501. By performing shake correction of the main subject using the vector information 502, it is possible to perform shooting suitable for panning shooting. On the other hand, since the vector information 502 is effective for the image signal acquired by the photometry unit 103, the main subject cannot be determined. Thus, the camera control unit 110 can determine the main subject area and the other areas by using a defocus map (relative distance information) as shown in FIG.

  However, as shown in FIG. 1, the photometric unit 103 and the autofocus unit 111 of the present embodiment have different optical systems. For this reason, as shown in FIG. 6, the information of the same position is not always obtained. Therefore, the camera control unit 110 determines a defocus amount corresponding to each vector detection frame 501, and recreates (updates) a defocus map.

  Next, a procedure for creating a defocus map (distance information) for performing a panning shot using a phase difference detection type sensor having a structure as shown in FIG. 3A will be described with reference to FIG. FIG. 7 is a flowchart showing a procedure for creating a defocus map. Each step in FIG. 7 is mainly executed by each unit of the camera control unit 110.

  First, in step S101, the camera control unit 110 (determination unit 110d) determines a panning direction (panning direction) based on a signal from the gyro (angular velocity sensor) 207. Subsequently, in step S102, the camera control unit 110 (the obtaining unit 110b) determines the defocus amount and the reliability based on the focus signals from the phase difference detection type sensors (the first sensor and the second sensor) arranged in different directions. (Defocus reliability) is obtained. In the present embodiment, the first sensor is the vertical sensor 302 and the second sensor is the horizontal sensor 303, but is not limited thereto.

  Subsequently, in step S103, the camera control unit 110 selects the vector detection frame 501 for which the defocus amount is to be determined. Subsequently, in step S104, the camera control unit 110 extracts the phase difference detection type sensors (the vertical direction sensor 302 and the horizontal direction sensor 303) that intersect with the vector detection frame 501 selected in step S103. The camera control unit 110 (acquisition unit 110b) acquires the reliability (defocus reliability) of the vector detection frame 501. In the present embodiment, the camera control unit 110 determines the first reliability (first defocus reliability), that is, the panning direction (panning direction), based on the focus signal (first focus signal) from the vertical direction sensor 302. Obtain the reliability in the orthogonal direction. Further, the camera control unit 110 acquires a second reliability (second defocus reliability), that is, a reliability in a direction parallel to the panning direction, based on the focus signal (second focus signal) from the horizontal direction sensor 303. I do.

  Subsequently, in step S105, the camera control unit 110 (correction unit 110c) compares the first reliability with the second reliability and determines whether the first reliability is higher than the second reliability. When the first reliability is higher than the second reliability, the process proceeds to step S106. In step S106, the camera control unit 110 selects, as the defocus amount for the vector detection frame 501, the defocus amount acquired from the vertical sensor 302 orthogonal to the panning direction, and proceeds to step S110.

  On the other hand, if the first reliability is lower than the second reliability in step 105, the process proceeds to step S107. In step S107, the camera control unit 110 (correction unit 110c) determines whether the first reliability is higher than a predetermined reliability. When the first reliability is higher than the predetermined reliability (when the reliability in the predetermined range is obtained), the process proceeds to step S108. In step S108, the camera control unit 110 selects the defocus amount acquired from the vertical direction sensor 302 orthogonal to the panning direction as the defocus amount for the vector detection frame 501, and proceeds to step S110.

  On the other hand, when the first reliability is lower than the predetermined reliability in step S107 (when the reliability in the predetermined range is not obtained), the process proceeds to step S109. In step S109, the camera control unit 110 selects the defocus amount acquired from the horizontal direction sensor 303 parallel to the panning direction as the defocus amount for the vector detection frame 501, and proceeds to step S110.

  In step S110, the camera control unit 110 determines whether the defocus amounts have been assigned (set) to all the vector detection frames 501. If there is an unassigned vector detection frame 501, the process returns to step S103, and steps S103 to S110 are repeated. On the other hand, when the allocation of the defocus amount to all the vector detection frames 501 is completed, it is determined that the creation of the defocus map has been completed, and the present flow ends.

  FIG. 8 is an example of a defocus map. In the present embodiment, the camera control unit 110 determines the main subject (main subject area) using the defocus map created as described above, thereby improving the accuracy of determining the main subject. . Note that FIG. 7 illustrates a case where a vertical direction sensor 302 and a horizontal direction sensor 303 intersect vertically as shown in FIG. 3A, but the present invention is not limited to this. That is, the first sensor and the second sensor may be arranged in directions different from each other (such as an oblique direction). When the vector detection frame and the detection area overlap, a sensor in a direction closer to the direction orthogonal to the panning direction among the first sensor and the second sensor may be prioritized (selected).

  As described above, in the present embodiment, the control device (camera control unit 110) includes the detection unit 110a, the acquisition unit 110b, and the correction unit 110c. The detecting means detects motion information (motion vector) of the subject. The acquisition unit 110b performs distance information (such as a defocus map) based on a focus signal from a first sensor (vertical sensor 302) or a second sensor (horizontal sensor 303) of a phase difference detection method arranged in different directions. Defocus information). The correction unit 110c performs image blur correction based on the motion information and the distance information. The correction unit 110c performs image blur correction using the distance information from the first sensor or the second sensor selected based on the panning direction (the panning direction).

  Preferably, the camera control unit 110 includes a determination unit 110d that determines a panning direction based on a signal from the angular velocity sensor (gyro 207). Preferably, the detecting means detects the motion information of the subject based on an image acquired from the photometric means (photometric unit 103) for measuring the brightness of the subject. More preferably, the acquisition unit acquires relative distance information from the focal position as the distance information.

  Preferably, the acquisition unit acquires, as the distance information, first distance information used for measuring luminance and second distance information used for image blur correction. More preferably, the acquiring unit acquires the second distance information using the distance information from the first sensor or the second sensor selected based on the panning direction. More preferably, the acquiring unit acquires a first reliability (first defocus reliability) related to the distance information based on the first focus signal from the first sensor, and converts the first reliability into a second focus signal from the second sensor. Based on the distance information, a second reliability (second defocus reliability) is acquired. Then, the correction unit performs the image blur correction using the distance information from the first sensor or the second sensor selected based on the panning direction, the first reliability, and the second reliability. More preferably, the first sensor and the second sensor are arranged in directions different from each other. More preferably, the direction of the first sensor is closer to the direction orthogonal to the panning direction than the direction of the second sensor.

  Preferably, when the first reliability is higher than the second reliability, the correction unit performs blur correction using the distance information from the first sensor (S105, S106). More preferably, when the first reliability is lower than the second reliability, the correction unit performs the image blur correction using the distance information from the second sensor (S105, S109). Preferably, the correction unit performs the image blur correction using the distance information from the first sensor when the first reliability is higher than the predetermined reliability, and performs the image blur correction when the first reliability is also lower than the predetermined reliability. Then, image blur correction is performed using the distance information from the second sensor (S107 to S109).

(Other embodiments)
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

  In the present embodiment, when performing panning shooting using a distance measurement sensor and a photometry sensor in finder shooting, it is possible to create defocus information suitable for finder shooting and improve the recognition accuracy of the main subject. . Therefore, according to the present embodiment, it is possible to provide a control device, an imaging device, a control method, a program, and a storage medium capable of performing highly accurate image blur correction in panning shooting.

  As described above, the preferred embodiments of the present invention have been described, but the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist.

110 Camera control unit (control device)
110a Detecting means 110b Acquiring means 110c Correcting means 302 Vertical sensor (first sensor)
303 Horizontal sensor (second sensor)

Claims (16)

Detecting means for detecting motion information of the subject;
Acquiring means for acquiring distance information based on a focus signal from a first sensor or a second sensor of a phase difference detection method arranged in different directions,
Correction means for performing image blur correction based on the motion information and the distance information,
The control device according to claim 1, wherein the correction unit performs the image blur correction using the distance information from the first sensor or the second sensor selected based on a panning direction.   The control device according to claim 1, further comprising a determination unit configured to determine the panning direction based on a signal from an angular velocity sensor.   3. The control device according to claim 1, wherein the detection unit detects the motion information of the subject based on an image acquired from a photometric unit that measures luminance of the subject. 4.   The control device according to claim 3, wherein the acquisition unit acquires, as the distance information, relative distance information from a focus position.   The said acquisition means acquires the 1st distance information used for the measurement of the said brightness | luminance, and the 2nd distance information used for the said image blur correction as said distance information, The Claims 3 or 4 characterized by the above-mentioned. Control device.   The said acquisition means acquires the said 2nd distance information using the said distance information from the said 1st sensor or the said 2nd sensor selected based on the said panning direction, The said 2nd distance information. Control device. The acquisition means,
Acquiring a first reliability related to the distance information based on a first focus signal from the first sensor;
Acquiring a second reliability related to the distance information based on a second focus signal from the second sensor;
The correction unit uses the distance information from the first sensor or the second sensor selected based on the panning direction, the first reliability, and the second reliability, and uses the distance information from the image sensor. 7. The control device according to claim 6, wherein correction is performed.   The control device according to claim 7, wherein the first sensor and the second sensor are arranged in directions different from each other.   9. The control device according to claim 8, wherein a direction of the first sensor is closer to a direction orthogonal to the panning direction than a direction of the second sensor.   10. The image blur correction device according to claim 9, wherein when the first reliability is higher than the second reliability, the correction unit performs the image blur correction using the distance information from the first sensor. Control device.   The method according to claim 10, wherein the correction unit performs the image blur correction using the distance information from the second sensor when the first reliability is lower than the second reliability. Control device. The correction means,
When the first reliability is higher than a predetermined reliability, the image blur correction is performed using the distance information from the first sensor,
11. The control device according to claim 9, wherein when the first reliability is also lower than the predetermined reliability, the image blur correction is performed using the distance information from the second sensor. 12. An image sensor that photoelectrically converts an optical image formed via the imaging optical system,
An imaging apparatus comprising: the control device according to claim 1. Detecting motion information of the subject;
Acquiring distance information based on a focus signal from a first sensor or a second sensor of a phase difference detection method arranged in different directions;
Performing image blur correction based on the motion information and the distance information,
The control method, wherein in the step of performing the image blur correction, the image blur correction is performed using the distance information from the first sensor or the second sensor selected based on a panning direction.   A program causing a computer to execute the control method according to claim 14.   A storage medium storing the program according to claim 15.