JP7182962B2 - IMAGING DEVICE, LENS DEVICE, AND CONTROL METHOD THEREOF - Google Patents

Description

The present invention relates to a technique for reducing image blurring and distortion in an imaging system that includes a lens device and a device main body that communicate with each other.

2. Description of the Related Art There is a technique for detecting camera shake or the like applied to an imaging device and correcting image blur caused by the shake. An image blur correction method in which an image blur correction lens is moved according to the detected shake is called optical image blur correction or optical image stabilization. Further, image blur correction that corrects blurring of a photographed image during moving image shooting by extracting and outputting a part of the photographed image according to the detected shake is called electronic image blur correction or electronic image stabilization.

In recent years, there has also been known a technique of increasing the image blur correction effect for large image blur caused by shooting while walking or the like by expanding the image blur correction range especially on the wide side (wide-angle side) during moving image shooting. By using both the optical image blur correction and the electronic image blur correction, a greater correction effect can be obtained and even larger image blur can be handled.

On the other hand, in an interchangeable lens type camera system, the lens device attached to the camera body has an optical image blur correction mechanism, and the camera body has an optical image blur correction section or an electronic image blur correction section. can be considered. In other words, it is a system in which the lens device and the camera body are combined so as to perform shake correction independently. In such a system, a technique has been disclosed in which the lens device and the camera body do not control shake correction independently, but cooperate with each other through mutual communication to improve the correction effect.

Japanese Unexamined Patent Application Publication No. 2002-200000 discloses a technique of correcting an image based on a motion vector detected by a motion vector detection means of a camera body and an output of a distortion correction amount calculation section of a lens unit. In Patent Document 2, an angular velocity detection period and ID information are transmitted in association with each other, and an angular velocity detected during the angular velocity detection period and ID information corresponding to the angular velocity are received in association with each other, and the ID information matches each other. , discloses a technique for calculating angular velocity information of a subject.

Japanese Patent No. 5984574 JP 2018-17751 A

In the case of an interchangeable lens type camera system, it is necessary to perform control without being conscious of individual lens specifications and camera specifications in order to support various combinations of interchangeable lenses and camera bodies. Also, if the amount and frequency of communication between the camera body and the lens device is large, there is a possibility that the processing cannot be completed within the specified time. is required.

In general, the CPU that controls the camera body in an interchangeable lens type camera system always performs parallel processing in order to operate various built-in functions. Processing to reduce shake and distortion may be delayed. Also, in data transfer (lens communication) through communication between the camera body and the interchangeable lens via the mount terminal, communication such as focus lens control, aperture control, and status acquisition is performed as necessary. Therefore, communication for reducing image blurring and distortion of an image may be delayed. As for the motion vector, the timing of completion of outputting the motion vector also changes depending on the specifications of the image sensor mounted on the camera body, the shooting mode, the frame rate, the shutter speed, and the like.

In other words, in an interchangeable lens camera system, due to depletion of the lens communication band or CPU load concentration, lens communication cannot be performed at the timing assumed in advance, or a motion vector output cannot be obtained at the timing. I can. In processing to reduce image blurring and distortion of an image, if a motion vector output whose detection timing does not match with the image blurring correction amount received from lens communication is used, an erroneous image blurring correction amount is calculated, resulting in deterioration of performance and operation. It may cause anomalies. Therefore, in order to improve the performance of reducing image blurring and distortion of an image by an interchangeable lens camera system, it is necessary to appropriately manage synchronization between motion vector output and transmission/reception in lens communication.

Japanese Unexamined Patent Application Publication No. 2002-200002 discloses a technique of correcting an image based on a motion vector detected by a motion vector detection means of a camera body and an output of a distortion correction amount calculation section of a lens unit. However, no mention is made of synchronizing motion vector output with reception from lens communications.

Japanese Patent Application Laid-Open No. 2002-200001 discloses a technique of transmitting ID information in association with a detection period of angular velocity, and confirming matching of the ID information by receiving the result. However, this is synchronous processing of transmission and reception in paired lens communication, and no mention is made of synchronization between motion vector output and transmission and reception in lens communication.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image pickup system comprising a lens apparatus and an image pickup apparatus main body that communicate with each other, in which image blur correction by a plurality of correction means and shake detection means is controlled in a coordinated and synchronized manner. It is to provide systems and methods of controlling them.

One aspect of the present invention is an imaging device to which a lens device can be attached, comprising: an imaging device that acquires an image signal corresponding to an image frame by exposure; communication control means that communicates with the lens device; Calculation means for calculating first image stabilization information corresponding to shake; Image stabilization control means for performing image stabilization control on an imaging frame; and setting means for setting in association with the timing corresponding to the exposure center of gravity after transmitting the first ID information set by the setting means in correspondence with the imaging frame. after a predetermined time has passed, a communication request is transmitted to the lens device, and in response to the communication request, second image stabilization information corresponding to vibration of the lens device and the first ID information are received. When the second ID information and the first ID information corresponding to the image pickup frame to be subjected to the anti-shake control match, the anti-shake control means controls the first anti-shake information and the first ID information. and the second image stabilization information.

An image pickup system comprising a lens device and an image pickup device to which the lens device can be attached, comprising: an image pickup device for acquiring an image signal corresponding to an image pickup frame by exposure; and camera communication control means for communicating with the lens device. a lens communication control means for communicating with the imaging device; a first calculation means for calculating first vibration isolation information corresponding to the vibration of the imaging device; and a second vibration isolation corresponding to the vibration of the lens device. second calculation means for calculating information; image stabilization control means for performing image stabilization control on a captured frame based on the first image stabilization information and the second image stabilization information; and the imaged frame and a first setting means for setting ID information corresponding to the image pickup frame in association with the first image stabilizing information, wherein the camera communication control means is set by the setting means in correspondence with the imaging frame. After a predetermined time has elapsed from the timing corresponding to the center of gravity of exposure after transmitting the first ID information, the second anti-vibration information calculated by the second calculation means and the first ID information are calculated. Received from the lens communication control means, and the anti-shake control means, when the second ID information corresponding to the imaging frame to be subjected to anti-shake control matches the first ID information, the It is characterized in that image stabilization control is executed for the imaging frame based on the first image stabilization information and the second image stabilization information.

According to the present invention, in an imaging system that includes a lens device and a main body of an imaging device that communicate with each other, image blur correction by a plurality of correctors and blur detectors can be controlled in a coordinated and synchronized manner.

1 is a block diagram showing a configuration example of an imaging system according to an embodiment of the present invention; FIG. FIG. 2 is a block diagram of a portion related to control of image blur correction; 3 is a block diagram illustrating details of an electronic image blur correction control unit; FIG. FIG. 4 is a diagram illustrating the timing of communication and shake detection in the first embodiment; 4 is a flowchart for explaining communication and control of a camera body in the first embodiment; 4 is a flowchart for explaining communication and control of the lens device in the first embodiment; 9 is a flow chart for explaining communication and control of a camera body in the second embodiment; FIG. 11 is a diagram for explaining timings of communication and shake detection in the fourth embodiment; FIG. 14 is a flow chart for explaining communication and control of a camera body in a fourth embodiment; FIG. FIG. 12 is a flowchart for explaining communication and control of the lens device in the fourth embodiment; FIG. 7 is a graph showing the relationship between focal length and image blur correction movable range; 4 is a graph showing the relationship between focal length and division factor; FIG. 4 is a conceptual diagram of rolling shutter distortion correction; 4A and 4B are explanatory diagrams of the pitch direction, the yaw direction, and the roll direction in the imaging device;

Hereinafter, each embodiment of the present invention will be described in detail with reference to the accompanying drawings. First, matters common to each embodiment will be described.

FIG. 1 is a block diagram showing the configuration of an imaging system according to one embodiment of the present invention. The imaging system is a lens-interchangeable digital camera for mainly taking still images and moving images. The scope of application of the present invention is not limited to digital cameras, and the present invention can be applied to various imaging systems.

The imaging system shown in FIG. 1 is composed of a lens device and a camera body, and the lens device is attached to the camera body for use. The lens system comprises 101-112, and the camera body comprises 113-125. A zoom unit 101 of the lens apparatus includes a zoom lens for changing magnification. A zoom drive control unit 102 drives and controls the zoom unit 101 . The diaphragm unit 103 has a diaphragm function. A diaphragm drive control unit 104 drives and controls the diaphragm unit 103 . The image blur correction unit 105 includes an image blur correction lens (hereinafter also referred to as a correction lens) such as a shift lens. The image blur correction unit 105 is a first image blur correction means, and the optical image blur correction control section 106 performs drive control. A focus unit 107 includes a focus lens that performs focus adjustment to form a subject image. A focus drive control unit 108 drives and controls the focus unit 107 .

A lens operation unit 109 is an operation unit used by the user to operate the lens apparatus. A lens shake detection unit 110 detects the amount of shake applied to the lens device and outputs a detection signal to the lens control unit 111 . A lens control unit 111 that controls the entire lens device includes a CPU (Central Processing Unit), and performs integrated control of each drive control unit and correction control unit of the lens device. The lens control unit 111 communicates with the control unit of the camera body through the lens communication control unit 112 .

Next, the camera body will be described. The camera main body has a shutter unit 113 . The shutter drive control section 114 drives and controls the shutter unit 113 . The imaging unit 115 includes an imaging element, photoelectrically converts an optical image formed by passing through each lens group, and outputs an electric signal (also referred to as an image signal). The imaging signal processing unit 116 converts the electrical signal output from the imaging unit 115 into a video signal. The video signal processing unit 117 processes the video signal output from the imaging signal processing unit 116 according to the application. For example, the clipping position of the video signal is changed according to the correction amount of the electronic image blur correction control unit 123 . The electronic image stabilization control unit 123 (an example of stabilization control means) is a second image stabilization means, and performs image stabilization control (an example of stabilization control) by clipping an image. . Note that the second image blur correction is not limited to electronic image blur correction, and includes image blur correction by drive control of an image sensor and image blur correction by drive control of a movable optical element in the camera body.

The display unit 118 displays an image as necessary based on the signal output from the video signal processing unit 117 . The storage unit 119 stores various data such as video information. The power supply unit 120 supplies power to the entire system according to the application. The camera operation unit 121 is used by the user to operate the camera system, and outputs operation signals to the camera control unit 124 . A camera shake detection unit 122 detects camera shake such as hand shake applied to the camera main unit by a motion vector, and outputs a camera motion signal corresponding to the camera shake to the camera control unit 124 . The camera control unit 124 has a CPU and controls the entire camera system. The camera control section 124 communicates with the lens communication control section 112 of the lens apparatus via the camera communication control section 125 . That is, mutual communication is performed by the lens communication control unit 112 and the camera communication control unit 125 in a state in which the lens device is attached to the camera body and electrically connected.

Next, a schematic operation of the imaging system having the above configuration will be described.

The lens operation unit 109 and the camera operation unit 121 include an image blur correction switch that can select ON/OFF of image blur correction. When the user operates the image blur correction switch to select image blur correction ON, the lens control unit 111 or the camera control unit 124 instructs the optical image blur correction control unit 106 or the electronic image blur correction control unit 123 to correct the image blur. Instructs corrective action. Each image blur control unit controls the image blur correction until an instruction to turn off the image blur correction is given.

The camera operation unit 121 also includes an image blur correction mode switch that can select a first mode or a second mode for image blur correction. The first mode is a mode in which image blur correction is performed only by optical image blur correction (first image blur correction). The second mode is a mode in which image blur correction is performed using both optical image blur correction and electronic image blur correction (second image blur correction). When the first mode is selected, the readout position of the image pickup unit 115 is fixed, and by widening the readout range accordingly, wider-angle imaging can be handled. Further, when the second mode is selected, instead of narrowing the clipping range of the video signal by the video signal processing unit 117, by changing the clipping position according to the image blur correction amount, it is possible to cope with a larger image blur. .

The camera operation unit 121 includes a shutter release button configured to turn on the first switch (SW1) and the second switch (SW2) in order according to the amount of depression. The first switch SW1 is turned on when the user presses the shutter release button about halfway, and the second switch SW2 is turned on when the user presses the shutter release button all the way. When SW1 is turned on, the focus drive control section 108 drives the focus unit 107 to perform focus adjustment, and the aperture drive control section 104 drives the aperture unit 103 to set an appropriate exposure amount. Image data obtained from the optical image exposed to the imaging unit 115 is stored in the storage unit 119 by turning on the SW2.

The camera operation unit 121 also includes a moving image recording switch. The camera starts recording a moving image after the moving image recording switch is pressed, and ends recording when the user presses the moving image recording switch again during recording. When the user operates the shutter release button during moving image shooting to turn on SW1 and SW2, a process of acquiring and recording a still image during moving image recording is executed. Also, the camera operation unit 121 includes a reproduction mode selection switch that can select a reproduction mode. When the playback mode is selected by operating the playback mode selection switch, the camera stops anti-vibration (image blur correction) operation.

Image blur correction control in the imaging system will be described with reference to FIGS. 2 and 14. FIG. FIG. 2 is a block diagram illustrating in more detail a part related to image blur correction control in the entire imaging system. FIG. 14 is a diagram for explaining the Pitch direction, Yaw direction, and Roll direction. A lens shake detection unit 110 in FIG. 2 detects angular velocity data using a gyro sensor as a shake detection sensor, and outputs a detected voltage. The lens shake detection unit 110 has a pitch direction shake detection sensor and a yaw direction shake detection sensor. Also, the camera shake detection unit 122 performs shake detection using a motion vector. As shown in FIG. 14, the optical axis of the imaging optical system in the imaging apparatus is defined as the Z axis, the vertical direction at the normal position as the Y axis, and the direction orthogonal to the Y and Z axes as the X axis. Therefore, the Pitch direction is the direction around the X-axis (tilting direction), the Yaw direction is the direction around the Y-axis (panning direction), and the Roll direction is the direction around the Z-axis (the imaging plane rotates on the plane perpendicular to the optical axis). direction). In other words, the Pitch direction is the tilting direction with respect to the horizontal plane in the vertical direction of the imaging device, and the Yaw direction is the tilting direction with respect to the vertical plane in the horizontal direction of the imaging system, which directions are perpendicular to each other.

The pitch direction shake detection sensor detects shake information corresponding to shake in the pitch direction. The Yaw direction shake detection sensor detects shake information corresponding to shake in the Yaw direction. Each shake information is acquired as angular velocity data. In FIG. 2, since the configuration is the same in the pitch direction and the yaw direction, only one axis will be described.

The lens shake detection unit 110 outputs angular velocity data obtained by an angular velocity sensor such as a gyro sensor as a detection voltage. The angular velocity detection AD converter 201 converts the detection signal output by the lens shake detector 110 into digital data. A high-pass filter 202 removes offset components and temperature drift components from the angular velocity data and outputs the data to the integrating section 203 . The integrator 203 mainly integrates the angular velocity data by pseudo-integration using a low-pass filter, and converts it into angle data. An optical image blur correction sensitivity multiplication unit 204 converts the angle data acquired from the integration unit 203 into a drive control amount (shift amount) of the image blur correction lens. This drive control amount is also called an image blur correction amount. The value of this sensitivity is changed each time the focal length of the imaging optical system changes. The sensitivity also reflects the amount of correction by sensitivity adjustment of the angular velocity sensor, and sensitivity variations are absorbed.

A division unit 205 divides the image blur correction amount output from the sensitivity multiplication unit 204 into two. The image blur correction amount is divided into an optical image blur correction amount applied to optical blur correction and an electronic image blur correction amount applied to electronic blur correction. The dividing unit 205 multiplies the image blur correction amount by a coefficient (denoted as K) in order to calculate the optical image blur correction amount. The coefficient K is determined by the following formula (1) based on the movable range of optical image blur correction (denoted as A) and the movable range of electronic image blur correction (denoted as B) at each focal length. The movable range is the range in which image blur correction can be controlled, and corresponds to the range in which drive control of the image blur correction unit 105 is possible in the case of optical image blur correction. In the case of electronic image blur correction, this corresponds to a range in which correction processing by clipping an image is possible.
K=A/(A+B) (1)

From equation (1), K takes a value of 1 or less. That is, by multiplying the coefficient K, the correction amount of the optical image blur correction (first image blur correction amount) is calculated with respect to the total image blur correction amount.

The optical image blur correction amount limiter unit 206 clamps the first image blur correction amount within the movable range of the image blur correction unit 105 . By doing so, it is possible to prevent a situation in which the correction lens reaches the end of the movable range of the optical image blur correction (the limit position of the drive control range). The output of limiter section 206 is input to subtraction section DEC.

The PID control unit 207 performs position control of the image blur correction lens according to the input from the subtraction unit DEC. Position control is performed by a combination of P (proportional) control, I (integral) control, and D (differential) control. A driver unit 208 supplies a current for driving the image blur correction unit 105 according to a control signal from the PID control unit 207 corresponding to the first image blur correction amount. The image blur correction unit 105 has an electromagnetic actuator and drives a movable unit including an image blur correction lens. A position detection unit 209 detects the position of the image blur correction unit 105 and outputs a detection voltage. The AD conversion section 210 converts the analog detection voltage output from the position detection section 209 into digital data, and outputs the digital data to the subtraction section DEC. The subtraction unit DEC calculates the difference (deviation) between the outputs of the limiter unit 206 and the AD conversion unit 210 and outputs it to the PID control unit 207 . Feedback control is thereby performed.

On the other hand, the division unit 205 multiplies the image blur correction amount output from the sensitivity multiplication unit 204 by a coefficient “1−K” in order to calculate the electronic image blur correction amount. The optical image blur correction amount is multiplied by the coefficient K, whereas the electronic image blur correction amount is divided by multiplication of the coefficient "1-K". The angle conversion unit 211 converts the electronic image shake correction amount (second image shake correction amount) into angle data. This conversion factor has a different value for each focal length and is changed each time the focal length changes. The converted data is transmitted to the electronic image blur correction control section 123 via the lens communication control section 112 and the camera communication control section 125 . The electronic image shake correction control unit 123 performs electronic image shake correction control according to the second image shake correction amount and the electronic image shake correction amount based on the shake amount obtained by the camera shake detection unit 122 . A movable range for image blur correction will be specifically described with reference to FIG. 11 .

FIG. 11 is a graph showing the relationship between the focal length of the camera and the movable range of image blur correction. The horizontal axis represents the focal length f, indicating the Wide (wide-angle) end, the Middle (middle) position, and the Tele (telephoto) end. The vertical axis represents the movable range (unit: degree). Graph lines (a), (b), and (c) respectively indicate the movable range A for optical image stabilization, the movable range B for electronic image stabilization, and the overall movable range for image stabilization (A+B). there is That is, there is a relationship of (a)+(b)=(c).

The movable range A of the optical image blur correction is determined by the optical characteristics of the photographing lens, and the movable range B of the electronic image blur correction is determined by the surplus pixels of the image sensor. The correction angles of both the movable range A for optical image blur correction and the movable range B for electronic image blur correction change depending on the zoom state. That is, even when the same camera shake is applied, the drive amount of the image blur correction unit 105 for correcting image blur differs depending on the zoom position (optical zoom magnification, focal length). Even if the same 1 degree shake is applied to the camera, the amount by which the shift lens of the image blur correction unit 105 moves at the Wide end to correct the image blur caused by this 1 degree shake is greater than the amount the shift lens moves at the Tele end. less than the amount to be Both the movable range A for the optical image blur correction and the movable range B for the electronic image blur correction change with the focal length f, and both are managed as angle-converted data in controlling the image blur correction.

12 is a graph showing the relationship between the focal length and the coefficient K. FIG. As in FIG. 11, the horizontal axis represents the focal length f, and the vertical axis represents the coefficient K for dividing the image blur correction amount. The coefficient K is determined by the movable range A for optical image blur correction and the movable range B for electronic image blur correction. Further, since the optical image blur correction and the electronic image blur correction operate respectively, there is no boundary between the movable ends of the optical image blur correction and the electronic image blur correction. As a result, image distortion due to overshoot of the optical image blur correction is suppressed.

The Wide end, Middle position, and Tele end shown in FIG. 12 will be specifically described. In the optical image blur correction, the image blur correction lens moves within the movable range A, and in the electronic image blur correction, image processing is performed within the movable range B. Together with these corrections, image blur corresponding to the entire image blur correction movable range can be corrected. As an example, the movable range A for optical image stabilization is set to (2, 0.75, 0.3) at the Wide end, Middle position, and Tele end, respectively. The movable range B of electronic image blur correction is set to (2.5, 1.6, 1.1) at the Wide end, Middle position, and Tele end, respectively. The units of the movable ranges A and B are degrees. In this case, the value of the coefficient K is (0.444, 0.319, 0.214) at the Wide end, Middle position, and Tele end, respectively.

When the second mode for performing optical image blur correction and electronic image blur correction is set, the correction lens is driven with the image blur correction amount obtained by multiplying K=A/(A+B), and the coefficient "1 -K' is multiplied to change the image blur correction amount. On the other hand, when the first mode in which only the optical image blur correction is performed is set, the division unit 205 sets the value of the coefficient K to 1. That is, drive control of the correction lens is performed with the total image blur correction amount as the optical image blur correction amount. Since electronic image blur correction is not performed, the value of the coefficient "1-K" relating to the electronic image blur correction amount is zero.

Next, still image shooting in the second mode will be described. When the shutter release button of the camera operation unit 121 is operated to turn on the second switch SW2, a still image exposure operation is performed. The dividing unit 205 sets the value of the coefficient K to one. The total image blur correction amount is the optical image blur correction amount. Since electronic image blur correction is not performed during still image exposure, the value of the coefficient "1-K" relating to the electronic image blur correction amount is zero. At the end of the still image exposure operation, the dividing unit 205 sets the coefficient K=A/(A+B) for the optical image blur correction, and sets the coefficient "1-K" for the electronic image blur correction. At the start and end of the still image exposure operation, in order to avoid sudden changes in the amount of correction for optical and electronic image blur correction, a predetermined output time is provided and the correction output is gradually changed. processing takes place.

FIG. 3 is a block diagram showing in detail the configuration of the electronic image blur correction control section 123. As shown in FIG.

The camera communication control unit 125 receives the electronic image blur correction amount from the lens device through communication. The electronic image blur correction amounts in the pitch direction and the yaw direction are transmitted as correction amounts converted into angles. A pixel conversion unit 301 converts the electronic image blur correction amount into a pixel-converted correction amount (the number of pixels), and outputs the pixel-converted correction amount to the limiter 305 . The transform coefficients have different values for different focal lengths and are changed whenever the focal length changes.

A camera shake detection unit 122 performs shake detection using a motion vector and outputs a detection signal to the limiter 302 . The motion vector output is the amount of movement in the up, down, left, and right directions with respect to the imaging plane. A limiter 302 generates an electronic image blur correction amount using the shake information detected by the lens and the shake information detected by the camera, and clamps the clipping range of the electronic image blur correction. A limiter 302 processes the outputs of the pixel conversion unit 301 and the camera shake detection unit 122 . That is, each level of the limiter is set for each pitch direction and each yaw direction. The correction amount after limit processing is input to the electronic image blur correction amount setting unit 303 . An electronic image blur correction amount setting unit 303 sets an electronic image blur correction amount in each correction axis direction.

[First embodiment]
A first embodiment of the present invention will be described below.

Referring to FIG. 4, detection timing and ID information of lens-side detected shake information (an example of shake-proof information) and detection timing and ID information of camera-side detected shake information (an example of shake-proof information) are shown. explain. In order to perform optical image blur correction and electronic image blur correction, it is necessary to transmit the exposure barycenter timing (406) of the imaging unit 115 from the camera body to the lens device. However, the communication between the camera body and the lens device includes many communications for AF (automatic focus adjustment), AE (automatic exposure), etc., in addition to image blur correction. If communication timing varies due to overlap with other communication, and accurate exposure center-of-gravity timing cannot be communicated, processing may be hindered. Therefore, in the present embodiment, in order to avoid the communication timing deviation, the communication processing is divided into two times, the reference time and the relative time, so that the camera body section conveys the exposure center-of-gravity timing to the lens device.

Moreover, if the amount of information transmitted and received in communication between the camera body and the lens device is large, it becomes difficult to complete the processing within the specified time. Furthermore, in order to deal with various interchangeable lenses, it is necessary to control without being conscious of individual lens specifications. Therefore, in the present embodiment, the lens apparatus mainly performs control and communicates angle conversion data for image blur correction.

VD shown in FIG. 4 indicates the timing of the vertical synchronizing signal, and V_BLK indicates the timing of the vertical blanking period. "CMOS drive" indicates the drive state of the imaging element. "C⇔L communication" indicates communication between the camera body (C) and the lens device (L). "C blurring" at the bottom indicates detection of camera body shake. A communication timing 404 of the first communication 401, a timing 405 for determining the exposure time, and an exposure center-of-gravity timing 406 are shown. F[n] is an index representing the n-th frame, and indicates timing 408 at which the frame number is notified.

Each time shown in FIG. 4 is as follows.
BT: Length of vertical blanking period IT: Image time AT: Time from communication timing 404 to timing 405 ET: Exposure time DT: Delay time from the center of the exposure period to exposure barycenter timing 406 .

The exposure center-of-gravity timing 406 with reference to the timing 405 is calculated by "IT+BT-ET/2+DT" based on the center of the exposure period.

In each frame, the center-of-gravity position of the parallelogram corresponds to the exposure center-of-gravity timing 406, and the smaller the exposure amount, the smaller the area of the parallelogram. When the exposure time ET (upper right vertex of the parallelogram) elapses from the start of exposure (upper left vertex of the parallelogram), signal readout of the image sensor starts.

Starting from the vertical synchronization signal of the imaging unit 115, a first communication 401 is performed from the camera body to the lens device. The first communication 401 serves as a reference for transmitting the exposure center-of-gravity timing 406 from the camera body to the lens apparatus. The lens apparatus acquires the timer time in the lens apparatus at the timing of receiving the information by the first communication 401, and uses that time as the reference time for calculating the exposure barycenter timing. As for the communication timing 404 of the first communication 401, communication may be performed at the same timing as the vertical synchronization signal, or may be performed before or after an arbitrary time from the vertical synchronization signal. However, every frame, communication is performed with a constant time difference from the vertical synchronization signal. Also, the communication timing 404 of the first communication 401 is set to timing that does not overlap with other communication. In the example shown in FIG. 4, the communication timing 404 of the first communication 401 is set at a point before (past) the vertical synchronization signal.

Next, a second communication 402 is made from the camera body to the lens device. In the second communication 402, information on the relative time 407 from the communication timing 404 of the first communication 401 to the exposure barycenter timing 406 with the first communication 401 as a reference is transmitted to the lens apparatus. In the second communication 402, the movable range B of electronic image blur correction at the current focal length and ID information for determining shake information are transmitted. An imaging frame number corresponding to the timing at which shake is detected is set in this ID information. Thereby, unique ID information is set. The communication timing of the second communication 402 is after the timing 405 at which the exposure time of the corresponding frame that conveys the exposure barycenter is determined. As a result, even when the exposure time varies for each frame, the accurate exposure center-of-gravity timing 406 can be transmitted to the lens apparatus. An exposure center-of-gravity timing 406 is obtained based on the determined exposure time and the signal readout time of the image sensor, and a relative time 407 is obtained from the difference based on the communication timing 404 of the first communication 401 . That is, the relative time 407 is calculated by "AT+IT+BT-ET/2+DT". Note that the timing 405 that determines the exposure time of each frame is not fixed. Also, the communication timing of the second communication 402 is before the timing 408 at which the frame number is notified. For this reason, ID information for discriminating shake information is set with a frame number next to an image pickup frame number that has been notified so far as an image pickup frame number corresponding to the timing of detecting shake.

The lens apparatus receives the information of the relative time 407 through the second communication 402 with reference to the reception timing of the first communication 401 . As a result, the exposure center-of-gravity timing 406 can be grasped by timer setting in the lens apparatus. In addition, the lens apparatus receives the movable range B of the electronic image blur correction through the second communication 402, so that the coefficient used in the dividing unit 205 including the movable range A of the optical image blur correction of the lens apparatus itself is K can be calculated. In the lens apparatus, the lens shake detection unit 110 detects shake information at the exposure center-of-gravity timing 406, and the division unit 205 divides the total image shake correction amount into the optical image shake correction amount in the lens apparatus and the camera body unit. It is distributed to the electronic image stabilization amount in . Until there is a communication request from the camera control unit 124, the lens control unit 111 holds in memory together with the distributed electronic image blur correction amount and the ID information for determining the shake information transmitted by the second communication 402. .

On the other hand, the camera body section also acquires the camera body shake information by the camera shake detection section 122 . The camera shake information detected in this embodiment is a motion vector. Prior to this, the camera body sets ID information for discriminating shake information. An imaging frame number corresponding to the timing at which shake is detected is set in this ID information. Thereby, unique ID information is set. The timing for setting the ID information is timing 409, which is timing after timing 408 for notifying the frame number. Therefore, the imaging frame number notified at the timing 408 when the frame number is notified is set as it is in the ID information for determining the shake information. Further, it becomes possible to acquire the shake information of the camera main unit at 410, which is the timing at which the detection of the motion vector for the frame in which the shake is to be detected is completed. Here, the shake information of the camera main body can be acquired, and the previously set ID information for discriminating the shake information is acquired.

Next, a third communication 403 is performed from the camera body to the lens device. In the third communication 403, upon receiving a communication request from the camera control unit 124, the lens control unit 111 sends the camera control unit 124 the sorted lens-side detected shake information (in this embodiment, the sorted electronic image shake correction amount) and ID information for discriminating the shake information transmitted by the second communication 402 are transmitted. As a result, shake information detected on the lens side is obtained together with ID information. The communication timing of the third communication 403 is after the exposure center-of-gravity timing 406 . Since the camera control unit 124 originally grasps the exposure center-of-gravity timing 406, communication is performed at an arbitrary timing after the exposure center-of-gravity timing.

The camera main unit determines whether the ID information of the lens-side detected shake information received from the lens control unit 111 and the camera-side detected shake information detected by the camera shake detection unit 122 match each other. If they match, the camera electronic image stabilization amount generated from the two pieces of shake information is passed to the electronic image stabilization control unit 123, and finally the electronic image stabilization amount setting unit 306 sets the compensation amount. . In this case, the generated camera electronic image blur correction amount and the previous ID information are set in association with each other.

The first communication, the second communication, and the third communication are performed in each frame, and the camera control section 124 notifies the lens control section 111 of the reference time through the first communication 401 . In the second communication 402, the camera control unit 124 notifies the ID information for discriminating the shake information, the relative time from the reference time, and the movable range of the electronic image stabilization. Through third communication 403 , the camera control unit 124 acquires ID information for discriminating shake information and lens-side detected shake information from the lens control unit 111 . On the other hand, the lens control unit 111 acquires the reference time in each frame through the first communication 401, and transmits ID information for discriminating shake information, relative time 407 from the reference time, and electronic image blur correction through the second communication. is obtained, and the amount of electronic image blur correction at the timing 406 of the center of gravity of exposure is distributed. The lens control unit 111 notifies the camera control unit 124 of the distributed electronic image blur correction amount and the ID information for determining the shake information through the third communication 403 .

The processing of this embodiment will be described with reference to FIGS. 5 and 6. FIG. FIG. 5 is a flow chart regarding communication and control contents performed by the camera control unit 124 . FIG. 6 is a flowchart relating to communication and control contents performed by the lens control unit 111 . The following processing is realized according to a predetermined program read from the memory and executed by the CPU of each control unit.

The processing shown in FIG. 5 is performed by the camera control unit 124 , and communication processing is performed with the lens control unit 111 via the camera communication control unit 125 and the lens communication control unit 112 .

In S<b>101 , the camera control unit 124 performs first communication 401 with the lens control unit 111 . The communication timing 404 of the first communication 401 becomes the reference time for the exposure center-of-gravity timing 406 .

Next, in S102, a second communication 402 is performed. By transmitting the relative time 407 from the reference time in the first communication 401, the exposure center-of-gravity timing 406 is transmitted. Then, the movable range of the electronic image blur correction at the current focal length is transmitted. Furthermore, the first ID information for discriminating the shake information is transmitted. An imaging frame number corresponding to the timing at which shake is detected is set in this ID information. As described above, the imaging frame number to be set is corrected and set in accordance with the timing of notification of the imaging frame number and the timing of detecting shake.

In S<b>103 , the camera control unit 124 sets second ID information for determining shake information of the camera body detected by the camera shake detection unit 122 . The ID information is also set with the imaging frame number corresponding to the timing of detecting the shake. As described above, the imaging frame number to be set is corrected and set in accordance with the timing of notification of the imaging frame number and the timing of detecting shake.

In S<b>104 , the camera control unit 124 determines whether or not a certain period of time (also referred to as a predetermined period of time) has passed since the exposure center-of-gravity timing 406 . The fixed time is a time set in advance from the exposure barycenter timing 406 . If the predetermined time has passed since the exposure center-of-gravity timing 406, the process proceeds to S105, and if not, the process returns to S104 and repeats the process. The reason why the camera controller 124 waits for the predetermined period of time to pass in S104 is that the camera controller 124 issues a communication request to the lens controller 111 after the lens controller 111 has completed the control processing at the exposure center-of-gravity timing 406 .

In S105, the camera control unit 124 executes third communication. That is, it transmits a communication request to the lens control unit 111 . Then, as a response to this communication request, the lens-side detected shake information distributed by the lens control unit 111 at the exposure center-of-gravity timing 406 and the first ID information for discriminating the shake information notified in S102, Received from the lens control unit 111 .

In the next step S106, the camera control unit 124 determines whether the camera shake information detected by the camera shake detection unit 122 has been detected. Specifically, it is determined by a motion vector detection completion interrupt. If shake information of the camera body is detected, the process proceeds to S107, and if not detected, the process returns to S106 and repeats the process.

In S107, the camera control unit 124 acquires the shake information of the camera body detected by the camera shake detection unit 122 and the second ID information for discriminating the shake information set in S103.

In S108, the camera control unit 124 retrieves the first ID information associated with the lens-side detected shake information acquired in S105 and the second ID information associated with the camera-side detected shake information acquired in S107. It is determined whether the ID information matches the ID information of . If the first ID information and the second ID information match, the process proceeds to S109, and if the shake information ID information does not match, the process returns to S101 and repeats the process. If the ID information of the shake information does not match, the previously set electronic image blur correction amount may be set. If the ID information mismatch continues for a predetermined number of times, it is determined that some kind of error such as a communication error has occurred, and reset processing (processing equivalent to turning off the power and then turning on the power again) is performed. You can do it.

In S109, the camera control unit 124 generates an electronic image blur correction amount using the lens-side detected shake information acquired in S105 and the camera-side detected shake information acquired in S107. More specifically, the inter-frame difference of the lens-side detected shake information is subtracted from the camera-side detected shake information and added to the lens-side detected shake information to generate the electronic image shake correction amount. When the electronic image blurring correction amount is generated, the generated electronic image blurring correction amount and the ID information of the vibration information are set in association with each other.

Finally, in S110, the camera control unit 124 instructs the electronic image blur correction control unit 123 to perform an image blur correction operation based on the electronic image blur correction amount generated in S109.

6 is the lens control unit 111, and communication processing is executed with the camera control unit 124 via the lens communication control unit 112 and the camera communication control unit 125. FIG. In S<b>201 , the lens control unit 111 receives the first communication 401 . At the communication timing 404 of the first communication 401, the process of acquiring the timer time in the lens apparatus is executed, and that time becomes the reference time for calculating the exposure center-of-gravity timing.

Next, in S202, the lens control unit 111 receives the second communication 402, and calculates the relative time 407 from the reference time at the communication timing 404 of the first communication 401, the movable range of the electronic image blur correction, and the shake information. 1st ID information for discriminating is acquired. Since the lens control unit 111 acquires the reference time at the communication timing 404 of the first communication 401, it receives the relative time 407 through the second communication and sets the exposure center-of-gravity timing by timer setting. Also, the lens control unit 111 acquires the movable range of the electronic image blur correction. The coefficient K used by the dividing unit 205 is calculated including this movable range and the movable range of the optical image blur correction of the lens apparatus itself, and the respective correction amounts related to the optical image blur correction and the electronic image blur correction are calculated. set.

In S203, the lens control unit 111 determines whether or not the exposure center-of-gravity timing set by the timer in S202 has arrived. If the exposure center-of-gravity timing 406 has arrived, the process proceeds to S204, and if the exposure center-of-gravity timing 406 has not arrived, the determination process of S203 is repeated.

In step S204, the lens control unit 111 acquires shake information from the lens shake detection unit 110 at the exposure barycenter timing 406, and the division unit 205 calculates the total amount of image shake correction according to the value of the coefficient K, and calculates the optical image shake. It is divided into the amount of correction and the amount of electronic image stabilization. Until the camera control unit 124 acquires the communication request transmitted in S105, the lens control unit 111 stores the first ID information for determining the distributed electronic image stabilization amount and the shake information acquired in the previous S202. temporarily stored together with

In S205, the lens control unit 111 determines whether or not the communication request for the third communication 403 transmitted from the camera control unit 124 in S105 has been received. If the communication request for the third communication has been received, the process proceeds to S206, and if the communication request has not been received, the determination process of S205 is repeated. In S206, as a response to the reception of the communication request of the third communication 403 transmitted in S105, the lens control unit 111 transmits the lens-side detected shake information sorted in S204 and the shake information acquired in S202. First ID information is notified to the camera control unit 124 for determination.

<Effects of First Embodiment>
In this embodiment, the timing information of the exposure period is not transmitted in one communication, but the reference time and the relative time are notified from the camera body to the lens device in the first and second communications, respectively. As a result, even if the communication timing of the second communication varies due to the influence of other communication, the reference time of the first communication and the relative time of the second communication are transmitted to the lens device. Accurate exposure center-of-gravity timing can be transmitted.

Also, even if the first communication overlaps with another communication and the communication is delayed, the relative time of the second communication can be set in consideration of the delay time when issuing the first communication. That is, the exposure center-of-gravity timing can be notified based on the relative time information corrected based on the delay time. Therefore, even if the communication timing of the first communication varies, more accurate exposure center-of-gravity timing can be transmitted from the camera body to the lens device.

The movable range of electronic image blur correction and the amount of electronic image blur correction are notified as angle conversion data. The camera control unit 124 transmits the movable range of electronic image stabilization to the lens control unit 111 and acquires the electronic image stabilization amount from the lens control unit 111 .

Furthermore, since the camera control unit 124 transmits or sets ID information for determining shake information for the lens-side shake information and the camera-side shake information, respectively, the ID information for the two pieces of shake information must match. synchronizing the shake information.

According to this embodiment, in a lens-interchangeable imaging system, a plurality of correction means for optical image blur correction and electronic image blur correction and a lens The correction of the angular velocity sensor on the side and the motion vector on the camera side by a plurality of shake detection means can be controlled in a coordinated and synchronized manner. This makes it possible to provide an imaging system that expands the image blur correction range.

[Second embodiment]
Next, a second embodiment of the invention will be described. In the present embodiment, by using the same reference numerals as those of the first embodiment, the description thereof will be omitted, and mainly the differences will be described. Such omission of description also applies to embodiments described later.

In the first embodiment, after the camera control unit 124 transmits the timing information of the exposure period to the lens control unit 111, the lens control unit 111 acquires the lens-side detected shake information sorted by the exposure center-of-gravity timing. Further, the camera shake such as hand shake applied to the camera body is detected by a motion vector. On the other hand, in the present embodiment, the camera shake detection unit 122 has an angular velocity sensor (not shown), and the angular velocity sensor is used to correct the image shake in the direction of the correction axis (Roll direction) other than the distribution target. You can increase the effect.

The contents of the control of this embodiment will be described with reference to the flowchart of FIG. FIG. 7 shows an example of control including communication processing and shake detection processing in the camera body. First, in S<b>301 , the camera control unit 124 performs first communication with the lens control unit 111 via the camera communication control unit 125 . The time of the first communication timing becomes the reference time of the exposure center-of-gravity timing. Next, in S302, a second communication 402 is performed. By transmitting the relative time 407 from the reference time in the first communication 401, the exposure center-of-gravity timing 406 is transmitted. Then, the movable range of the electronic image blur correction at the current focal length is transmitted. Furthermore, the first ID information for discriminating the shake information is transmitted. An imaging frame number corresponding to the timing at which shake is detected is set in this ID information. The imaging frame number to be set is corrected and set in accordance with the timing of notification of the imaging frame number and the timing of detecting the shake.

In S<b>303 , the camera control unit 124 sets second ID information for determining shake information of the camera body detected by the camera shake detection unit 122 . The ID information is also set with the imaging frame number corresponding to the timing of detecting the shake. The imaging frame number to be set is corrected and set in accordance with the timing of notification of the imaging frame number and the timing of detecting the shake.

In S304, the camera control unit 124 determines whether or not the exposure center-of-gravity timing has arrived by timer setting. If it is determined that the exposure center-of-gravity timing has arrived, the process proceeds to S305, and if it has not arrived, the determination processing of S304 is repeated.

In S305, the camera control unit 124, at the exposure center-of-gravity timing, acquires the second ID information for distinguishing between the shake information of the camera body unit detected by the camera shake detection unit 122 and the shake information set in S303. get. The shake information detected here is shake information about a correction axis different from the correction axis related to the electronic image blur correction amount acquired from the lens control unit 111 in S307, that is, detection information of sensors with different detection directions. In this embodiment, it is the shake information of the correction axis in the Roll direction.

In S306, the camera control unit 124 determines whether or not a certain period of time has passed since the exposure center-of-gravity timing. If it is determined that the predetermined time has passed since the timing of the center of gravity of exposure, the process proceeds to S307, and if not, the determination process of S306 is repeated. The reason for waiting for the lapse of the predetermined time in S306 is as explained in S104 of FIG.

In S<b>307 , the camera control unit 124 performs third communication with the lens control unit 111 via the camera communication control unit 125 . That is, the camera control section 124 transmits a communication request to the lens control section 111 . Then, in response to the reception of this communication request by the lens control unit 111, the lens control unit 111 discriminates between the lens-side detected vibration information distributed by the lens control unit 111 at the exposure center-of-gravity timing 406 and the vibration information notified in S302. acquire first ID information for Here, the lens-side detected shake information is shake information about a correction axis different from the shake information detected in S304, specifically, shake information about the correction axes in the Pitch and Yaw directions.

In S308, the camera control unit 124 determines whether the ID information of the camera-side detected shake information acquired in S305 matches the ID information of the lens-side detected shake information acquired in S307. If the ID information of the shake information matches, the process proceeds to S309, and if the ID information of the shake information does not match, the process returns to S301 and repeats the process. As in S108, if the ID information of the shake information does not match, the previously set electronic image blur correction amount may be set. Further, when the ID information mismatch of the shake information continues for a predetermined number of times, it is determined that some kind of error such as a communication error has occurred, and reset processing (processing equivalent to turning off the power and then turning on the power again) is performed. You can do it.

In S309, the camera control unit 124 generates an electronic image blur correction amount using the camera-side detected shake information acquired in S305 and the lens-side detected shake information acquired in S307. At that time, the generated electronic image blur correction amount and the ID information of the shake information are set in association with each other.

Finally, in S310, the camera control unit 124 instructs the electronic image stabilization control unit 123 to perform an image stabilization operation based on the electronic image stabilization amount generated in S309.

<Effects of Second Embodiment>
In this embodiment, the amount of electronic image stabilization distributed by the lens control unit 111 and the shake information detected by the angular velocity sensor of the camera body are acquired to perform image stabilization control. The electronic image shake correction amount is the amount of correction in the Pitch direction and the Yaw direction, and the camera body shake information is the shake detection information in the correction axis direction (Roll direction) that is not distributed.

The camera control unit 124 transmits or sets ID information for discriminating shake information with respect to the lens side shake information and the camera side shake information, respectively. Therefore, it is possible to synchronize the shake information in the three axial directions by matching the ID information of the shake information of the camera body (Roll direction) and the shake information of the lens side (Pitch direction and Yaw direction). Since the electronic image blur correction in the three axial directions is performed based on the shake information whose timing is synchronized in this way, the image blur correction effect can be enhanced.

[Third Embodiment]
Next, in a third embodiment of the present invention, application to rolling shutter distortion (hereinafter referred to as RS distortion) correction as other electronic correction will be described.

The exposure method of the imaging unit 115 includes a global shutter method and a rolling shutter method. In a global shutter type device represented by a CCD (charge-coupled device) image sensor, the exposure time between pixels and the exposure start time are substantially the same in one frame image. In an apparatus having a CMOS (Complementary Metal Oxide Semiconductor) image sensor, the exposure method is a rolling shutter method.

In the rolling shutter method in which the exposure timing differs for each pixel line, image distortion (RS distortion) occurs due to deviations in exposure timing and signal readout time for each line. Imager shake affects the signal reading line by line, causing RS distortion. Even if the imaging device is installed on a tripod or the like, RS distortion occurs when vibration is applied to the device due to disturbance such as wind. Since the RS distortion is a distortion that occurs in the captured image due to the exposure timing being different for each pixel line, it is possible to correct the movement amount of each pixel line as the correction amount based on the shake signal of the imaging device. .

RS distortion correction will be described with reference to the conceptual diagram of FIG. 11 . An image 1301 (dotted rectangular frame) before RS distortion and an image 1302 (solid parallelogram frame) after RS distortion are illustrated assuming that the imaging device moves in the horizontal direction. The diagram on the right illustrates by a plurality of points 1303 the amount of horizontal movement (shake amount) of the imaging device that occurs during the exposure period. The horizontal axis represents pixel positions, and the vertical axis corresponds to the time axis. In the illustrated example, 11 points are exemplified.

In an imaging device, the horizontal movement amount (shake amount) of the device that occurs during an exposure period is calculated at a plurality of points. These points are also called correction points. By interpolating between a plurality of points, the amount of movement of each line is acquired as a correction amount, and correction processing is performed by changing the reading position for each line with respect to horizontal shake. That is, the lens shake detection unit 110 (FIG. 2) detects shake that causes RS distortion. The video signal processing unit 117 electronically corrects the RS distortion. Calculation of the correction amount for correcting the RS distortion is performed by the division unit 205 in the same manner as the electronic image blur correction amount, and the image blur correction amount is multiplied by the coefficient “1−K”. The camera communication control unit 125 (FIG. 3) acquires the RS distortion correction amount from the lens device. A pixel conversion unit 301 converts the electronic image blur correction amount transmitted in angle conversion into a pixel conversion value. The conversion coefficients used at this time have different values for each focal length, and are changed each time the focal length changes. Furthermore, the limiter 302 performs clamping within the movable range of RS distortion correction. A limit value is set for each correction axis direction. An electronic image blur correction amount setting unit 303 sets an RS distortion correction amount in each correction axis direction.

Next, communication regarding RS distortion correction performed between the lens communication control unit 112 and the camera communication control unit 125 and its timing will be described with reference to FIG. VD, V_BLK, etc. are the same as in FIG.

In order to correct the RS distortion, it is necessary to transmit the exposure barycenter timing (1009) of the imaging unit 115 from the camera body to the lens device. As described above, in the RS distortion correction, there are a plurality of correction points (in this embodiment, for example, 11 points obliquely continuing from 1006 to 1010 with respect to the frame corresponding to the exposure barycenter timing 1009). As the time relative to the reference time, the exposure center-of-gravity timing 1009 and the difference between the leading correction timing (the correction timing corresponding to the correction point at which the correction timing arrives first among the plurality of correction points) and the next correction point. A time (RS distortion corrected leading delta time 1008, described below) is conveyed. Also in this embodiment, in order to avoid communication timing deviation, communication is performed in two parts, the reference time and the relative time.

In FIG. 8 , the first communication 1001 is indicated by communication timing 1004 . The second communication 1002 is executed after the timing 1005 at which the exposure time of the relevant frame to convey the exposure barycenter is determined. In the second communication, the lens control unit 111 transmits the exposure center-of-gravity timing 1009 for RS distortion correction and the RS distortion correction head differential time 1008 to the camera control unit 124 . The lens control unit 111 mainly performs control, and performs communication using angle conversion data for RS distortion correction.

Starting from the vertical synchronization signal (see VD) of the imaging unit 115 , the camera control unit 124 performs first communication 1001 to the lens control unit 111 . The time at the communication timing 1004 corresponding to the first communication is the reference time for transmitting the leading correction timing 1006 for RS distortion correction from the camera control unit 124 to the lens control unit 111 . The lens control unit 111 acquires an internal timer time at the timing of receiving the first communication 1001, and uses the acquired time as a reference time for calculating the head correction timing of RS distortion correction. Note that the communication timing 1004 of the first communication 1001 may be the same timing as the vertical synchronization signal, or an arbitrary time before or after the vertical synchronization signal. However, it is assumed that communication is performed with a constant time difference with respect to the vertical synchronizing signal every frame. Also, the first communication timing 1004 is desirably a timing that does not overlap with other communication. In the example of FIG. 10, the communication timing 1004 of the first communication 1001 is set at a point before the vertical synchronization signal.

Next, the camera control unit 124 performs second communication 1002 . In the second communication 1002 , the time of the first communication 1001 is used as a reference time, and a relative time 1007 from the reference time and an RS distortion correction top difference time 1008 are transmitted to the lens control unit 111 . The RS distortion correction head difference time 1008 is the difference time from the exposure barycenter timing to the head correction timing of the RS distortion correction. The camera control unit 124 determines a difference time 1007 for calculating a correction point for RS distortion correction, an RS distortion correction head difference time 1008, a movable range B for electronic image blur correction at the current focal length, and shake information. to the lens control unit 111. An imaging frame number corresponding to the timing at which shake is detected is set in this ID information. The second communication 1002 is after the timing 1005 at which the exposure time of the corresponding frame is determined to transmit the head correction timing 1006 of the RS distortion correction. As a result, even when the exposure time varies for each frame, the correct head correction timing 1006 for RS distortion correction can be transmitted. The head correction timing 1006 for RS distortion correction is obtained from the determined exposure time and the signal readout time of the image sensor, and the relative time 1007 and RS distortion correction head difference time 1008 are obtained from the difference from the reference time by the first communication 1001. .

Also, the communication timing of the second communication 1002 is before the timing 1011 at which the frame number is notified. For this reason, ID information for discriminating shake information is set with a frame number next to an image pickup frame number that has been notified so far as an image pickup frame number corresponding to the timing of detecting shake.

The lens control unit 111 has already acquired the reference time at the timing of accepting the first communication 1001 . Therefore, the lens control unit 111 acquires the relative time 1007 and the RS distortion correction top differential time 1008 through the second communication 1002, and can set the RS distortion correction top correction timing 1006 by setting the internal timer. . Further, the lens control unit 111 obtains the movable range B for electronic image blur correction, and calculates the coefficient K used in the division unit 205, including the movable range A for optical image blur correction of the lens apparatus. can be done. The lens control unit 111 acquires shake information from the lens shake detection unit 110 at the head correction timing 1006 of RS distortion correction. The division unit 205 multiplies the image blur correction amount by a coefficient “1−K” to calculate the RS distortion correction amount. The lens control unit 111 holds the RS distortion correction amount until a communication request is received from the camera control unit 124 .

Furthermore, when a certain RS distortion correction timing has passed, the lens control unit 111 sets the next RS distortion correction timing by setting an internal timer. In setting the next RS distortion correction timing, the lens control unit 111 determines the correction timing of the current RS distortion correction and the difference time between the correction points of the RS distortion correction using the RS distortion correction start difference time 1008 and the number of correction points. calculated by The timer setting is repeated until all correction timings of RS distortion correction are completed. In FIG. 10, since the correction points for RS distortion correction are 11 points, the sixth correction point from the head correction timing 1006 corresponds to the exposure barycenter. That is, the timing information of the exposure period includes the information of the exposure barycenter timing 1009 . In this way, the correction points for RS distortion correction are set so that one of the plurality of correction points for RS distortion correction corresponds to the exposure gravity center timing 1009 . The lens control unit 111 also acquires the electronic image blur correction amount at the exposure center-of-gravity timing 1009 . These pieces of acquired information are held in a memory together with ID information for discriminating shake information transmitted by the second communication 1002 .

On the other hand, the camera body section also acquires the camera body shake information by the camera shake detection section 122 . The camera shake information detected in this embodiment is a motion vector. Prior to this, the camera body sets ID information for discriminating shake information. An imaging frame number corresponding to the timing at which shake is detected is set in this ID information. Thereby, unique ID information is set. The timing for setting the ID information is 1012, which is the timing after the timing 1011 at which the frame number is notified. Therefore, the imaging frame number notified at the timing 1011 at which the frame number is notified is set as it is in the ID information for determining the shake information. Further, it becomes possible to acquire the shake information of the camera main unit at 1013, which is the timing at which the detection of the motion vector for the frame for which shake detection is to be completed. Here, the shake information of the camera main body can be acquired, and the previously set ID information for discriminating the shake information is acquired.

Next, the camera control section 124 performs a third communication 1003 to the lens control section 111 . In the third communication 1003 , the lens control unit 111 receives a communication request from the camera control unit 124 , and determines the retained RS distortion correction amount and the shake information transmitted in the second communication 1002 . Convey ID information. As a result, shake information detected on the lens side is acquired together with ID information. The communication timing of the third communication 1003 is after the final correction timing 1010 of RS distortion correction. Final correction timing 1010 corresponds to the 11th correction point. Since the camera control unit 124 originally grasps the RS distortion correction timing, communication is performed at an arbitrary timing after the final correction timing 1010 of the RS distortion correction. The camera control unit 124 passes the RS distortion correction amount acquired from the lens control unit 111 to the electronic image blur correction control unit 123, and finally the electronic image blur correction amount setting unit 306 sets the correction amount.

The first communication, the second communication and the third communication are performed every frame. The camera control unit 124 notifies the reference time through the first communication 1001 , notifies the relative time from the reference time and the movable range of the electronic image stabilization through the second communication 1002 , and sends the RS signal through the third communication 1003 . Receives the amount of distortion correction. The lens control unit 111 acquires the reference time through the first communication 1001, and acquires the start correction timing of the RS distortion correction and the movable range of the electronic image blur correction by the relative time from the reference time through the second communication 1002. . The lens control unit 111 calculates the correction amount at each RS distortion correction timing, and notifies the camera control unit 124 of ID information for determining the correction amount and shake information through the third communication 1003 .

Contents of communication and control including RS distortion correction performed by the camera control unit 124 will be described with reference to the flowchart of FIG. 9 .

First, in S<b>401 , the camera control unit 124 performs first communication to the lens control unit 111 via the camera communication control unit 125 . The time of the first communication is the reference time of the head correction timing 1006 for RS distortion correction.

Next, in S<b>402 , the camera control unit 124 performs second communication to the lens control unit 111 via the camera communication control unit 125 . Using the time of the first communication as a reference time, a relative time 1007 from the reference time and an RS distortion correction head difference time 1008 are transmitted, and a head correction timing 1006 of RS distortion correction is transmitted to the lens control unit 111 . The camera control unit 124 transmits to the lens control unit 111 the movable range of the electronic image stabilization at the current focal length and the first ID information for determining the shake information. An imaging frame number corresponding to the timing at which shake is detected is set in this ID information. The imaging frame number to be set is corrected and set in accordance with the timing of notification of the imaging frame number and the timing of detecting the shake.

In step S<b>403 , the camera control unit 124 sets second ID information for determining shake information of the camera body detected by the camera shake detection unit 122 . The ID information is also set with the imaging frame number corresponding to the timing of detecting the shake. The imaging frame number to be set is corrected and set in accordance with the timing of notification of the imaging frame number and the timing of detecting the shake.

In S404, the camera control unit 124 determines whether a certain period of time has passed since the final correction timing 1010 of the RS distortion correction. The certain period of time is a preset period of time. For RS distortion correction, if a certain period of time has passed since the final correction timing 1010, the process proceeds to S405, and if the certain period of time has not passed, the determination processing of S404 is repeated. The reason for waiting in S404 for a certain period of time is that the camera control unit 124 makes a communication request to the lens control unit 111 after the lens control unit 111 has completed the processing of the final correction point of the RS distortion correction.

In S<b>405 , the camera control unit 124 performs third communication to the lens control unit 111 via the camera communication control unit 125 . Specifically, the camera controller 124 transmits a communication request to the lens controller 111 . Then, as a response to the communication request, the electronic image blur correction amount distributed by the lens control unit 111 at the exposure center-of-gravity timing 1009, the RS distortion correction amount at each of the plurality of correction points for RS distortion correction, and the First ID information for determining the notified shake information is acquired from the lens control unit 111 .

In the next step S406, the camera control unit 124 determines whether the camera shake information detected by the camera shake detection unit 122 has been detected. Specifically, it is determined by a motion vector detection completion interrupt. If shake information of the camera body is detected, the process proceeds to S407, and if not detected, the process returns to S406 and repeats the process. In S407, the camera control unit 124 acquires the shake information of the camera body detected by the camera shake detection unit 122 and the second ID information for distinguishing the shake information set in S403.

In S408, the camera control unit 124 determines whether the ID information of the lens-side detected shake information acquired in S405 matches the ID information of the camera-side detected shake information acquired in S407. If the ID information of the shake information matches, the process proceeds to S409, and if the ID information of the shake information does not match, the process returns to S401 and repeats the process. As in S108, if the ID information of the shake information does not match, the previously set electronic image blur correction amount may be set. Further, when the ID information mismatch of the shake information continues for a predetermined number of times, it is determined that some kind of error such as a communication error has occurred, and reset processing (processing equivalent to turning off the power and then turning on the power again) is performed. You can do it.

In S409, the camera control unit 124 generates an electronic image blur correction amount using the lens-side detected shake information acquired in S405 and the camera-side detected shake information acquired in S407. At that time, the generated electronic image blur correction amount and the ID information of the shake information are set in association with each other.

Finally, in S410, the camera control unit 124 instructs the electronic image blur correction control unit 123 to perform image correction based on the electronic image blur correction amount and the RS distortion correction amount generated in S409.

Communication and control contents including RS distortion correction performed by the lens control unit 111 will be described with reference to the flowchart of FIG. 10 .

First, in S<b>501 , the lens control unit 111 receives a first communication from the camera control unit 124 via the lens communication control unit 112 . The lens control unit 111 acquires the internal timer time at the first communication timing, and uses that time as the reference time for calculating the leading correction timing of the RS distortion correction.

Next, in step S<b>502 , the lens control unit 111 receives second communication from the camera control unit 124 via the lens communication control unit 112 . In the second communication, the lens control unit 111 determines the relative time from the reference time in the first communication, the head differential time of RS distortion correction, the movable range of electronic image blur correction, and the shake information. Acquire the first ID information. Since the lens control unit 111 acquires the time of the first communication timing 1004 as the reference time, it receives the relative time 1007 and the RS distortion correction head difference time 1008 through the second communication 1002 . Then, the head correction timing of RS distortion correction is set by setting an internal timer. Further, the lens control unit 111 receives the movable range of the electronic image blur correction, and calculates and sets the coefficient K used in the division unit 205 including the movable range of the optical image blur correction of the lens device.

In S503, the lens control unit 111 determines whether or not the correction timing for RS distortion correction has arrived. If it is determined that the correction timing for RS distortion correction has arrived, the process proceeds to S504, and if the correction timing has not arrived, the determination process of S503 is repeated.

In S504, the lens control unit 111 acquires shake information from the lens shake detection unit 110 at the correction timing of RS distortion correction. Furthermore, the dividing unit 205 calculates the optical image blur correction amount of the lens apparatus and the correction amount of the RS distortion correction. Until there is a communication request from the camera control unit 124, the lens control unit 111 temporarily stores the RS distortion correction amount and the first ID information for determining the shake information acquired in the previous step S502. In the example of this embodiment, 11 correction points are used for RS distortion correction, and the sixth correction point from the top corresponds to the exposure barycenter. The lens control unit 111 also acquires the electronic image blur correction amount at the exposure center-of-gravity timing 1009 .

In S505, the lens control unit 111 determines whether or not all correction timings for RS distortion correction have ended. If all correction timings for RS distortion correction have arrived, the process advances to S507, and if all have not arrived, the process advances to S506.

In S506, the lens control unit 111 sets the correction timing for the next RS distortion correction by setting an internal timer. In setting the next correction timing, the lens control unit 111 sets the correction timing of the current RS distortion correction, the difference time between the correction points of the RS distortion correction, the RS distortion correction top difference time 1008 acquired in S502, and the number of correction points. Calculated using After setting the timer, the process returns to S503.

In step S<b>507 , the lens control unit 111 determines whether or not there is a communication request for the third communication from the camera control unit 124 via the lens communication control unit 112 . If there is a communication request for the third communication, the process proceeds to S508, and if there is no such communication request, the determination processing of S507 is repeated.

In S<b>508 , the lens control unit 111 executes third communication as a response to the communication request received from the camera control unit 124 via the lens communication control unit 112 . The lens control unit 111 determines the electronic image blur correction amount distributed at the timing of the exposure center of gravity, the RS distortion correction amount at each of a plurality of correction points for RS distortion correction, and the shake information acquired in S502. ID information is notified to the camera control unit 124 .

<Effects of the third embodiment>
In the present embodiment, the exposure center-of-gravity timing and the RS distortion correction leading differential time are transmitted from the camera control section to the lens control section. Further, the camera correction point lens control unit 111 grasps the timing of each correction point for RS distortion correction, and calculates the difference time between the correction points for RS distortion correction, thereby obtaining the RS distortion correction amount. The RS distortion correction amount is notified to the camera control unit, and the RS distortion correction is performed.

In addition, the camera control unit 124 transmits or sets ID information for discriminating shake information for the lens side shake information and the camera side shake information, respectively, in accordance with the timing of detecting shake. Therefore, it is possible to synchronize the shake information by matching the ID information of the two pieces of shake information.

According to this embodiment, in a lens-interchangeable imaging system, a plurality of correction means for electronic image blur correction including optical image blur correction and RS distortion correction, and an angular velocity sensor on the lens side with less communication traffic. The correction of motion vectors on the camera side by a plurality of shake detection means can be controlled in a coordinated and synchronized manner. This makes it possible to provide an imaging system that expands the image blur correction range.

In the present embodiment, a case has been described in which camera control performs correction using the RS distortion correction amount acquired from the lens control unit. Further, the camera control unit performs RS distortion correction in a correction axis direction (Roll direction) other than the acquired axis, so that the image blur correction effect can be further enhanced. Also, in this embodiment, 11 points are illustrated as RS distortion correction points. It is not necessary to fix the number of RS distortion correction points to a predetermined number, and the number of correction points may be changed according to, for example, the communication speed that can be handled by the lens device. In this case, the camera control unit acquires the communication speed that can be handled by communicating with the lens device, and notifies the lens control unit of the number of correction points for RS distortion correction during the second communication. Accordingly, the number of RS distortion correction points acquired by the lens control unit is determined.

[Other embodiments]
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

105 image shake correction unit 106 optical image shake correction controller 110 lens shake detector 111 lens controller 112 lens communication controller 122 camera shake detector 123 electronic image shake correction controller 124 camera controller 125 camera communication controller

Claims (9)

An imaging device to which a lens device can be attached,
an imaging device that acquires an image signal corresponding to the image frame by exposure;
communication control means for communicating with the lens device;
calculation means for calculating first image stabilization information corresponding to shake of the imaging device;
anti-vibration control means for performing anti-vibration control on an imaging frame;
setting means for setting the ID information corresponding to the imaging frame in association with the first vibration isolation information;
The communication control means sends a communication request to the lens when a predetermined time elapses from the timing corresponding to the exposure center of gravity after the first ID information set by the setting means corresponding to the imaging frame is transmitted. receiving the second image stabilization information corresponding to the vibration of the lens device and the first ID information in response to the communication request while transmitting to the device;
When the second ID information and the first ID information corresponding to the imaging frame to be subjected to the anti-vibration control match, the anti-vibration control means sets the first anti-vibration information and the second ID information. image stabilization control based on the image stabilization information. The second image stabilization information is image stabilization information based on information detected by an angular velocity sensor detected in the lens device, and the first image stabilization information is movement calculated based on a plurality of imaging frames. 2. The image pickup apparatus according to claim 1, wherein the image stabilization information is based on vectors or angular velocities. The anti-vibration control means generates third anti-vibration information by synchronizing the first anti-vibration information and the second anti-vibration information. 2. The imaging apparatus according to claim 1, wherein the ID information and the second ID information are set in association with each other. 2. The imaging apparatus according to claim 1, wherein said first ID information and said second ID information are imaging frame numbers corresponding to the timing of detecting shake. 5. The imaging apparatus according to claim 4, wherein the timing at which said setting means sets said first ID information and the timing at which said setting means sets said second ID information are different timings. 3. The anti-vibration control means corrects and sets the first ID information and the second ID information in accordance with the notification timing of the imaging frame number and the timing of detecting the shake. 6. The imaging device according to claim 4 or 5. An imaging system comprising a lens device and an imaging device to which the lens device can be attached,
an imaging device that acquires an image signal corresponding to an imaging frame by exposure;
camera communication control means for communicating with the lens device;
lens communication control means for communicating with the imaging device;
a first calculation means for calculating first image stabilization information corresponding to shake of the imaging device;
a second calculation means for calculating second image stabilization information corresponding to shake of the lens device;
anti-vibration control means for performing anti-vibration control on a captured frame based on the first anti-vibration information and the second anti-vibration information;
a first setting means for setting ID information corresponding to the imaging frame in association with the first vibration isolation information;
After transmitting the first ID information set by the setting means corresponding to the imaging frame, the camera communication control means performs the second calculation after a predetermined time has elapsed from the timing corresponding to the exposure center of gravity. receiving the second image stabilization information and the first ID information calculated by the means from the lens communication control means;
When the second ID information and the first ID information corresponding to the imaging frame to be subjected to the anti-vibration control match, the anti-vibration control means sets the first anti-vibration information and the second ID information. image stabilization control for the imaging frame based on the image stabilization information. A control method for an imaging device having an imaging device capable of mounting a lens device and acquiring an image signal corresponding to an imaging frame by exposure, comprising:
a communication control step for communicating with the lens device;
a calculation step of calculating first vibration isolation information corresponding to the shake of the imaging device;
an anti-shake control step of performing anti-shake control on the imaging frame;
a setting step of setting ID information corresponding to the imaging frame in association with the first vibration isolation information;
In the communication control step, after the first ID information set in the setting step corresponding to the imaging frame is transmitted, a communication request is sent to the lens when a predetermined time has elapsed from the timing corresponding to the exposure center of gravity. receiving the second image stabilization information corresponding to the vibration of the lens device and the first ID information in response to the communication request while transmitting to the device;
In the anti-shake control step, when the second ID information corresponding to the imaging frame to be subjected to anti-shake control matches the first ID information, the first anti-shake information and the second ID information are matched. A control method for an image pickup apparatus, characterized in that image stabilization control is executed based on the image stabilization information. A control method for an imaging system including a lens device and an imaging device having an imaging device to which the lens device can be attached and which obtains an image signal corresponding to an imaging frame by exposure, comprising:
a camera communication control step for communicating with the lens device;
a lens communication control step for communicating with the imaging device;
a first calculation step of calculating first image stabilization information corresponding to shake of the imaging device;
a second calculation step of calculating second image stabilization information corresponding to shake of the lens device;
a stabilization control step of performing stabilization control on a captured frame based on the first stabilization information and the second stabilization information;
a first setting step of setting ID information corresponding to the imaging frame in association with the first vibration isolation information;
In the camera communication control step, after transmitting the first ID information set in the setting step corresponding to the imaging frame, after a predetermined time has passed from the timing corresponding to the exposure center of gravity, the second calculation is performed. receiving the second image stabilization information and the first ID information calculated in the step in the lens communication control step;
In the anti-shake control step, when the second ID information corresponding to the imaging frame to be subjected to anti-shake control matches the first ID information, the first anti-shake information and the second ID information are matched. A control method for an imaging system, characterized in that the image stabilization control is executed for the imaging frame based on the image stabilization information.

Images