JP2014003417A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce rolling shutter distortion appropriately even in cases where the subject moves fast.SOLUTION: An image pickup device 100 comprises: a first image sensor 101 for picking up a subject image; a second image sensor 111 for picking up a subject image; first rolling shutter control means 103 for differentiating the exposure timing of the first image sensor 101 in a designated direction for each pixel row; second rolling shutter control means 113 for differentiating the exposure timing of the second image sensor 111 in a direction opposite to the designated direction for each pixel row; determination means 117 for determining a correction target region on the basis of a difference between a first image acquired by the first image sensor 101 and a second image acquired by the second image sensor 111; and correction means 115 for shifting for each pixel row a main subject region within the correction target region, of the first image, which was determined by the determination means 117.

Description

  The present invention relates to an imaging apparatus.

  Due to the difference in exposure timing between horizontal lines of the solid-state image sensor, distortion (rolling shutter distortion) may occur in the acquired image. In order to suppress the rolling shutter distortion, a technique is known in which after the first imaging, the second imaging is performed by changing the scanning direction between the horizontal lines, and the distortion of the image is corrected based on the two obtained images. (See Patent Document 1).

JP 2010-154390 A

  In the conventional technique, there is a time difference between the two imaging timings, and thus there is a problem that it is difficult to apply to shooting a fast moving subject.

  An image pickup apparatus according to the present invention includes a first image pickup element that picks up a subject image, a second image pickup element that picks up a subject image, and a first rolling shutter that varies the exposure timing of the first image pickup element in a predetermined direction for each pixel column. Acquired by the control means, the second rolling shutter control means for changing the exposure timing of the second image sensor in a direction opposite to the predetermined direction for each pixel row, and the first image and the second image sensor acquired by the first image sensor. Determining means for determining a correction target area based on a difference from the second image to be corrected; and correcting means for shifting a main subject area in the correction target area determined by the determining means for each pixel column in the first image; It is characterized by providing.

  In the imaging apparatus according to the present invention, rolling shutter distortion can be appropriately reduced.

It is a block diagram which illustrates the composition of the camera by one embodiment of the present invention. It is a flowchart which illustrates the flow of the imaging | photography process which CPU performs. It is a flowchart explaining the detail of a rolling shutter distortion correction process. It is a figure which illustrates the 1st picture which photoed the main subject which stops. It is a figure which illustrates the 1st picture which photoed the main subject moving from the screen left to the right. It is a figure which illustrates the 2nd picture which photoed the main subject moving from the screen left to the right. It is a figure which illustrates the difference of the 1st image and the 2nd image. It is a figure which illustrates the 1st image after moving data for every horizontal line. It is a figure which illustrates the 1st image after interpolating data for every horizontal line. It is a figure which illustrates the 1st image which photoed the main subject which moves from the screen right to the left. It is a figure which illustrates the 2nd image which photoed the main subject which moves from the screen right to the left. It is a figure which illustrates the difference of the 1st image and the 2nd image. It is a figure which illustrates the 1st image after moving data for every horizontal line. It is a figure which illustrates the 1st image after interpolating data for every horizontal line. It is a figure which illustrates the 1st image which imaged the main subject which moves from the screen left to the right and stops. It is a figure which illustrates the 2nd image which imaged the main subject which moves from the screen left to the right and stops. It is a figure which illustrates the difference of the 1st image and the 2nd image. It is a figure which illustrates the 1st image after moving data for every horizontal line. It is a figure which illustrates the 1st image after interpolating data for every horizontal line.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating the configuration of a camera 100 according to an embodiment of the present invention. The camera 100 of the present embodiment images the same subject using the two imaging elements 101 and 111. For this purpose, the camera 100 includes a first image sensor 101, an A / D converter 102, an image controller 103, a second image sensor 111, an A / D converter 112, an image controller 113, and an image. The processing unit 115, the memory 116, the CPU 117, the operation member 118, the photographing lens 121, the lens driving unit 122, the aperture driving unit 123, and the beam splitter 124 are included.

  The photographing lens 121 forms subject images on the imaging surfaces of the first imaging element 101 and the second imaging element 111 via the beam splitter 124. The lens driving unit 122 moves a focusing lens and a zoom lens (not shown) forward and backward in the optical axis direction based on instructions from the CPU 117.

  The aperture driving unit 123 limits the luminous flux incident on the beam splitter 124 from the photographing lens 121 by driving an aperture (not shown) based on an instruction from the CPU 117. The beam splitter 124 is composed of, for example, a half mirror. The beam splitter 124 splits the incident subject light into two light beams, emits one light beam toward the second image sensor 111, and emits the other light beam toward the first image sensor 101. As a result, subject images are formed on the respective imaging surfaces of the first imaging element 101 and the second imaging element 111.

  The first image sensor 101 and the second image sensor 111 are constituted by CMOS image sensors having the same number of pixels. Both the image sensors 101 and 111 are controlled so as to start exposure for each horizontal line and sequentially read out photoelectric conversion signals (rolling shutter control). The imaging control unit 103 controls imaging of the first imaging element 101 so that exposure and readout are performed from the horizontal line on the upper side of the screen toward the horizontal line on the lower side of the screen. The imaging control unit 113 controls imaging of the second imaging element 111 so that exposure and readout are performed from the horizontal line on the lower side of the screen toward the horizontal line on the upper side of the screen.

  The A / D converter 102 converts the photoelectric conversion signal read from the first image sensor 101 into digital data. The A / D conversion unit 112 converts the photoelectric conversion signal read from the second image sensor 111 into digital data. The image processing unit 115 performs rolling shutter distortion correction processing, which will be described later, and predetermined image processing on the image data after digital conversion. The memory 116 is used as a temporary storage for image data and a work memory for the CPU 117. The CPU 117 controls the overall operation of the camera 100 by executing a predetermined program. The operation member 118 sends an operation signal corresponding to the user operation to the CPU 117.

  A flow of photographing processing executed by the CPU 117 of the camera 100 described above will be described with reference to a flowchart illustrated in FIG. The CPU 117 repeatedly performs the process of FIG. 2 in the power-on state in which the setting for performing the rolling shutter distortion correction process is performed. Switching between setting whether or not to perform the rolling shutter distortion correction processing is performed by, for example, a menu operation.

  In step S <b> 10 of FIG. 2, the CPU 117 determines whether or not the release button constituting the operation member 118 has been half-pressed. When the half-press operation signal is input from the operation member 118, the CPU 117 makes a positive determination in step S10 and proceeds to step S20. When the half-press operation signal is not input from the operation member 118, the CPU 117 makes a negative determination in step S10. The process according to 2 ends.

  In step S20, the CPU 117 sends an instruction to the imaging control unit 103 to cause the imaging device 101 to perform imaging control for live view display and exposure control. In step S30, the CPU 117 performs exposure calculation and live view display processing. Thus, exposure calculation is performed based on the image data acquired by the image sensor 101, live view display processing is performed on the acquired image data, and a live view image is displayed on a liquid crystal monitor (not shown). . The live view image refers to a monitor image acquired at a predetermined interval (for example, 30 frames / second) before a shooting instruction.

  In step S40, the CPU 117 performs AF calculation. In the present embodiment, AF calculation is performed by a contrast detection method. Specifically, while detecting the contrast based on the live view image, an instruction is sent to the lens driving unit 122 in step S50, and the focusing lens is moved in a direction in which the contrast is detected to be higher.

  In step S60, the CPU 117 determines whether or not the release button constituting the operation member 118 has been fully pressed. When the full-press operation signal is input from the operation member 118, the CPU 117 makes a positive determination in step S60 and proceeds to step S70. If the full-press operation signal is not input from the operation member 118, the CPU 117 makes a negative determination in step S60 and returns to step S10.

  In step S70, the CPU 117 performs automatic exposure control (AE control) based on the exposure calculation result in step S30. Specifically, an instruction is sent to the aperture control unit 123 to control the aperture value based on the calculation result. In step S <b> 80, the CPU 117 sends an instruction to the imaging control unit 103 based on the exposure calculation result in step S <b> 30 to cause the first imaging element 101 to perform imaging control. In step S <b> 90, the CPU 117 sends an instruction to the imaging control unit 113 based on the exposure calculation result in step S <b> 30 to cause the second imaging element 111 to perform imaging control. By performing the imaging in step S80 and step S90 in parallel, the image data acquired by the imaging elements 101 and 111 substantially simultaneously is sent to the image processing unit 115.

  In step S100, the CPU 117 sends an instruction to the image processing unit 115, and performs rolling shutter distortion correction processing. In the present embodiment, a range in which rolling shutter distortion occurs is obtained based on the difference between the first image acquired by the first image sensor 101 and the second image acquired by the second image sensor 111, and is necessary within this range. Corrective processing is performed. Details of the rolling shutter distortion correction processing will be described later. In step S110, the CPU 117 further performs image processing such as contour emphasis processing and white balance processing on the image after the rolling shutter distortion correction, and proceeds to step S120.

  In step S120, the CPU 117 displays the image after image processing on a liquid crystal monitor (not shown), and proceeds to step S130. In step S130, the CPU 117 accepts an OK operation by the user. For example, a user who has confirmed the image after image processing displayed on the liquid crystal monitor operates an OK button (not shown) constituting the operation member 118. If an OK operation signal is input from the operation member 118, the CPU 117 makes an affirmative determination in step S130 and proceeds to step S140. If an OK operation signal is not input from the operation member 118, the CPU 117 makes a negative determination in step S130 and proceeds to step S10. Return. When returning to step S10, the CPU 117 prepares for the next shooting without recording the shot image data. In step S140, the CPU 117 records the captured image data on a recording medium (not shown), ends a series of shooting processes, and ends the process of FIG.

<Rolling shutter distortion correction>
Details of the rolling shutter distortion correction process (S100), which is a feature of the present embodiment, will be described with reference to the flowchart illustrated in FIG. In step S101 in FIG. 3, the CPU 117 sends an instruction to the image processing unit 115 to perform a difference calculation between the first image and the second image in units of pixels.

  The difference calculation will be described with reference to FIGS. FIG. 4 is a diagram illustrating a first image (referred to as an image P) acquired by the first image sensor 101 when a still main subject is photographed. The image data is composed of 20 horizontal lines × 40 vertical lines, and a quadrangle having coordinates (16,6), (16,15), (25,6), and (25,15) as vertices is the imaging surface. It represents the main subject in the center. Note that the second image acquired by the second image sensor 111 is substantially the same as the image P when the main subject is stationary.

(Main subject moving from left to right on the screen)
FIG. 5 is a diagram illustrating a first image (referred to as image A) acquired by the first image sensor 101 when the main subject moves from the left to the right of the screen. As in the case of FIG. 4, it is composed of 20 horizontal lines × 40 vertical lines, and is a rectangle having coordinates (16,6), (25,15), (25,6), and (34,15) as vertices. Represents the main subject. Due to the rolling shutter control described above, the exposure timing is earlier in the horizontal line on the upper side of the screen than on the horizontal line on the lower side of the screen, so that the main subject image is deformed compared to the image P due to rolling shutter distortion.

  FIG. 6 is a diagram illustrating a second image (referred to as image B) acquired by the second image sensor 111 when the main subject moves from the left to the right of the screen. Image A and image B are acquired at substantially the same time except that the scanning direction of the horizontal lines is different. In the case of FIG. 6 as well, it is composed of 20 horizontal lines × 40 vertical lines, and a rectangle having vertices at coordinates (25,6), (16,15), (34,6), and (25,15) is the main. Represents the subject. Due to the rolling shutter control described above, the exposure timing is earlier in the horizontal line on the lower side of the screen than in the horizontal line on the upper side of the screen, so that the main subject image is deformed compared to the image P due to rolling shutter distortion.

  FIG. 7 is a diagram illustrating the difference calculation result (image AB) in step S101 (FIG. 3). The image processing unit 115 takes a difference between the image A and the image B and detects an area where rolling shutter distortion occurs in at least one of the image A and the image B. In order to eliminate the influence of noise and the like included in images A and B, data whose calculation result is less than a predetermined value (for example, ± 5 LSB (least significant bit)) is treated as 0. In the present embodiment, a region where the calculation result is not 0 is set as a correction processing target region.

  In step S102 of FIG. 3, the CPU 117 determines whether there is a rolling shutter distortion region. The CPU 117 makes an affirmative determination in step S102 when the correction processing target area for which the difference calculation result is not 0 is present, and proceeds to step S103. The CPU 117 skips the subsequent processes and ends the process of FIG. 3 because the distortion correction process is unnecessary when the correction process target area does not exist.

  In step S103, the CPU 117 sends an instruction to the image processing unit 115 to extract the correction processing target area and the area a corresponding to the main subject from the first image (image A). In step S104, the CPU 117 sends an instruction to the image processing unit 115 to extract the correction processing target area and the area b corresponding to the main subject from the second image (image B).

  In step S105, the CPU 117 sends an instruction to the image processing unit 115 to search for data that matches the data in the area a for each horizontal line in the extracted area b. For example, a data string including data of a predetermined pixel (for example, 5 pixels) from pixel data on the leftmost side (coordinate (xa, y)) of the data line in the horizontal line direction constituting the area a is used as a template. A data string that matches the template (difference is less than ± 5 LSB) is searched from data constituting the region b on the same horizontal line. The coordinate of the leftmost pixel data in the data string of the area b detected by this search is defined as (xb, y).

  In step S <b> 106, the CPU 117 sends an instruction to the image processing unit 115, and calculates a movement amount for the line data in the area “a” for each horizontal line. In the present embodiment, ½ of the distance of the corresponding data between the area a and the area b, that is, (xb−xa) / 2 is set as the movement amount for the line data of the area a in the line. The reason why the movement amount is ½ of the distance is to simplify the calculation.

  In step S107, the CPU 117 sends an instruction to the image processing unit 115, and moves the line data constituting the area a for each horizontal line by (xb−xa) / 2 in the horizontal direction. Specifically, each data of the area a is translated from the coordinate (xa, y) to the coordinate (((xb−xa) / 2 + xa), y). FIG. 8 is a diagram illustrating a first image (referred to as an image D) after data is moved from the image A for each horizontal line.

  In step S108, the CPU 117 sends an instruction to the image processing unit 115, and for each horizontal line, the data position vacated by the movement in step S107 is filled with the data of the corresponding second image (image B) and interpolated. Specifically, data adjacent to region b in image B is pasted. FIG. 9 is a diagram illustrating a first image (referred to as an image E) after data is interpolated from the image D for each horizontal line.

  In step S109, the CPU 117 determines whether or not the processing of S105 to S108 has been completed for all the horizontal lines in the correction processing target area. When the CPU 117 completes all horizontal lines, it makes a positive determination in step S109, ends the processing in FIG. 3, and proceeds to step S110 in FIG. If the CPU 117 has not finished processing for all horizontal lines, the CPU 117 makes a negative determination in step S109 and returns to step S105. When returning to step S105, the processing described above is repeated for the next horizontal line.

(When the main subject moves from right to left on the screen)
An example in which the moving direction of the main subject is different from the moving direction described above will be described. FIG. 10 is a diagram illustrating a first image (referred to as image A1) acquired by the first image sensor 101 when the main subject moves from the right to the left of the screen. A quadrangle composed of 20 horizontal lines × 40 vertical lines, each having a vertex at coordinates (25,6), (16,15), (34,6), and (25,15) represents the main subject. Since the exposure timing is earlier in the horizontal line on the upper side of the screen than the horizontal line on the lower side of the screen by the rolling shutter control described above, the main subject image that is originally rectangular is deformed due to the rolling shutter distortion.

  FIG. 11 is a diagram illustrating a second image (referred to as image B1) acquired by the second image sensor 111 when the main subject moves from the right to the left of the screen. The image A1 and the image B1 are acquired at substantially the same time except that the scanning direction of the horizontal line is different. Also in the case of FIG. 11, it is composed of 20 horizontal lines × 40 vertical lines, and a quadrangle having coordinates (16,6), (25,15), (25,6), and (34,15) as vertices is mainly used. Represents the subject. Due to the rolling shutter control described above, the exposure timing is earlier in the lower horizontal line than in the upper horizontal screen, so that the main subject image, which is originally rectangular, is deformed due to the rolling shutter distortion.

  FIG. 12 is a diagram illustrating a difference calculation result (image AB1) in step S101 (FIG. 3). A difference between the image A1 and the image B1 is taken to detect a region where rolling shutter distortion occurs in at least one of the image A1 and the image B1. As described above, data whose calculation result is less than a predetermined value is treated as 0, and an area where the calculation result is not 0 is set as a correction processing target area.

  FIG. 13 is a diagram illustrating a first image (referred to as image D1) after data is moved from the image A1 for each horizontal line. FIG. 14 is a diagram illustrating a first image (referred to as image E1) after data is interpolated for each horizontal line with respect to image D1. According to FIG. 14, the deformation of the main subject can be suppressed compared to the case of FIG.

(When the main subject moves from left to right on the screen and stops)
An example of stopping after the main subject moves will be described. FIG. 15 is a diagram illustrating a first image (referred to as image A2) acquired by the first image sensor 101 when the main subject moves from the left to the right on the screen and stops. It consists of 20 horizontal lines x 40 vertical lines, and coordinates (16,6), (23,13), (23,15), (25,6), (32,13), (32,15) Each hexagon with its apex represents the main subject. Since the exposure timing is earlier in the horizontal line on the upper side of the screen than the horizontal line on the lower side of the screen by the rolling shutter control described above, the main subject image that is originally rectangular is deformed due to the rolling shutter distortion.

  FIG. 16 is a diagram illustrating a second image (referred to as image B2) acquired by the second imaging element 111 when the main subject moves from the left to the right on the screen and stops. The image A2 and the image B2 are acquired at substantially the same time except that the scanning direction of the horizontal line is different. 16 also includes 20 horizontal lines × 40 vertical lines, and coordinates (23,6), (23,8), (16,15), (32,6), (32,8), ( Hexagons with vertices 25 and 15) represent the main subject. Due to the rolling shutter control described above, the exposure timing is earlier in the lower horizontal line than in the upper horizontal screen, so that the main subject image, which is originally rectangular, is deformed due to the rolling shutter distortion.

  FIG. 17 is a diagram illustrating the difference calculation result (referred to as image AB2) in step S101 (FIG. 3). A difference between the image A2 and the image B2 is taken to detect a region where rolling shutter distortion occurs in at least one of the image A2 and the image B2. As described above, data whose calculation result is less than a predetermined value is treated as 0, and an area where the calculation result is not 0 is set as a correction processing target area.

  FIG. 18 is a diagram illustrating a first image (referred to as an image D2) after data is moved from the image A2 for each horizontal line. FIG. 19 is a diagram illustrating a first image (referred to as an image E2) after data is interpolated for each horizontal line with respect to the image D1. According to FIG. 19, the deformation of the main subject can be suppressed compared to the case of FIG.

According to the embodiment described above, the following operational effects can be obtained.
(1) The camera 100 is arranged such that the first image sensor 101 that captures a subject image, the second image sensor 111 that captures a subject image, and the exposure timing of the first image sensor 101 are horizontal from the upper side of the screen toward the lower side of the screen. The image pickup drive unit 103 that varies for each line, the image pickup drive unit 113 that varies the exposure timing of the second image sensor 111 from the lower side of the screen toward the upper side of the screen, and the first image sensor 101. The CPU 117 that determines the correction target area based on the difference between the first image A and the second image B acquired by the second image sensor 111, and the correction target area in the correction target area determined by the CPU 117 in the first image A And an image processing unit 115 for shifting the main subject area a for each horizontal line. Thereby, even when the subject moves fast, the rolling shutter distortion can be appropriately reduced.

(2) In the camera 100, the image processing unit 115 extracts a template from the main subject region a of the first image A for each horizontal line, and extracts data that matches the template from the main subject region b of the second image B. Detection is performed, and the shift amount of the main subject area a is determined based on the relative distance (xb−xa) between the template and the detection data. Since only the first image A side is subjected to a simple process of shifting the data of the main subject area a in the horizontal direction only in a range where the rolling shutter distortion occurs, the rolling shutter distortion can be reduced with a light processing load.

(3) In the camera 100, the image processing unit 115 causes the main subject region so that the template moves to a position (xb−xa) / 2 corresponding to approximately ½ of the relative distance between the template and the detection data. Since the shift amount is determined, the calculation can be simplified and the rolling shutter distortion can be reduced with a light processing load.

(4) In the camera 100, the image processing unit 115 further interpolates the vacant data position by shifting the main subject area a within the correction target area with the corresponding data of the second image B. Thus, the interpolation can be performed without performing a simple interpolation calculation, and the image size after the rolling shutter distortion correction can be prevented from being reduced.

(5) The camera 100 further includes a beam splitter 124 that branches the subject light beam into the first light beam and the second light beam, and the first image sensor 101 captures a subject image by the first light beam, and the second image sensor 111. Took a subject image by the second light flux. Since two subject images can be obtained through one photographing lens 121, it can be made more compact than a case where two photographing lenses are provided.

(Modification 1)
In the description of step S106 (FIG. 3) described above, when calculating the movement amount for the data of the area a for each horizontal line, the distance of the corresponding data between the area a and the area b, that is, (xb In the example described above, xa) / 2 is set as the movement amount with respect to the data of the region a in the line. As a result, the data after movement is positioned at the approximate center of the region a and the region b. Instead, the amount of movement may be calculated so that the data after movement is located at the leftmost part of the region a and the region b. Further, the movement amount may be calculated so as to be located at the rightmost part of the region a and the region b.

(Modification 2)
When a color filter is provided for each pixel of the image sensor 101 and the image sensor 111, the above-described distortion correction processing is performed between pixel data of the same color. For example, when an RGB color filter is provided, the above-described distortion correction is performed for each of pixel data provided with an R color filter, pixel data provided with a G color filter, and pixel data provided with a B color filter. Process.

(Modification 3)
Note that in the case where a color filter is provided for each pixel of the image sensor 101 and the image sensor 111, the above distortion correction processing is performed only on pixel data in which any one color (for example, G color) color filter is provided. The obtained result may be reflected in pixel data of other colors (R color, B color). Alternatively, the luminance Y may be calculated for the RGB data, the above processing is performed only for the luminance Y data, and the obtained result may be reflected in the pixel data of all colors (R, G, B). .

(Modification 4)
In the above description, the rolling shutter control when the CMOS image sensor is used as the first image sensor 101 and the second image sensor 111 has been described. In addition, the present invention can be applied to a case where a CCD image sensor is used as the first image sensor 101 and the second image sensor 111. In this case, focal plane shutters are provided on the imaging surfaces of the two CCD image sensors, and slits in the horizontal line direction are formed by the blades of these focal plane shutters. Then, the focal plane shutter is controlled to be opened and closed so as to perform so-called slit exposure by moving the horizontal slit opening in the vertical direction. Here, the slit opening on the first image sensor 101 side moves from the top to the bottom of the screen, and the slit opening on the second image sensor 111 side moves from the bottom to the top of the screen.

  The accumulation time of the two CCD image sensors is from the movement start point to the movement end point of the slit opening. Rolling shutter distortion also occurs when mechanically performing rolling shutter control as in Modification 4. However, by performing distortion correction processing as in the above-described embodiment, appropriate distortion correction is possible.

(Modification 5)
In the above embodiment, an example in which two photographed images are acquired by the first image sensor 101 and the second image sensor 111 by branching the subject light flux that has passed through one photographing lens by the beam splitter 124 has been described. Instead, a first photographing lens that forms a subject image on the first image sensor and a second photographing lens that forms a subject image on the second image sensor, both of which acquire the same subject image. You may make it the structure to carry out. Both image sensors pick up images at substantially the same time only in the scanning direction of the horizontal line.

(Modification 6)
In the above description, the scanning direction for rolling shutter control of the first image sensor 101 is the direction from the horizontal line on the upper side of the screen to the horizontal line on the lower side of the screen, and the scanning direction for controlling the rolling shutter of the second image sensor 111 is The orientation was from the horizontal line at the bottom of the screen to the horizontal line at the top of the screen. These scanning directions need only be relatively opposite to each other between the first image sensor 101 and the second image sensor 101, and the vertical lines of the screen may be scanned in the left-right direction of the screen.

  The above description is merely an example, and is not limited to the configuration of the above embodiment.

DESCRIPTION OF SYMBOLS 100 ... Camera 101 ... 1st image sensor 102, 112 ... A / D conversion part 103, 113 ... Imaging drive part 111 ... 2nd image sensor 115 ... Image processing part 116 ... Memory 117 ... CPU
118: Operation member 121 ... Photography lens

Claims (5)

A first image sensor that images a subject image;
A second image sensor that images the subject image;
First rolling shutter control means for varying the exposure timing of the first image sensor for each pixel column in a predetermined direction;
Second rolling shutter control means for varying the exposure timing of the second image sensor for each pixel column in a direction opposite to the predetermined direction;
Determining means for determining a correction target region based on a difference between a first image acquired by the first image sensor and a second image acquired by the second image sensor;
Correction means for shifting a main subject area in the correction target area determined by the determination means in the first image for each pixel column;
An imaging apparatus comprising: The imaging device according to claim 1,
The correcting means is provided for each pixel column.
Extracting a template from the main subject area of the first image;
Detecting data matching the template from the main subject area of the second image;
An image pickup apparatus, wherein a shift amount of the main subject region is determined based on a relative distance between the template and the detection data. The imaging device according to claim 2,
The correction device determines an amount of shift of the main subject region so that the template moves to a position corresponding to approximately a half of a relative distance between the template and the detection data. . The imaging device according to claim 3.
The image pickup apparatus according to claim 1, wherein the correction unit further interpolates a position vacated by shifting the main subject area within the correction target area using corresponding data of the second image. In the imaging device according to any one of claims 1 to 4,
Branching means for branching the subject light beam into the first light beam and the second light beam;
The first imaging element captures a subject image by the first light flux,
The image pickup apparatus, wherein the second image pickup device picks up a subject image by the second light flux.

Images