JP2012169990A - Imaging device and image distortion correction method for imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of reducing a delay time and correcting image distortion due to a rolling shutter system while maintaining a small circuit scale.SOLUTION: An imaging device 1 comprises: a first image sensor 4 that images a subject using a rolling shutter system; a second image sensor 5 that images the subject using the rolling shutter system and performs exposure from a direction opposite to an exposure direction of the first image sensor 4; a subtraction unit 31 that compares a first image signal output from the first image sensor 4 and a second image signal output from the second image sensor 5, and calculates a difference value in the same pixel row; a subject area discrimination unit 34 that discriminates the pixel row into a stationary subject area and a moving subject area based on the difference value; and an image distortion correction unit 47 that outputs the average value of an image signal obtained by shifting the first image signal in the horizontal direction and an image signal obtained by shifting the second image signal in the horizontal direction as a correction processing result in the moving subject area.

Description

  The present invention relates to an image pickup apparatus that corrects image distortion caused by a rolling shutter type image pickup device and an image distortion correction method for the image pickup apparatus.

  In recent years, with the widespread use of high-definition televisions and recorders, video cameras have become higher in definition, and as a result, image sensors have evolved from CCD sensors to CMOS sensors. A CMOS sensor is characterized by high-speed reading, high sensitivity, and low power consumption compared with a CCD sensor, and is superior in high definition.

  Such a CMOS sensor is driven by a rolling shutter. Unlike the CCD sensor, this shutter function is a method of reading a signal by sequentially scanning a number of pixels arranged two-dimensionally for each pixel row. . For this reason, there is a time difference of approximately one vertical period in the charge accumulation time at the top and bottom of the frame, and when the subject is moving, the exposure period (time) is shifted for each pixel row and the image flows at the top and bottom of the screen. . In particular, when the movement of the subject is fast, there is a problem that the distortion of the image becomes large.

  Therefore, in the conventional imaging device disclosed in Patent Document 1, a motion vector amount is detected from an image signal for a plurality of frames, and the number of lines between a correction line and a reference line and the detected motion vector amount are used. A method for correcting the position of a moving subject by calculating a correction amount has been proposed.

JP 2009-141717 A

 However, the conventional imaging device disclosed in Patent Document 1 described above has a problem that a delay time until a correction result is obtained increases because it is necessary to accumulate image signals for a plurality of frames. It was. Further, since the motion vector amount is detected and the correction amount is calculated from the positional relationship between the motion vector amount and the correction line, there is a problem that the circuit scale becomes large.

 Therefore, the present invention has been proposed in view of the above-described circumstances, and an imaging apparatus and an imaging apparatus that can reduce the delay time and can correct image distortion due to the rolling shutter system while the circuit scale is small. An object is to provide an image distortion correction method.

  In order to achieve the above-described object, an imaging apparatus according to the present invention captures a subject with a rolling shutter system and a subject with a rolling shutter system, and reverses the exposure direction of the first imaging device. The second image sensor exposed from the direction, the first image signal output from the first image sensor and the second image signal output from the second image sensor are compared, and a difference value in the same pixel row is obtained. A subtraction unit; an edge detection unit that detects an edge position of the subject as edge position information in each row corresponding to the same pixel row of each of the first image signal and the second image signal; and the difference value Based on the edge position information, in each row corresponding to the same pixel row of each of the first image signal and the second image signal, the determination target region is a stationary subject region. A subject area discriminating unit for discriminating whether the subject area is a moving subject area; and an image signal obtained by moving the first image signal in a horizontal direction to a first position according to the movement of the subject in the moving subject area; An image distortion correction unit that outputs, as a correction processing result, an average value of the image signal obtained by moving the second image signal in the horizontal direction to a second position according to the movement of the subject. To do.

  Further, the image distortion correction unit of the imaging device according to the present invention may be configured to calculate an average value of the first image signal and the second image signal or the first image signal and the second image signal in the still subject region. Either one is output as a correction processing result.

  Furthermore, the image distortion correction method of the imaging apparatus according to the present invention includes a first imaging step of imaging a subject by a first imaging element that images by a rolling shutter system, and exposure of the first imaging element by imaging by a rolling shutter system. A second imaging step in which the subject is imaged by a second imaging element that is exposed from a direction opposite to the direction; a first image signal output from the first imaging element; and a second image signal output from the second imaging element. And subtracting the difference value in the same pixel row to determine the edge position of the subject in each row corresponding to the same pixel row in each of the first image signal and the second image signal. Based on the edge detection step that is detected as information, the difference value, and the edge position information, the first image signal and the second image signal, respectively. In each row corresponding to the same pixel row, a subject region determination step for determining whether the determination target region is a stationary subject region or a moving subject region, and in the moving subject region, the first image signal is transmitted in a horizontal direction. The average value of the image signal moved to the first position according to the movement of the subject and the image signal obtained by moving the second image signal to the second position according to the movement of the subject horizontally. And an image distortion correction step for outputting as a correction processing result.

  According to the image pickup apparatus and the image distortion correction method of the image pickup apparatus according to the present invention, the delay time can be reduced and the image distortion by the rolling shutter system can be corrected while the circuit scale is small.

It is a block diagram which shows the structure of the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure which shows a response | compatibility with the scanning line number and output signal of an image pick-up element in the imaging device which concerns on one Embodiment to which this invention is applied. It is a block diagram which shows the structure of the image distortion correction process part of the imaging device which concerns on one Embodiment to which this invention is applied. It is a block diagram which shows the structure of the image distortion correction process part of the imaging device which concerns on one Embodiment to which this invention is applied. It is a flowchart which shows the process sequence of the image distortion correction process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure which shows an example for demonstrating the image distortion correction process by the imaging device which concerns on one Embodiment to which this invention is applied. It is a figure which shows an example for demonstrating the image distortion correction process by the imaging device which concerns on one Embodiment to which this invention is applied.

  Hereinafter, an embodiment to which the present invention is applied will be described with reference to the drawings.

[Configuration of imaging device]
FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus according to the present embodiment. As shown in FIG. 1, an imaging apparatus 1 according to the present embodiment includes a lens 2 on which light from a subject is incident, a separation optical system 3 that separates a subject image incident through the lens 2 in two directions, and a subject. Of the first imaging device 4, the second imaging device 5 that images the subject by the rolling shutter method and exposes the subject from the direction opposite to the exposure direction of the first imaging device 4, and the first imaging device 4. The first storage unit 6 that stores the output signal SD1, the second storage unit 7 that stores the output signal SD2 of the second image sensor 5, the output signal FD1 of the first storage unit 6, and the output signal of the second storage unit 7 An image distortion correction processing unit 8 that corrects image distortion by the rolling shutter system using the FD 2 is provided.

  Here, the first image pickup device 4 and the second image pickup device 5 are image pickup devices that sequentially expose the subject image to be formed from opposite directions perpendicular to each other. For example, the first image pickup device 4 is an upper portion of the subject image. The second image sensor 5 sequentially scans from the lower part of the subject image.

  The first storage unit 6 and the second storage unit 7 are configured by a frame memory having a storage capacity for one screen of the subject image, and are controlled by a control circuit (not shown). The output signals SD1 and SD2 of the first and second image sensors 4 and 5 are sequentially stored for each line, and the image signal stored one frame before is selectively output for each line.

  The image distortion correction processing unit 8 performs image distortion correction processing shown below using the output signals FD1 and FD2 of the first and second storage units 6 and 7, thereby correcting image distortion due to the moving subject. The correction processing result Q is output.

[Signal processing up to image distortion correction processing section]
Next, signal processing from the first and second imaging elements 4 and 5 to the image distortion correction processing unit 8 will be described with reference to FIG. Assuming that the output signal of the first image sensor 4 is SD1, the output signal of the second image sensor 5 is SD2, and the scanning line number in the vertical synchronization period is LN, the number of scanning lines in one vertical synchronization period as shown in FIG. Can be expressed as LN = 1 to LN = 1125.

  Here, if the vertical effective image area of the subject image is 1080 lines, and the signals of the respective lines corresponding to the subject image are represented as P1 to P1080, the scanning line number LN, the output signal SD1 of the first image sensor 4 and the second signal. The relationship between the output signal SD2 of the image sensor 5 can be represented by FIG. That is, the first image sensor 4 outputs the uppermost line P1 of the subject image on the 42nd line, and the second image sensor 5 outputs the lowermost line P1080 of the subject image on the same 42th line.

  The output signal SD1 of the first image sensor 4 is stored in the first storage unit 6, and the output signal SD2 of the second image sensor 5 is stored in the second storage unit 7. When the output signal of the first storage unit 6 is FD1, and the output signal of the second storage unit 7 is FD2, the scanning line number LN, the output signal FD1 of the first storage unit 6, and the output signal FD2 of the second storage unit 7 are used. This relationship can be represented by FIG. That is, the first storage unit 6 and the second storage unit 7 both output the uppermost line P1 of the subject image stored one frame before in the 42nd line, and both the lowermost line of the subject image in the 1121st line. P1080 is output.

  The output signals FD1 and FD2 of the first and second storage units 6 and 7 thus output are input to the image distortion correction processing unit 8.

[Configuration of image distortion correction processing unit]
Next, the configuration of the image distortion correction processing unit 8 according to the present embodiment will be described. 3 and 4 are block diagrams showing the configuration of the image distortion correction processing unit 8 according to this embodiment. As shown in FIGS. 3 and 4, the image distortion correction processing unit 8 according to this embodiment subtracts a difference value between the output signal FD1 of the first storage unit 6 and the output signal FD2 of the second storage unit 7. Unit 31, a first edge detection unit 32 that detects an edge of the output signal FD1 of the first storage unit 6, a second edge detection unit 33 that detects an edge of the output signal FD2 of the second storage unit 7, and an output signal A subject region discriminating unit 34 that discriminates one pixel row into a stationary subject region and a moving subject region based on a difference value between the FD1 and the output signal FD2, and a signal for selecting a signal to be output as the correction processing result Q A signal selection unit 35 that outputs a selection signal for each subject area, a third storage unit 41 that temporarily stores the output signal FD1 of the first storage unit 6, and an output signal FD2 of the second storage unit 7 that is temporarily stored. The fourth storage unit 42 to store and the first storage unit 6 The first horizontal moving unit 43 that moves the output signal FD1 in the horizontal direction, the second horizontal moving unit 44 that moves the output signal FD2 in the second storage unit 7 in the horizontal direction, and the average value of the image signals in the still subject area A first averaging unit 45 to calculate, a second averaging unit 46 to calculate the average value of the image signal in the moving subject region, and an image that outputs the image distortion correction processing result Q for each subject region based on the selection signal And a distortion correction unit 47.

[Image distortion correction procedure]
Next, the procedure of image distortion correction processing by the imaging apparatus according to the present embodiment will be described with reference to the flowchart of FIG. Here, in the present embodiment, as shown in FIG. 6A, when a round object as a subject is stationary on the left side of the screen and a rectangular object is moving on the right side of the screen from the right to the left of the screen is imaged. Will be described as an example. FIG. 7A shows a subject image when the first image sensor 4 captures such a situation. In FIG. 7A, the round subject on the left side of the screen is output without distortion and is rounded, and the rectangular subject on the right side is output distorted and the rectangular subject is output as a parallelogram. Since the first image sensor 4 sequentially scans from the upper part to the lower part of the subject image, the lower part of the exposure period (time) is distorted to the left in the moving direction from the upper part of the early exposure period. On the other hand, in the subject image output from the second image sensor 5 shown in FIG. 7B, the lower part of the exposure period (time) in the lower part of the exposure period (time) is scanned from the upper part of the slower exposure period in order to scan sequentially from the lower part to the upper part. It can be seen that is distorted to the right as opposed to the moving direction.

  The output signals FD1 and FD2 of the first and second storage units 6 and 7 corresponding to the line period of the scanning line number LN = 41 + m, that is, the pixel row Pm among the subject images shown in FIGS. It shows to FIG.7 (c), (d). Here, it is assumed that the background signal level is BGL, the round subject is darker (the signal level is lower) than the background, and the rectangular subject is brighter (the signal level is higher) than the background. Further, when the number of pixels in the horizontal direction (horizontal coordinate h) is 1920 and this is expressed as horizontal coordinates h = 0 to 1919, the left end position of the round subject in the pixel row Pm is h = x1, and the right end position is h = x2. The left end position of the rectangular subject imaged by the first image sensor 4 is h = x4, the right end position is h = x6, the left end position of the rectangular subject imaged by the second image sensor 5 is h = x3, and the right end position. Is set to h = x5.

  Hereinafter, processing for correcting such image distortion will be described with reference to the flowchart of FIG. As shown in FIG. 5, first, when the output signals FD1 and FD2 of the first and second storage units 6 and 7 corresponding to the pixel row Pm are input to the image distortion correction processing unit 8, the subtraction unit 31 performs the step S101. The difference value SB = FD1-FD2 is obtained and output. This difference value SB is shown in FIG.

  As shown in FIG. 7 (e), FD1 and FD2 have different exposure periods (time), but shoot the same subject image. Therefore, the subject can be a background region other than the region where the subject is moving. There is no difference in the signal level even in a region where is stationary, and the difference value SB is almost 0 (SB = 0). More specifically, since the horizontal coordinate h = 0 to x3 is 0, and there is a moving subject having a high signal level only in FD2 during the period from x3 to x4, the difference value SB is negative (SB <0). Become. Next, there is a moving subject in both FD1 and FD2 during the period from h = x4 to x5, but it is considered that the signal level of the moving subject is substantially uniform here, so the difference value SB is stationary. Similar to the subject area, 0 (SB = 0). During the period from h = x5 to x6, there is a moving subject with a high signal level only in FD1, so the difference value SB is positive (SB> 0). Finally, during the period from h = x6 to h = 1919, the difference value SB becomes 0 as a background area.

  Next, in step S102, the output signals FD1 and FD2 are input to the first and second edge detectors 32 and 33, and the edge of the portion corresponding to the boundary of the subject is detected. As an edge detection method, the edge corresponding to the boundary of the subject is detected by calculating the difference between several pixels on the left and right adjacent to the target pixel. Accordingly, the first edge detection unit 32 detects the horizontal coordinates h = x1, x2, x4, x6 as the edges of the FD1 and outputs the detection result ED1, and the second edge detection unit 33 outputs the horizontal coordinates h = x1, x2. , X3, x5 are detected as edges of FD2, and a detection result ED2 is output.

  Next, in step S103, the subject region determination unit 34 determines the pixel row Pm as a stationary subject region and a moving subject region based on the difference value SB and the edge detection results ED1 and ED2. More specifically, as shown in FIG. 7E, when the difference value SB is larger than the predetermined value + GP or smaller than the predetermined value −GP, the output signal FD1 (the pixel row Pm of the first image sensor 4). Output signal) or FD2 (output signal of the pixel row Pm of the second image sensor 5), it is determined that only the determination target region is the moving subject region, and the edge detection results ED1, ED2 Based on the above, it is determined which of FD1 and FD2 is the moving subject area. Further, when the difference value SB is between the predetermined value + GP and the predetermined value −GP, any discrimination target area of FD1 and FD2 is a moving subject, or any discrimination target area of FD1 and FD2. Is determined to be a still subject region, and further, based on the edge detection results ED1 and ED2, it is determined whether the discrimination target region between FD1 and FD2 is a moving subject region or a still subject region. Based on these determination results, the subject area determination unit 34 outputs an area determination signal ST.

  Hereinafter, a specific example in which the subject region determination unit 34 determines and outputs the value of the region determination signal ST will be described in detail with reference to FIG.

  In the region where 0 ≦ h <x1, the difference value SB is | SB | <GP, and since no edge is detected in both ED1 and ED2, the region where 0 ≦ h <x1 is a stationary subject region in both FD1 and FD2. And ST = 0 is output.

  In the region of x1 ≦ h <x2, the difference value SB is | SB | <GP, and h = x1 is detected as an edge in both ED1 and ED2, so the region of x1 ≦ h <x2 is FD1, FD2. Both are determined to be still subject areas and ST = 0 is output.

  In the region of x2 ≦ h <x3, the difference value SB is | SB | <GP, and both ED1 and ED2 are detected with h = x2 as an edge, so the region of x2 ≦ h <x3 is FD1, FD2. Both are determined to be still subject areas and ST = 0 is output.

  In the region of x3 ≦ h <x4, the difference value SB becomes | SB | ≧ GP, and only ED2 detects h = x3 as an edge, and only FD2 has a state change (stationary → moving), so x3 In the region of ≦ h <x4, it is determined that FD1 is a stationary subject region and FD2 is a moving subject region, and ST = 2 is output.

  In the region of x4 ≦ h <x5, the difference value SB is | SB | <GP, and only ED1 detects h = x4 as an edge, and only FD1 has a state change (stationary → moving). It is determined that the region of ≦ h <x5 is a moving subject region for both FD1 and FD2, and ST = 3 is output.

  In the region of x5 ≦ h <x6, the difference value SB is | SB | ≧ GP, and only ED2 detects h = x5 as an edge, and only FD2 has a state change (moving → still), so x5 In the region of ≦ h <x6, it is determined that FD1 is a moving subject region and FD2 is a stationary subject region, and ST = 1 is output.

  In the region of x6 ≦ h <1919, the difference value SB is | SB | <GP, and only ED1 detects h = x6 as an edge, and only FD1 has a state change (moving → still), so x6 In the region of ≦ h <1919, both FD1 and FD2 are determined to be stationary subject regions and ST = 0 is output.

  Here, if the moving subject does not deform with time, the ST = 2 period (x4-x3) and the ST = 3 period (x6-x5) have the same length.

  Next, in step S104, first, the signal selection unit 35 calculates the coordinates CT of the intermediate point during the period when the region determination signal is ST = 2 and the coordinates CB of the intermediate point during the period when ST = 1. In the example shown in FIG. 7E, CT = (x3 + x4) / 2 and CB = (x5 + x6) / 2. And based on this calculation result, the signal selection part 35 outputs the selection signal SEL shown below.

  When 0 ≦ h <x3, the selection signal SEL = 0 for selecting the average value of the still subject areas of FD1 and FD2 is output.

  In x3 ≦ h <CT, a selection signal SEL = 1 for selecting the image signal of FD1 is output.

  In CT ≦ h <CB, a selection signal SEL = 3 for selecting the average value of the moving subject areas of FD1 and FD2 is output.

  In CB ≦ h <x6, the selection signal SEL = 2 for selecting the image signal of FD2 is output.

  In x6 ≦ h <1919, a selection signal SEL = 0 for selecting the average value of the still subject areas of FD1 and FD2 is output.

  In step S105, the signal selection unit 35 calculates a horizontal movement correction amount DL1 (= x4-CT) with respect to FD1 in order to correct distortion due to the moving subject. With this horizontal movement correction amount DL1, the image (signal) of the moving subject area of FD1 can be advanced from x4 to CT. Similarly, in order to delay the image (signal) of the moving subject area of FD2 from x3 to CT, a horizontal movement correction amount DL2 (= CT−x3) for FD2 is calculated.

  When the horizontal movement correction amounts DL1 and DL2 are calculated in the line period of the scanning line number LN = 41 + m in this way, in step S106, FD1 is stored for one line in the third storage unit 41, and FD2 is stored in the fourth storage unit 42. 1 line is stored. The third and fourth storage units 41 and 42 are delayed by one line, and specifically output FD1 and FD2 as LD1 and LD2 as they are in the line period of the scanning line number LN = 41 + m + 1. Here, the signal waveforms of LD1 and LD2 are shown in FIGS. 6 (b) and 6 (c).

  On the other hand, in step S107, the first horizontal movement unit 43 acquires the horizontal movement correction amount DL1 output from the signal selection unit 35, and scans the FD1 stored one line before by shifting the timing by the horizontal movement correction amount DL1. It outputs as MD1 in the line period of line number LN = 41 + m + 1. In this embodiment, the timing is shifted so that FD1 is advanced by DL1. Similarly, the second horizontal movement unit 44 shifts the timing of FD2 by the horizontal movement correction amount DL2 and outputs it as MD2 in the line period of the scanning line number LN = 41 + m + 1. In the present embodiment, the FD2 is outputted with the timing shifted so as to delay the FD2 by DL2.

  When LD1, LD2, MD1, and MD2 are output in this way, in step S108, the first averaging unit 45 calculates an average value for each pixel of LD1 and LD2 and outputs a still subject average value SAV. On the other hand, in step S109, the second averaging unit 46 calculates an average value for each pixel of MD1 and MD2 and outputs a moving subject average value MAV.

  Then, the image distortion correction unit 47 finally selects one of the signals LD1, LD2, SAV, and MAV according to the selection signal SEL in step S110 and outputs it as a correction processing result Q.

  As a result, in the moving subject region, the image distortion correction unit 47 moves the image signal FD1 output from the first image sensor 4 in the horizontal direction and the image signal FD2 output from the second image sensor 5. A moving subject average value MAV, which is an average value with the image signal MD2 moved in the horizontal direction, is output as the correction processing result Q. In the still subject region, the still subject average value SAV, which is the average value of the image signal FD1 output from the first image sensor 4 and the image signal FD2 output from the second image sensor 5, or the first image sensor 4 One of the image signal FD1 output from the image signal FD1 and the image signal FD2 output from the second image sensor 5 is output as the correction processing result Q.

  Specifically, in the present embodiment, the image distortion correction unit 47 operates as follows according to the selection signal SEL.

  When 0 ≦ h <x3, Q = SAV is output corresponding to the selection signal SEL = 0.

  In x3 ≦ h <CT, Q = LD1 is output corresponding to the selection signal SEL = 1.

  In CT ≦ h <CB, Q = MAV is output corresponding to the selection signal SEL = 3.

  In CB ≦ h <x6, Q = LD2 is output corresponding to the selection signal SEL = 2.

  In x6 ≦ h <1919, Q = SAV is output corresponding to the selection signal SEL = 0.

  As a result, the correction processing result Q in the line period of LN = 41 + m + 1 has a waveform as shown in FIG. 6D, and it can be seen that the image distortion due to the moving subject is corrected. Moreover, since the average value output is selected for the correction processing result Q in many periods, the SN ratio is improved.

  By performing the above-described operation in units of lines and repeating this over one frame period, an image with corrected image distortion can be output as shown in FIG. 6A, and the imaging according to the present embodiment The image distortion correction process by the apparatus ends.

  Note that the present invention is not limited to the case where the image distortion correction processing unit 8 of FIG. 1 is configured by hardware, and the image distortion correction process is performed by software by a computer program that executes the processing procedure shown in FIG. You can also. In this case, the computer program may be taken into the computer from a recording medium or may be taken into the computer via a network.

[Effect of the embodiment]
As described above in detail, according to the imaging apparatus and the image distortion correction method of the imaging apparatus according to the present embodiment, two imaging elements that are exposed from opposite directions are provided, and the image signals output from these are compared. The moving subject region is discriminated between the stationary subject region and the moving subject region, and the average value of the values obtained by moving the two image signals in the horizontal direction is output as the correction processing result Q in the moving subject region. Image distortion due to the rolling shutter system can be corrected while the circuit scale is small. In particular, when the moving speed of the subject is fast, the delay time is small, which is effective.

  Further, according to the imaging apparatus according to the present embodiment, the stationary subject average value SAV or the image signal FD1 (= LD1) output from the first imaging element 4 and the second imaging element 5 are output in the stationary subject area. Since either one of the image signals FD2 (= LD2) is output as the correction processing result Q, the delay time can be reduced and the image distortion by the rolling shutter system can be corrected while the circuit scale is small.

  The above-described embodiment is an example of the present invention. For this reason, the present invention is not limited to the above-described embodiment, and even if it is a form other than this embodiment, as long as it does not depart from the technical idea of the present invention, it depends on the design and the like. Of course, various modifications are possible.

DESCRIPTION OF SYMBOLS 1 Imaging device 2 Lens 3 Separation optical system 4 1st imaging device 5 2nd imaging device 6 1st memory | storage part 7 2nd memory | storage part 8 Image distortion correction process part 31 Subtraction part 32 1st edge detection part 33 2nd edge detection part 34 Subject area determination unit 35 Signal selection unit 41 Third storage unit 42 Fourth storage unit 43 First horizontal movement unit 44 Second horizontal movement unit 45 First averaging unit 46 Second averaging unit 47 Image distortion correction unit

Claims (3)

A first image sensor that images a subject by a rolling shutter system;
A second image sensor that captures an image of a subject using a rolling shutter system and exposes the subject from an opposite direction to the exposure direction of the first image sensor;
A subtraction unit that compares the first image signal output from the first image sensor and the second image signal output from the second image sensor to obtain a difference value in the same pixel row;
An edge detector that detects an edge position of the subject as edge position information in each row corresponding to the same pixel row of each of the first image signal and the second image signal;
Based on the difference value and the edge position information, in each row corresponding to the same pixel row of each of the first image signal and the second image signal, whether the determination target region is a stationary subject region or a moving subject region A subject area discriminating unit that discriminates whether or not
In the moving subject area, an image signal obtained by moving the first image signal in the horizontal direction to a first position according to the movement of the subject and a second image signal obtained in accordance with the movement of the subject in the horizontal direction. And an image distortion correction unit that outputs an average value of the image signal moved to position 2 as a correction processing result.   In the still subject area, the image distortion correction unit uses an average value of the first image signal and the second image signal or one of the first image signal and the second image signal as a correction processing result. The imaging apparatus according to claim 1, wherein the imaging apparatus outputs the image. A first imaging step of imaging a subject by a first imaging element that images with a rolling shutter system;
A second imaging step in which the subject is imaged by a second imaging element that is imaged by a rolling shutter system and exposed from a direction opposite to the exposure direction of the first imaging element;
A subtraction step of comparing the first image signal output from the first image sensor and the second image signal output from the second image sensor to obtain a difference value in the same pixel row;
An edge detection step of detecting an edge position of the subject as edge position information in each row corresponding to the same pixel row of each of the first image signal and the second image signal;
Based on the difference value and the edge position information, in each row corresponding to the same pixel row of each of the first image signal and the second image signal, whether the determination target region is a stationary subject region or a moving subject region A subject area determination step for determining whether or not
In the moving subject area, an image signal obtained by moving the first image signal in the horizontal direction to a first position according to the movement of the subject and a second image signal obtained in accordance with the movement of the subject in the horizontal direction. And an image distortion correction step of outputting an average value of the image signal moved to position 2 as a correction processing result.

Images