JP2014138331A - Imaging apparatus and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus and a program capable of preventing a control system that utilizes a corrected photographed image from making a false recognition even a corrected amount is large in correction of coordinate deviation of the photographed image.SOLUTION: An imaging apparatus includes: one or more imaging section for photographing an image; an update section for periodically updating a coefficient for a correction formula applied for correcting the coordinate deviation in a photographed image resulting from aging positional deviation of the imaging section on the basis of the one or more photographed images; a division update section for updating the coefficient in two or more times by calculating the difference of the coefficients before and after the update for each item in the correction formula, and adding a value in which the difference was divided in two or more times to the coefficient before the update one after another; and a correction section for correcting the photographed image using the correction formula whose coefficient is updated.

Description

  The present invention relates to a photographing apparatus and a program.

  There is known a technique for correcting a deviation in coordinates of a photographed image caused by a positional deviation of the photographing apparatus over time. For example, a technique for correcting a shift in coordinates of a captured image captured by a stereo camera is known. In the stereo camera, the distance to the subject can be measured from the difference in image position on each captured image by photographing the subject simultaneously with two cameras. Stereo cameras are used in, for example, vehicle control systems. In the vehicle control system, photographing and ranging are repeated at a constant frame interval, and a parallax image including information on the distance to the subject in the image is generated as a moving image. The vehicle control system uses the parallax image to detect a departure from a route or an obstacle and uses it for avoidance control and the like.

  In order to accurately measure the distance with a stereo camera, the distortion characteristics of each monocular camera and the relative position between the two cameras must be known. However, in reality, minute deformation of the photographing apparatus due to change with time, temperature characteristics, and the like is unavoidable. For this reason, automatic calibration for correcting the deviation of the coordinates of the captured image is performed in parallel with the periodic distance measuring operation. In automatic calibration, information on the captured image and other sensors is used to measure deviations in camera characteristics such as the relative position and direction between the two cameras, and the deviation in the coordinates of the captured image is automatically corrected. doing.

  Patent Document 1 is known as an automatic calibration technique. Patent Document 1 discloses an automatic calibration technique that uses vanishing points of parallel lines such as white lines on both sides of a lane included in a captured image.

  However, when correcting the deviation of the coordinates of the captured image, if the correction amount is large, erroneous recognition may occur in the control system using the corrected captured image.

  The present invention has been made in view of the above, and does not cause misrecognition in a control system that uses a corrected captured image even when the correction amount is large when correcting a coordinate shift of the captured image. An object is to provide a photographing apparatus and a program.

  In order to solve the above-described problems and achieve the object, the imaging device of the present invention includes one or more imaging units that capture a captured image and the time-lapse of the imaging unit based on the one or more captured images. An update unit that periodically updates the coefficient of the correction formula that corrects the shift of the coordinates of the captured image caused by the position shift, the coefficient before the update, and the coefficient after the update for each term of the correction formula A division update unit that updates the coefficient in a plurality of times by sequentially adding a value obtained by dividing the difference into a plurality of the coefficients before the update, and the correction formula that updates the coefficient And a correction unit for correcting the captured image.

  In addition, the program of the present invention causes the computer to periodically calculate a coefficient of a correction formula for correcting a shift in the coordinates of the captured image caused by a positional shift of the captured unit over time based on one or more captured images. For each item of the update unit to be updated and the correction formula, the difference between the coefficient before update and the coefficient after update is calculated, and the value obtained by dividing the difference into a plurality of the coefficients before update is sequentially added. Thus, the coefficient update unit functions as a division update unit that updates the coefficient in a plurality of times and a correction unit that corrects the captured image using the correction formula in which the coefficient is updated.

  According to the present invention, it is possible to provide a photographing apparatus and a program that do not cause erroneous recognition in a control system that uses a corrected photographed image even when the correction amount is large when correcting the coordinate deviation of the photographed image. There is an effect that can be done.

FIG. 1 is a diagram illustrating an example of the configuration of the photographing apparatus according to the present embodiment. FIG. 2 is a diagram for explaining the first photographing unit and the second photographing unit of the photographing apparatus of the present embodiment. FIG. 3 is a diagram illustrating an example of correction (coordinate conversion) of coordinate shift of a captured image of the imaging apparatus of the present embodiment. FIG. 4 is a flowchart for explaining an example of a method for correcting a photographed image of the photographing apparatus according to the present embodiment. FIG. 5 is a diagram for explaining an example of a method for selecting coordinates for calculating the moving distance of the photographing apparatus according to the present embodiment. FIG. 6 is a diagram for explaining an example in which a captured image of the imaging apparatus according to the present embodiment is corrected by being divided into a plurality of times. FIG. 7 is a diagram illustrating an example of a hardware configuration of the imaging apparatus according to the present embodiment.

  Embodiments of a photographing apparatus and a program will be described in detail below with reference to the accompanying drawings. FIG. 1 is a diagram illustrating an example of a configuration of an imaging device 10 according to the present embodiment. The imaging apparatus 10 according to the present embodiment includes a first imaging unit 1, a second imaging unit 2, a storage unit 3, an update unit 4, a division update unit 5, a correction unit 6, and a parallax calculation unit 7. The storage unit 3 stores the captured image 11, the correction parameter 12, the division correction parameter 13, the corrected image 14, and the parallax image 15.

  The first photographing unit 1 and the second photographing unit 2 are stereo cameras that photograph a subject. Hereinafter, a photographed image photographed by the first photographing unit 1 is referred to as a “first photographed image”. The captured image captured by the second imaging unit 2 is referred to as a “second captured image”. Further, when the first captured image and the second captured image are not distinguished, they are simply referred to as “captured images”. The photographing apparatus 10 can photograph the same subject by the first photographing unit 1 and the second photographing unit 2 in synchronization with photographing timing.

  Here, the principle of distance measurement of the stereo camera and the stereo camera by the first photographing unit 1 and the second photographing unit 2 will be described. FIG. 2 is a diagram for explaining the first photographing unit 1 and the second photographing unit 2 of the photographing apparatus 10 of the present embodiment.

In the example of FIG. 2, the focal length f, the optical center O ₀ , and the first imaging unit 1 on the imaging surface S ₀ are arranged with the Z axis as the optical axis direction. Further, the focal length f, the optical center O ₁ , and the second imaging unit 2 on the imaging surface S ₁ are arranged with the Z axis as the optical axis direction. The first imaging unit 1 and the second imaging unit 2 are disposed in parallel to the X axis at positions separated by a distance B (baseline length).

The subject A located at a distance d in the optical axis direction from the optical center O ₀ of the first imaging unit 1 forms an image at P ₀ , which is the intersection of the straight line A-O ₀ and the imaging surface S ₀ . On the other hand, the second imaging unit 2, the same subject A is forms an image at a position P ₁ on the imaging surface S _1.

Here, an intersection of a straight line passing through the optical center O ₁ of the second imaging unit 2 and parallel to the straight line A-O ₀ and the imaging surface S ₁ is defined as P ₀ ′. Further, the distance between P ₀ ′ and P ₁ is p. The distance p represents a positional shift amount (hereinafter referred to as “parallax”) on an image obtained by capturing the same subject image with two cameras. The triangle AO ₀ -O ₁ and the triangle O ₁ -P ₀ ′ -P ₁ are similar. Therefore, d = B × f / p. That is, the distance d to the subject A can be obtained from the base line length B, the focal length f, and the parallax p.

  Here, an example in which the control system erroneously recognizes the combination of the distance measurement process and the automatic calibration process of the stereo camera will be described by taking a case where the control system is a vehicle control system as an example.

  When the parallax calculation unit 7 calculates the parallax using the corrected image 14 in which the deviation of the coordinates of the captured image 11 is corrected by automatic calibration, the distance measurement result to the subject changes. For example, consider a stereo camera with 100 m and 1 pixel parallax (by focal length f, baseline length B, and image sensor). Further, it is assumed that the correct distance of a certain subject is 100 m. That is, in the captured image 11 (the first captured image and the second captured image), it is assumed that the parallax of the subject is one pixel.

  Here, in this stereo camera, it is assumed that the parallax offset (parallax error) based on the deviation of the coordinates of the captured image 11 is −0.5 pixel. At this time, it is assumed that a measurement result of parallax 0.5 pixels (= correct parallax (1.0) −error (0.5) = corresponding to a distance of 200 m) is obtained in a certain frame (frame 0). At this time, if the error 0.5 pixel is corrected correctly before executing the parallax calculation process in the next frame, the measurement result of frame 1 is 1 pixel (= 100 m). That is, the photographing apparatus 10 measures a subject that is actually present at a certain distance of 100 m so that it is at a distance of 200 m in the frame 0 and 100 m in the next frame 1. When the vehicle control system naturally extrapolates only from the results of these two frames, the next frame 2 predicts a distance of 0 m, that is, a collision. The vehicle control system may determine that an operation such as emergency braking or avoidance is necessary.

  However, normally, a vehicle control system is provided with a margin that requires multiple collision determinations to start control. Therefore, unnecessary control does not always occur in the above situation, but it is naturally desirable that the stereo camera module alone has high reliability. As described above, the automatic calibration process for improving the accuracy of distance measurement may cause a problem that the subject appears to move apparently. Normally, if the automatic calibration is working effectively, the error should be kept near zero because the correction is made immediately if a deviation occurs.

  However, when the auto-calibration is updated for the first time after the stereo camera is activated or when the camera characteristics change greatly in a short time, there is a possibility that a large deviation is measured at a time. If the correction unit 6 corrects this shift at once, the distance of the measurement result may change greatly. Examples of the camera characteristics that change greatly in a short time include, for example, movement of the vehicle from the shade to the sun (or the reverse direction), or a change in temperature due to operation or stop of the interior air conditioning. Here, the case of the vehicle control system has been described as an example, but the photographing apparatus 10 of the present embodiment may be used other than the vehicle control system.

  Returning to FIG. 1, the captured image 11 is a first captured image or a second captured image. The correction parameter 12 is a coefficient of a correction equation that corrects a coordinate shift of the first captured image (second captured image) caused by a positional shift of the first capturing unit 1 (second captured unit 2) with time. For example, the correction formula is the following formula.

x ′ = a ₀ + a ₁ x + a ₂ y + a ₃ x ² + a ₄ xy + a ₅ y ²
y ′ = b ₀ + b ₁ x + b ₂ y + b ₃ x ² + b ₄ xy + b ₅ y ²

Here, (x, y) is the coordinates of the conversion source, and (x ′, y ′) is the coordinates after the conversion. At this time, the correction parameter 12 _is a 0 ~a ₅ and _b 0 ~b _5. The correction formula may be managed with the coordinates of the corrected image as (x, y) and the coordinates of the pre-correction image (captured image) as (x ′, y ′). In other words, the correction formula may be based on coordinates after coordinate conversion, or may be managed based on coordinates before coordinate conversion. In the present embodiment, the correction formula is a quadratic formula, but any order may be used. Details of the division correction parameter 13, the correction image 14, and the parallax image 15 will be described later.

  Based on one or more photographed images, the update unit 4 detects a coordinate shift of the first photographed image (second photographed image) due to a temporal positional shift of the first photographing unit 1 (second photographer 2). The coefficient of the correction formula to be corrected is updated periodically. Note that a specific method for determining the coefficient of the correction formula for correcting the shift of the coordinates based on one or more captured images may be any method. For example, the update unit 4 determines the coefficient of the correction formula by approximating the coefficient (x, y) before conversion and the coordinate (x ′, y ′) after conversion by the least square method or the like ( Update).

  The division update unit 5 calculates the difference between the coefficient before update and the coefficient after update for each term of the correction formula, and sequentially adds the value obtained by dividing the difference into a plurality of coefficients before update. Is updated in several times. Note that the imaging apparatus 10 does not always have to operate the division update unit 5. For example, when the difference between the correction result based on the coefficient correction formula before the update and the correction result based on the coefficient correction formula after the update is small, the imaging apparatus 10 does not have to operate the split update unit 5. For example, in the present embodiment, if the difference is 1/10 of the length of one pixel (hereinafter referred to as “1/10 pixel”) or less, the division update unit 5 may not be operated.

  The correction unit 6 corrects the captured image using a correction formula with updated coefficients. The correction unit 6 corrects the captured image 11 at a time when the captured image is corrected using the correction formula whose coefficient is updated by the update unit 4. The correction unit 6 corrects the captured image 11 in a plurality of times when the captured image is corrected using the correction formula whose coefficient is updated by the division update unit 5.

  The parallax calculation unit 7 calculates the parallax from the corrected first captured image and the corrected second captured image. The parallax calculation unit 7 uses the parallax to generate a parallax image including parallax information of the coordinates of the captured image 11. A specific parallax calculation method is the same as that of a known parallel equipotential stereo camera. The parallax calculation unit 7 calculates the correlation while shifting the corresponding candidate area in units of one pixel on the image area around the feature point extracted from the reference camera image and the comparison camera image, and calculates the correlation value near the maximum correlation value. By interpolating with a parabola or the like, the parallax is calculated with an accuracy of a pixel unit or less. The parallax calculation unit 7 of the present embodiment outputs the result of rounding up to one pixel in the parallax calculation unit 7 of the present embodiment because the parallax of the minimum parallax is one pixel. The parallax image 15 having an accuracy of 1/10 pixel is generated.

  The imaging device 10 is periodically operated by the above-described processing units (the first imaging unit 1, the second imaging unit 2, the storage unit 3, the update unit 4, the division update unit 5, the correction unit 6, and the parallax calculation unit 7). The captured image is repeatedly captured, corrected, and calculated for parallax, and the parallax image 15 is continuously generated.

  FIG. 3 is a diagram illustrating an example of correction (coordinate conversion) of coordinate deviation of the captured image 11 of the imaging apparatus 10 according to the present embodiment. The correction unit 6 converts the coordinates of the corrected image and the coordinates of the pre-correction image (the photographed image 11) by using the correction formula and the correction parameter 12 (the coefficient of the correction formula term). In order to determine the pixel value of the corrected image, the pixel value of the corresponding captured image is referred to by the correction formula. At this time, the coordinate value after conversion may not be an integer. The correction unit 6 performs processing (pixel value interpolation processing) in which pixel values of coordinates whose coordinate values are not integers are determined by interpolation from surrounding pixel values on the pre-correction image. For example, the correction unit 6 performs pixel value interpolation processing by bilinear interpolation (bilinear interpolation). The correction unit 6 performs geometric correction of the captured image by coordinate conversion using a correction formula and pixel value interpolation processing for all pixels to generate a corrected image.

  FIG. 4 is a flowchart for explaining an example of a correction method for the captured image 11 of the imaging apparatus 10 according to the present embodiment. The first photographing unit 1 and the second photographing unit 2 photograph the photographed image 11 (first photographed image and second photographed image) (step S1). Based on one or more captured images 11, the update unit 4 corrects the displacement of the coordinates of the captured image 11 caused by the positional displacement of the first capturing unit 1, the second capturing unit 2, or both capturing units over time. The coefficient (correction parameter 12) of the correction formula to be updated is updated and stored in the storage unit 3 (step S2).

  The division update unit 5 determines whether or not to divide the coefficient (correction parameter 12) of the correction formula (step S3).

  Here, an example of the determination method in step S3 will be described. The division updating unit 5 calculates the coefficient (correction parameter 12) of the correction formula based on the distance between the coordinate after correction using the correction formula before updating the coefficient and the coordinate after correction using the correction formula after updating the coefficient. You may determine whether to divide | segment. Here, an example of which coordinate is selected as the coordinate for obtaining the distance will be described. FIG. 5 is a diagram for explaining an example of a method for selecting coordinates for calculating the moving distance of the photographing apparatus 10 according to the present embodiment. In general, the four corners often move greatly due to the rotation of the camera. Therefore, here, for the coordinates of the upper, lower, left and right ends (points a to d) of the image area, the coefficients are the coordinates after correction by the correction formula before update. Then, the distance from the coordinate after correction by the correction formula after updating the coefficient is calculated. Next, the division update unit 5 acquires, for example, the maximum value from the four distances. The division update unit 5 may determine whether or not to divide the coefficient of the correction formula (correction parameter 12) based on the maximum value of the distance.

  Returning to FIG. 4, when the coefficient (correction parameter 12) of the correction formula is divided (step S3, Yes), the process proceeds to step S4. The division update unit 5 determines the division correction parameter 13 from the correction parameter 12 (step S4). Specifically, for example, the division update unit 5 calculates the difference between the coefficient before update and the coefficient after update, and sequentially adds the value obtained by dividing the difference into a plurality of coefficients before update. A coefficient (division correction parameter 13) for updating the plurality of times is determined. The division update unit 5 stores the division correction parameter 13 in the storage unit 3 (step S5). Details of the processes of steps S4 and S5 by the division update unit 5 will be described later. When the coefficient (correction parameter 12) of the correction formula is not divided (step S3, No), the process proceeds to step S6.

  The correction unit 6 corrects the captured image 11 (step S6). The correction unit 6 corrects the captured image 11 at a time when the correction parameter 12 is used. When the division correction parameter 13 is stored in the storage unit 3, the correction unit 6 uses the division correction parameter 13 to correct the captured image 11 in a plurality of times.

  Next, details of the processing of steps S4 and S5 by the division update unit 5 will be described. FIG. 6 is a diagram for explaining an example in which a captured image of the imaging apparatus 10 according to the present embodiment is corrected by being divided into a plurality of times. (X, y) is the coordinates of the captured image. (X ′, y ′) are coordinates after correction by the correction formula before updating the coefficient of the correction formula. (X ″, y ″) are coordinates after correction by the correction formula after updating the coefficient of the correction formula. As the distance from (x ′, y ′) to (x ″, y ″), for example, the distance calculated in the description of FIG. 5 is used.

  In the present embodiment, the division update unit 5 changes the position of the correction-type correction image by one change of the division correction parameter 13 to a change of 1/10 pixel at the maximum. The reason for this will be described. In the imaging device 10 according to the present embodiment, the minimum parallax output by the parallax calculation unit 7 is one pixel. Therefore, even if the parallax is changed by a distance sufficiently smaller than this one pixel, for example, 1/10 pixel by correction (automatic calibration) by the correction unit 6, the measured distance is 1 / 1.1≈90. .9% (distance is proportional to the reciprocal of parallax) and changes of 10% or less. That is, the influence on the processing of the parallax calculation unit 7 is small. Therefore, erroneous recognition does not occur in a control system that uses the parallax information.

  The division update unit 5 generates the division correction parameter 13 by dividing the distance from (x ′, y ′) to (x ″, y ″) by the number of times divided by the length of 1/10 pixel. If the remainder length occurs, the division update unit 5 generates a division correction parameter 13 for correcting once more by the remainder length. If the result of division is too large, processing may take time. When the time until the end of the update process becomes a problem, the upper limit of the number of divisions may be limited to 10 times, for example. In this case, assuming that the processing interval of the stereo camera (the first photographing unit 1 and the second photographing unit 2) of the present embodiment is 10 fps, the time until the end of the update process corresponds to one second. Note that the length of the division update unit 5 that divides the distance from (x ′, y ′) to (x ″, y ″) (hereinafter referred to as “division length”) is not limited to 1/10 pixel.

Specifically, the division update unit 5 determines the division correction parameter 13 as follows. The division update unit 5 determines the coefficient of the correction formula used for the i-th correction for each term of the correction formula by p ₀ + {i * (p ₁ −p ₀ )} / N. Here, p ₀ is a coefficient of the correction formula before being updated by the updating unit 4. Further, p ₁ is a coefficient of the correction formula after being updated by the updating unit 4. N is the number of divisions. By using such a coefficient, the coordinates after the divisional update at each stage draws a trajectory as illustrated in FIG. 6 and is finally corrected by the coefficient of the correction formula updated by the update unit 4. Converted to the resulting coordinates. In other words, the division update unit 5 moves the coordinates of the conversion destination by equal distances by equally dividing the variation amount of the coefficient of each term of the correction formula. The division update unit 5 stores the division correction parameter 13 in the storage unit 3.

  The correction unit 6 corrects the coefficient of the correction formula once for each frame operation of the stereo camera (the first imaging unit 1 and the second imaging unit 2) so that the corrected image 14 is gradually deformed for each frame. Run one by one. When a series of division corrections of the captured image 11 by the correction unit 6 is completed, the imaging apparatus 10 determines the coefficient of the correction formula according to the positional deviation of the stereo camera (the first imaging unit 1 and the second imaging unit 2) with time. The process of updating is started again.

  Next, an example of a hardware configuration of the imaging apparatus 10 according to the present embodiment will be described. FIG. 7 is a diagram illustrating an example of a hardware configuration of the imaging apparatus 10 according to the present embodiment. The photographing apparatus 10 according to the present embodiment includes a camera 21, a camera 22, a control unit 23, a main storage unit 24, an auxiliary storage unit 25, and a communication I / F unit 26. The camera 21, camera 22, control unit 23, main storage unit 24, auxiliary storage unit 25, and communication I / F 26 are connected to each other via a bus 27.

  The control unit 23 executes the program read from the auxiliary storage unit 25 to the main storage unit 24. The main storage unit 24 is a memory such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The auxiliary storage unit 25 is a memory card or the like. The communication I / F 26 is an interface for communicating with other devices via a network.

  The program executed by the photographing apparatus 10 of the present embodiment is an installable or executable file and can be read by a computer such as a CD-ROM, memory card, CD-R, or DVD (Digital Versatile Disk). Recorded on a simple recording medium and provided as a computer program product.

  Further, the program executed by the imaging apparatus 10 of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the program executed by the photographing apparatus 10 of the present embodiment may be configured to be provided or distributed via a network such as the Internet.

  Further, the program of the photographing apparatus 10 according to the present embodiment may be configured to be provided by being incorporated in advance in a ROM or the like.

  The program executed by the imaging apparatus 10 according to the present embodiment has a module configuration including the above-described functional blocks (update unit 4, division update unit 5, correction unit 6, and parallax calculation unit 7). As the actual hardware, each functional block is loaded onto the main storage unit 24 when the control unit 23 reads and executes the program from the storage medium. That is, each functional block is generated on the main storage unit 24.

  In addition, some or all of the above-described units (update unit 4, division update unit 5, correction unit 6, and parallax calculation unit 7) are realized by hardware such as IC (Integrated Circuit) without being realized by software. May be. The storage unit 3 is, for example, an auxiliary storage unit 25. The first photographing unit 1 corresponds to the camera 21. The second photographing unit 2 corresponds to the camera 22. In addition, in the imaging device 10 of this Embodiment, the number of imaging | photography parts was set as two, and the case where it was a stereo camera was demonstrated to the example. However, the number of photographing units is not limited to two, and one or more photographing units may be used.

  When the correction amount is large when correcting the deviation of the coordinates of the captured image 11, the imaging apparatus 10 of the present embodiment uses the correction formula obtained by dividing the coefficient and updates the captured image in multiple times. 11 is corrected. Therefore, according to the imaging device 10 of the present embodiment, there is an effect that no erroneous recognition occurs in the control system that uses the corrected captured image 11 (corrected image 14).

DESCRIPTION OF SYMBOLS 1 1st imaging | photography part 2 2nd imaging | photography part 3 Memory | storage part 4 Update part 5 Division | segmentation update part 6 Correction | amendment part 7 Parallax calculation part 10 Imaging device 11 Captured image 12 Correction parameter 13 Division | segmentation correction parameter 14 Correction | amendment image 15 Parallax image 21 Camera 22 Camera 23 Control Unit 24 Main Storage Unit 25 Auxiliary Storage Unit 26 Communication I / F Unit 27 Bus

JP 2001-160137 A

Claims (5)

One or more shooting sections for taking a shot image;
An update unit that periodically updates a coefficient of a correction formula for correcting a shift in coordinates of the captured image caused by a positional shift of the captured unit over time based on the one or more captured images;
For each term of the correction formula, the difference between the coefficient before update and the coefficient after update is calculated, and the coefficient is obtained by sequentially adding a value obtained by dividing the difference into a plurality of coefficients before update. A split update unit that updates in batches,
A correction unit that corrects the captured image using the correction formula in which the coefficient is updated. The split update unit
The imaging device according to claim 1, wherein the difference is divided into a plurality of values so that values obtained by dividing the difference into a plurality of values are equal. The split update unit
Divide the distance between the coordinate after the correction by the correction formula before updating the coefficient and the coordinate after the correction by the correction formula after updating the coefficient by a division length representing a predetermined length, and The imaging apparatus according to claim 1, wherein the coefficient is updated in a plurality of times so that the correction formula in which the corrected coordinates move by the division length is determined by the correction. The imaging device
The first photographing unit for photographing the first photographed image and the two photographing units as the second photographing unit for photographing the second photographed image,
The split update unit
The imaging apparatus according to claim 3, wherein the division length is set to a value smaller than a minimum parallax representing a length as a unit of measurement of parallax between the first captured image and the second captured image. Computer
An update unit that periodically updates a coefficient of a correction formula for correcting a shift in the coordinates of the captured image caused by a positional shift of the captured unit over time based on one or more captured images;
For each term of the correction formula, the difference between the coefficient before update and the coefficient after update is calculated, and the coefficient is obtained by sequentially adding a value obtained by dividing the difference into a plurality of coefficients before update. A split update unit that updates in batches,
The program for functioning as a correction | amendment part which correct | amends the said picked-up image using the said correction formula which updated the said coefficient.

Images