JP6443502B2 - Image processing apparatus, photographing apparatus, control system, moving object, and program - Google Patents

Description

The present invention relates to an image processing device, a photographing device, a control system, a moving object, and a program .

  In recent years, in-vehicle distance measuring devices have been put into practical use. In such a distance measuring device, the front of the vehicle is photographed by a stereo camera mounted on an automobile, an obstacle is recognized from the photographed image, and the distance to the obstacle is measured. When the distance measuring device is applied to a moving body such as an automobile, the driver is warned to prevent a collision based on the measurement result from the distance measuring device, or the brake, steering, etc. are used for controlling the distance between the vehicles. It is possible to operate the control device.

Here, the principle of distance measurement using a stereo camera will be described.
FIG. 10 is a diagram illustrating the principle of distance measurement using a stereo camera.
A stereo camera is configured, for example, by arranging two cameras in parallel. In FIG. 10, a first camera 100 a having a focal length f, an optical center O ₀ , and an imaging plane S ₀ is arranged with the upper surface of the paper as the optical axis direction, and a second camera 100 b having the same focal length f is The first camera 100a is arranged in parallel on the right side by a distance B.
The image of the subject A located at a distance d in the optical axis direction from the optical center O ₀ of the first camera 100 a forms an image at P ₀ that is the intersection of the straight line A-O ₀ and the imaging surface S ₀ .

On the other hand, the second camera 100b, the same subject A is forms an image at a position _{P 1} on the imaging surface _{S 1.}
Here, as the optical center _{O 1} of the second camera 100b, straight lines parallel to the _{A-O 0,} an intersection between the imaging surface _{S 1} 'and, _{P 0' P 0} a distance of _{P 1} and p And P ₀ ′ is the same position as the image P ₀ on the first camera 100a, and the distance p represents the amount of positional deviation of the same subject image on the images taken by the two cameras. This is called parallax.
Since the triangle A-O ₀ -O ₁ and the triangle O ₁ -P ₀ ′ -P ₁ are similar,
d = B × f / p (1)
If the base line length B and the focal length f are known, the distance d can be obtained from the parallax p.

The above is the principle of distance measurement using a stereo camera.
In order to establish this principle, the first camera 100a and the second camera 100b must be accurately arranged as shown in FIG.
However, since it is very difficult to eliminate the tolerance when assembling the two cameras and to arrange them strictly in parallel, various techniques for correcting by signal processing have been proposed.

For example, in Patent Document 1, after assembling a stereo camera, a test chart set at a known position is photographed to accurately measure the relative positional relationship between the two cameras and use the information. Thus, there is disclosed a technique for realizing a pseudo parallel arrangement by deforming captured image data. At this time, in Patent Document 1, affine transformation is adopted as a modification of image data.
Patent Document 2 discloses a technique using a table in order to correct a non-linear shift that cannot be dealt with by affine transformation alone, such as distortion aberration of an imaging lens system. The contents of the table are determined based on data obtained by photographing a test chart set at a known position.
Patent Document 3 discloses a technique for dealing with a deviation from an initial state that occurs during actual use due to vibration or temperature change. During actual use, it is difficult to photograph a known test chart as at the time of manufacture, so the amount of deviation is measured using only the parallelism of the white lines on the road surface.

By the way, in order for the distance measuring apparatus as described above to maintain high accuracy, accurate calibration is required. For this reason, various calibration techniques have been conventionally proposed.
Conventional calibration techniques can be broadly divided into two types: calibration execution timing and image deformation freedom.
Furthermore, the timing for executing calibration can be divided at the time of manufacture and at the time of use. For example, in calibration at the time of manufacturing, optical distortion of a lens, deviation at the time of assembly, and the like are measured and corrected at the time of camera manufacturing. On the other hand, in the calibration at the time of use, a shift due to the influence of a change with time due to temperature change or vibration is measured and corrected during use.
The degree of freedom of image deformation includes affine transformation, nonlinear table transformation, parallel movement (fixed offset), and the like.

However, in the calibration performed at the time of use, for example, since a known subject such as a test chart cannot be photographed, image conversion other than a certain offset cannot be performed. For this reason, it has been difficult to accurately measure the amount of lateral image shift.
This is because, when the two cameras that make up the stereo camera are installed in an ideal parallel state with no tolerance during assembly, for example, the left image taken with one camera and the other camera Since the corresponding point in the right image always has a horizontal parallax, if there is a vertical component of the positional difference between the corresponding points, it is easy to cause a vertical image shift from the positional difference between the corresponding points. Can be estimated. On the other hand, since the true value of the parallax in the horizontal direction depends on the distance to the subject, the amount of shift in the horizontal direction is difficult to detect and cannot be estimated easily.

  In Patent Document 3, the parallelism of white lines and the number of still subjects are used to estimate the parallax offset, that is, the horizontal translation amount of the entire screen of the left and right images. Since the horizontal image shift of a stereo camera with two cameras arranged in parallel is not necessarily a parallel movement, measure the horizontal image shift without using a known test chart during actual use. Corrections could not be made.

Here, the types of lateral deviation that occur in a stereo camera will be described.
For example, the lateral shift generated in the stereo camera includes a lateral shift due to rotation around the vertical axis and a lateral shift due to a change in distortion characteristics.
The lateral shift due to the rotation around the vertical axis moves in the opposite direction (lower upper right and left) of the captured image when the line-of-sight direction of the camera is slightly changed up and down and left and right. However, the movement of the image at this time is not strictly a parallel movement. For example, FIG. 11B shows how a rectangle facing the camera is deformed when the camera 100 is tilted to the left as shown in FIG.
Although it is particularly emphasized in FIG. 11B, when the camera 100 is tilted leftward as shown in FIG. 11A, the captured image of the camera 100 is deformed from the rectangle 110 to the trapezoid 120. In this case, the image shifts to the right on the entire screen, but the left side of the screen is reduced and closer to the center. Therefore, the point A at the left end moves greatly to the right compared to the point B near the center of the screen. The point C at the right end also moves greatly to the right in order to enlarge and move away from the center. As a result, the amount of shift in the right direction of the image is not uniform over the entire screen as shown in FIG.
Conventionally, if the angle of deviation is very small, the trapezoid 120 shown in FIG. 11B is very close to the rectangle 110, so it was regarded as approximately parallel movement, but for more accurate parallax detection. Is not a simple parallel movement, but must be corrected assuming that there is a shift amount according to the pixel position.

Next, a lateral shift due to a change in distortion characteristics will be described. The imaging lens system generally has distortion characteristics. In the initial state, a non-linear distortion can be corrected using a table as in Patent Document 2, but the distortion characteristics may change over time. In this case, for example, there is little change near the center of the screen, but image displacement other than translation occurs, such as enlargement or reduction near the periphery. In addition, if any of the lens system such as lens tilt and decentration is symmetric with respect to the optical axis, anisotropic distortion may occur, resulting in a more complicated shift.
As described above, the conventional calibration technique in the distance measuring apparatus has a problem that the image shift amount cannot be accurately measured and corrected in a state where the distance measuring apparatus is actually operated.
The present invention has been made in view of the above points, and an object thereof is to calculate a parallax correction amount corresponding to a pixel position in consideration of a non-uniform image shift amount.

In order to achieve the above object, the invention according to claim 1 is characterized in that an extraction means for extracting a plurality of feature points in a first image photographed by a first photographing means mounted on a moving body, the first image, A plurality of parallaxes respectively corresponding to the plurality of feature points extracted using a second image photographed by a second photographing unit mounted on the moving body from a viewpoint different from the first image at the same time as the first image Parallax calculating means for calculating data, each of a plurality of parallax data for the first image and the second image taken at the first time, and taken at a second time different from the first time in association with the each of the plurality of parallax data for the first image and the second image, specifying means for specifying a plurality of parallax data pairs, the image position of the feature point, each corresponding to the plurality of parallax data pairs Meet Te, the characteristics and the correction amount calculating means for calculating a parallax correction amount corresponding to the image position, by the disparity correction amount corresponding to the image position, that and a parallax correction means for correcting in accordance with the image position To do.

  According to the present invention, it is possible to calculate a parallax correction amount corresponding to a pixel position.

It is the block diagram which showed the structure of the whole hardware of the distance measuring device which concerns on 1st Embodiment of this invention. It is a block diagram of the whole software module performed with the parallax calculation part 20 shown in FIG. It is the flowchart which showed the 1st process operation | movement of the parallax image generation process part. It is the figure which showed an example of the area | region division of a screen. It is a figure which shows the example of calculation of the parallax correction amount by 1st Embodiment. It is the flowchart which showed the 2nd processing operation of the parallax image generation process part. It is the figure which showed an example of the deformation | transformation process of a reference | standard image. It is the figure which showed an example of the deformation | transformation process of a reference image. The schematic diagram which shows the vehicle which can apply the distance measuring device of this embodiment. It is a principle explanatory view of distance measurement by a stereo camera. It is the figure which showed an example of the example of the distortion of the image by rotation of a camera. It is the figure which showed the amount of lateral deviation of the image by rotation of a camera.

Hereinafter, a distance measuring device according to an embodiment of the present invention will be described.
<First Embodiment>
FIG. 1 is a block diagram showing the overall hardware configuration of the distance measuring apparatus according to the first embodiment of the present invention.
A distance measuring device 1 according to this embodiment shown in FIG. 1 includes a stereo camera (image capturing unit) 11 and a parallax calculating unit (parallax calculating unit) 20 in which a left camera 11a and a right camera 11b are arranged in parallel. .
The left camera 11a and the right camera 11b capture images from different viewpoints.
The parallax calculation unit 20 receives images taken by the left camera 11a and the right camera 11b, and outputs parallax image data.
The parallax calculation unit 20 is configured by a general electronic computer including a CPU (Central Processing Unit), a DRAM (Dynamic Random Access Memory) memory, a nonvolatile flash memory, and the like connected by a bus. The parallax calculation function is recorded in a flash memory and is implemented in the form of a software program that is executed using a CPU, DRAM, or the like.
In the present embodiment, the case where the stereo camera 11 includes two cameras 11a and 11b has been described as an example. However, this is merely an example, and the configuration is configured using three or more cameras. It goes without saying that it is also good.

FIG. 2 is a block diagram of the entire software module executed by the parallax calculation unit 20 shown in FIG.
The camera control unit 21 controls the left camera 11a and the right camera 11b, and inputs image data. Also, synchronization of two cameras, camera initialization, exposure control, etc. are executed. Although not described here, it is desirable to perform image correction processing other than the parallax offset, which is generally effective for stereo distance measurement, such as distortion due to an optical system and image rotation, as necessary. In addition, the camera control unit 21 sequentially outputs two images taken simultaneously by the two left and right cameras.
Here, a captured image captured by the right camera 11b is referred to as a reference image, and a captured image captured by the left camera 11a is referred to as a reference image.
The parallax image generation processing unit 22 calculates the parallax for each pixel from the two pieces of image data of the standard image and the reference image input from the camera control unit 21 and generates a pre-correction parallax image. Details of the parallax image generation processing unit 22 will be described later.

The parallax correction amount calculation processing unit (estimating unit) 23 has a function as, for example, a region parallax correction amount calculation processing unit 23a that calculates a parallax correction amount for each region. Then, the parallax correction amount for each X coordinate is calculated using the reference image input from the camera control unit 21 and the pre-correction parallax image input from the parallax image generation processing unit 22. That is, [i, off (i)] table information corresponding to the number of pixels of the screen width is generated with the parallax correction amount of X coordinate = i being off (i). The calculated parallax correction amount is sent to the parallax correction processing unit 24 and used for the parallax image correction processing. Details of the area parallax correction amount calculation processing unit 23a will be described later.
The parallax correction processing unit (correction unit) 24 subtracts the correction amount corresponding to the pixel position from the correction amount input from the parallax correction amount calculation processing unit 23 from the parallax value of each pixel of the parallax image. Correct the image data. That is, if the parallax value at the pixel position (i, j) in the input parallax image is d (i, j) and the parallax correction amount at the X coordinate = i is off (i), O (i, j) = D (i, j) -off (i) is output as the corrected parallax value.

Next, the processing operation of the parallax image generation processing unit 22 will be described with reference to the flowchart shown in FIG.
In step S1, a stereo image is input. That is, images taken simultaneously by two cameras arranged in parallel are input. In the subsequent step S2, feature points are extracted.
In step S3, a corresponding point is searched. That is, first, a plurality of feature points having a sharp change in density are extracted from a reference image taken by the right camera 11b as a reference. Next, from the reference image taken by the opposite left camera 11a, the position of the corresponding point where the same subject as the area near each feature point (block) on the base image is found is searched. A known technique such as SAD (Sum of Absolute Difference) or POC (Phase Only Correlation) can be used for the corresponding point search.
In step S4, the parallax is calculated. That is, the difference between the corresponding point positions on the left and right images calculated by the corresponding point search process is taken to calculate the parallax.
In step S5, a parallax image is output. That is, parallax image data (parallax information) having the calculated parallax amount as a pixel value is output.
By periodically executing the above processing, the parallax image in front of the camera is always output.

Next, the regional parallax correction amount calculation processing unit 23a will be described.
The area parallax correction amount calculation processing unit 23a calculates a parallax offset based on a set of parallax data pairs of the same subject (feature point) between two stereo shootings as a parallax offset calculation process for each area. However, depending on which of the three regions (the first region E1, the second region E2, and the third region E3) in FIG. 4 the position of the feature point on the first reference image belongs, Classify into three sets. That is, as shown in FIG. 4, the X coordinate value of the boundary between the first region E1 and the second region E2 is X _th1 , the boundary of the second region E2 and the third region E3 is X _th2 , and the coordinate value of the feature point is (X , Y)
Feature points _satisfying X <X _th1 are in set 1,
Feature points _satisfying X> X _th2 are in set 3,
Otherwise, it is determined as set 2.
Statistical processing is performed on each set independently to estimate three parallax offset amounts for each region.
Here, if the number of feature points belonging to any region is smaller than a predetermined threshold, the estimation result up to the previous frame is used as it is without performing the parallax offset estimation for that region.

Next, the parallax correction amount calculation process for each X coordinate will be described.
The X coordinates of the centers of the first region E1, the second region E2, and the third region E3 are referred to as X ₁ , X ₂ , and X ₃ , respectively.
The result of the parallax offset calculation step for each of the first region E1 to the third region E3 is regarded as a correction amount at three points of X coordinates = X ₁ , X ₂ , X ₃ , A correction amount at the X coordinate is calculated. That is, the correction amount in the range of X <X ₂ is obtained by linear interpolation of the parallax offset in X ₁ and X ₂ , and the correction amount in the range of X ₂ <X is linear in the parallax offset in X ₂ and X ₃ Find by interpolation. A table of (X coordinate, correction amount) corresponding to the image width for each pixel thus obtained is output to the parallax correction processing unit 24.

FIG. 5 is a diagram illustrating a calculation example of the parallax correction amount according to the first embodiment.
The horizontal axis represents the X coordinate, and the vertical axis represents the parallax correction amount. In the figure, white circles represent parallax offsets (= parallax correction amounts) for every three regions, and solid line A represents a correction amount for all X coordinates obtained by linear interpolation.
In FIG. 5, the ideal correction amount shown in FIG. 12 is indicated by a broken line B, and a case where a constant parallax offset as in the conventional case is used is indicated by a one-dot chain line C. FIG. 5 shows that the correction amount (solid line A) according to the present embodiment is closer to the ideal amount (dashed line B) than the conventional constant value (dashed line C).
In the first embodiment, the horizontal division shown in FIG. 4 is adopted as the area division. However, this is merely an example, and may be set in accordance with the characteristics of the camera over time. For example, the number of divisions can be increased or decreased and vertical and horizontal two-dimensional division can be performed, and gaps or overlaps can be provided between regions.
In the first embodiment, as an example of the parallax correction amount calculation processing unit 23, the region parallax correction amount calculation processing unit 23a that calculates the parallax correction amount for each of a plurality of regions has been described as an example. This is merely an example, and the parallax correction amount calculation processing unit 23 may have other configurations.

In the first embodiment, the correction amount table for each pixel of the X coordinate is adopted as the correction amount according to the pixel position. However, other expressions for expressing a line in the X coordinate-correction amount space such as a polynomial, a broken line, and a spline are used. Format data can also be used.
In the first embodiment, the parallax correction amount calculation process is performed every two frames. However, the reliability evaluation is performed to reject information on frames with low reliability, or frame accumulation is performed to obtain more accuracy. And stability can be improved.
Thus, in the distance measuring device 1 of the first embodiment, the same subject (feature point) between two parallax images taken from two points (time) using the two cameras 11a and 11b. A large number of two pairs of disparity data on. Then, the regional parallax correction amount calculation processing unit 23a measures the parallax offset by statistically processing the set of parallax data pairs.
Since the feature points originally occupy a specific position on the screen, in the first embodiment, the parallax data pairs for each image area are detected by classifying the parallax data pairs according to the positions of the feature points. Yes.

For example, as shown in FIG. 4, the entire screen is divided into a first area E1, a second area E2, and a third area E3 from the left, depending on which area the three feature points belong to on the captured image. Thus, the parallax data pairs are classified into three sets. By performing statistical processing on each set, the region parallax correction amount calculation processing unit 23a can estimate the amount of parallax offset in each image region.
Exactly, what can be estimated by this method is not the amount of deviation of the captured image itself as shown in FIGS. 11 and 12, but the amount of change in parallax between the two captured images. Since the parallax changes according to the subject distance, this does not coincide with the difference in the shift amount at the same position of the two images. However, when the change in the shift amount changes gradually with respect to the pixel position as shown in FIG. 12, and the target parallax is small with respect to the change rate (for example, a parallax of 1/20 or less of the screen width). In the case of handling), the parallax offset can be regarded as a difference between the shift amounts of the two images.
Furthermore, the smaller the image shift, the greater the effect of the smaller the parallax. For example, a shift of 0.5 pixels is only an error of 0.5% for a parallax of 100 pixels, but an error of 50% for a parallax of 1 pixel.

Therefore, in the first embodiment, the parallax offset amount corresponding to the pixel position is regarded as a difference between the two image shift amounts, and the parallax correction processing unit 24 serving as a correction unit uses this to obtain parallax data (parallax information). Was corrected.
As a result, a non-uniform lateral image shift can be corrected without using a known subject such as a test chart during use, so that a more accurate parallax image can be generated.

In the first embodiment, the parallax generation processing unit (corresponding point position difference calculating unit) 22 searches for corresponding points between a plurality of images, and calculates parallax data that is a difference in coordinates between the corresponding points. Since the parallax correction processing unit (parallax correction unit) 24 corrects the parallax data based on the correction amount from the parallax correction amount calculation processing unit (estimation unit) 23, the pixel position is calculated by a simple calculation. There is an advantage that the pixel lateral shift corresponding to can be corrected.
Incidentally, in the conventional method based on the white line on the road surface as in Patent Document 1, only an average parallax offset amount in the entire image area where the white line on the road surface is captured can be measured. And since the position where the white line on the road surface appears on the camera installed in front of the car does not always change around the lower half of the screen, the offset measurement for each area cannot be performed. On the other hand, in this embodiment, offset measurement for each region can be realized by using a parallax offset measurement method based on statistical processing based on feature points.

Second Embodiment
Next, a second embodiment of the present invention will be described.
In the first embodiment, the parallax data is corrected, but in the second embodiment, the reference image used for generating the parallax image is corrected. In this case, if there is an image correction process for correcting distortion, assembly error, and the like, the post-processing of correcting the parallax image becomes unnecessary by incorporating the correction process at that stage.
The overall configuration and hardware configuration of the second embodiment are the same as those of the first embodiment shown in FIG. Hereinafter, differences from the first embodiment of the software processing contents will be mainly described.

FIG. 6 is a block diagram of the entire software module executed by the parallax calculation unit 20. The same blocks as those in FIG. 2 are denoted by the same reference numerals and description thereof is omitted.
The camera control unit 21 shown in FIG. 6 controls two cameras as in the first embodiment, and outputs a standard image and a reference image.
The reference image correction processing unit 31 deforms and corrects the image according to the deviation and distortion characteristics measured when the camera is manufactured.

FIG. 7 is a diagram illustrating an internal block configuration of the reference image correction processing unit 31.
In general, the image transformation process includes a pixel position generation table (coordinate generation processing unit) 41 that calculates pixel positions on the corresponding input image for all pixel positions of the output image, and an input corresponding to the calculated pixel position. The pixel interpolation unit 42 calculates pixel values on the image. The pixel value is not necessarily an integer value.
In the second embodiment, XY coordinates for all pixels are recorded in advance as a table, and the pixel position on the input image is generated by referring to the table. The table is generated by photographing a test chart placed at a known position at the time of camera manufacture, interpolating between feature points, and recording it inside the camera. In the second embodiment, a table is used, but the pixel position can also be generated by a method other than the table, such as coordinate conversion by polynomial calculation.
The image correction amount calculation processing unit 32 is the same as the parallax correction amount calculation processing unit 23, and divides the image area as shown in FIG. 4, obtains a parallax offset for each area, and interpolates them. Output the correction amount for all pixels. However, in the second embodiment, the correction amount output by the image correction amount calculation processing unit 32 is used for image correction.
The reference image correction processing unit (image correction unit) 33 deforms and corrects an image in the same manner as the standard image correction processing unit 31. However, although the standard image was corrected based on the deviation characteristics measured at the time of camera manufacture, the reference image correction occurred in addition to the characteristics measured and recorded in advance, and occurred at the time of use calculated by the image correction amount calculation processing unit 32 Also correct the deviation.

FIG. 8 shows an internal block diagram of the reference image correction processing unit 33.
The reference image correction processing unit 33 includes a pixel position generation table 41, a pixel interpolation unit 42, and a pixel position correction processing unit 43. Note that the same blocks as those in FIG.
The table contents of the pixel position generation table 41 are measured and recorded when the camera is manufactured, as described above.
The pixel position correction processing unit 43 uses the image correction amount for each X coordinate input from the image correction amount calculation processing unit 32 to correct the pixel position data recorded in the table.
Specifically, the correction amount off (i) corresponding to the X coordinate is added to the X coordinate of the coordinates (X, Y) read from the table and output. Thereby, when the parallax is too large, that is, when the parallax is on the right side compared to the base image, the reference image is deformed to move to the left side, so that the parallax calculated later can be reduced. When the X coordinate is not an integer, it may be converted to an integer by rounding off, for example.

The parallax image generation processing unit 22 receives the corrected standard image and reference image, and generates a parallax image by the same processing as in the first embodiment. Since the input reference image is shifted in the horizontal direction by the amount of the parallax offset that has already been calculated by the image correction amount calculation processing, noise for the amount of parallax offset is reduced.
In the second embodiment, the result of the image correction amount calculation processing unit 32 is reflected only in the correction of the reference image. Conversely, the result is reflected in the correction on the base image side, or the correction amount is distributed to the base image and the reference image. You can also That is, not only the reference image correction processing unit 33 but also the standard image correction processing unit 31 or both the reference image correction processing unit 33 and the standard image correction processing unit 31 may function as image correction means.

As described above, in the second embodiment, the parallax offset amount corresponding to the pixel position is regarded as the difference between the two image shift amounts, and the reference image is corrected using this difference.
This makes it possible to correct non-uniform lateral image shifts without using a known subject such as a test chart during operation.
In the second embodiment, the reference image correction processing unit 33 corrects the reference image based on the correction amount from the image correction amount calculation processing unit 32. Then, the corresponding point between the reference image corrected by the parallax image generation processing unit 22 and the standard image is searched, and the parallax data that is the difference in coordinates between the corresponding points is calculated. There is an advantage that the pixel lateral shift according to the position can be corrected.

<Other embodiments>
FIG. 10 is a schematic diagram of an image processing system to which the distance measuring device 1 of the present embodiment is applied and a vehicle on which the image processing system is mounted.
The image processing system shown in FIG. 10 performs processing such as calculating the distance to an imaging unit 51 for acquiring an image in front of the vehicle and another vehicle existing in front of the vehicle 50 based on the acquired image. An image analysis unit 52 is included. The imaging unit 51 is installed at the position of a seat mirror or the like so that an image in front of the vehicle 50 traveling can be taken. An image in front of the vehicle imaged by the imaging unit 51 is converted into an image signal and input to the image analysis unit 52. The image analysis unit 52 analyzes the image signal output from the imaging unit 51.
As the imaging unit 51, the stereo camera 11 of the above embodiment can be applied.
Further, as a part of the function of the image analysis unit 52, the parallax calculation unit 20 of the above-described embodiment can be applied.
The vehicle travel control unit 53 can also control the steering wheel and the brake based on the distance calculated by the image analysis unit 52.

  DESCRIPTION OF SYMBOLS 11 ... Stereo camera, 11a ... Left camera, 11b ... Right camera, 20 ... Parallax calculation part, 21 ... Camera control part, 22 ... Parallax image generation process part, 23 ... Parallax correction amount calculation process part, 24 ... Parallax correction process part Reference image correction processing unit 32 Image correction amount calculation processing unit 33 Reference image correction processing unit 41 Pixel position generation table 42 Pixel interpolation unit 43 Pixel correction processing unit

Japanese Patent No. 3792932 Japanese Patent No. 3261115 Japanese Patent No. 457397

Claims (8)

Extracting means for extracting a plurality of feature points in the first image photographed by the first photographing means mounted on the moving body ;
The plurality of feature points extracted by using the second image taken by the second photographing unit mounted on the moving body from a different viewpoint from the first image and the first image simultaneously with the first image. Parallax calculation means for calculating a plurality of corresponding parallax data,
Each of the plurality of parallax data regarding the first image and the second image captured at the first time, and the first image and the second image captured at a second time different from the first time A specifying means for associating each of the plurality of parallax data and specifying a plurality of parallax data pairs;
In accordance with the image position of the feature point, each corresponding to the plurality of parallax data pairs, a correction amount calculation means for calculating the parallax correction amount corresponding to the image position,
Parallax correction means for performing correction according to the image position with a parallax correction amount according to the image position;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the correction amount calculating unit sets a plurality of areas in a screen and calculates the parallax correction amount for each of the plurality of areas. A corresponding point position difference calculating unit that searches for corresponding points between a plurality of captured images and calculates a difference in coordinates between the corresponding points is provided.
The image processing apparatus according to claim 1 , wherein the parallax correction unit corrects the corresponding point position difference calculated by the corresponding point position difference calculation unit based on the parallax correction amount.   The image processing apparatus according to claim 1, further comprising an image correction unit configured to correct at least one of the plurality of captured images based on the parallax correction amount. First photographing means for photographing the first image;
A second photographing means for photographing the second image;
Photographing apparatus characterized by comprising an image processing apparatus according to any one of claims 1 to 4. An imaging device according to claim 5;
Control system characterized in that it comprises a control unit that performs control based on the disparity data calculated by the image processing apparatus. A control system according to claim 6,
A moving body controlled by the control unit. Computer
Extracting means for extracting a plurality of feature points in the first image photographed at a predetermined time by the first photographing means mounted on the moving body ;
A plurality of parallaxes respectively corresponding to the plurality of feature points extracted using the first image and a second image photographed by a second photographing means mounted on the moving body at the same time as the first image. Disparity calculating means for calculating data;
Each of the plurality of parallax data regarding the first image and the second image captured at a first time, and the first image and the first image captured at a second time different from the first time A specifying unit that associates each of a plurality of parallax data for two images and specifies a plurality of parallax data pairs;
In accordance with the image position of the feature point, each corresponding to the plurality of parallax data pairs, a correction amount calculation means for calculating the parallax correction amount corresponding to the image position,
Parallax correction means for performing correction according to the image position with a parallax correction amount according to the image position;
A program for causing an image processing apparatus to function.

Images