JP2017062198A - Geometrical distortion removal reproduction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a geometrical distortion removal reproduction device capable of evaluating influences to be exerted upon a captured image by geometrical distortion of a windshield for each car type, without requiring huge man-power or cost.SOLUTION: A stereo camera 2 consisting of a reference camera 2a and a comparison camera 2b images a lattice chart board in a state where a windshield is interposed in front of the stereo camera for each of different types or is not interposed. In a geometrical table creation part 8, coordinates of lattice points of the lattice chart board are found from the captured images, and a geometrical table is created for storing geometrical distortion correction data for each lattice point based on a difference between the coordinates and ideal coordinates corresponding thereto. Geometrical distortion of a travel image based on actual car travel is geometrically calibrated by the geometrical distortion correction data being stored in the geometrical table, and influences to be exerted upon the captured image by the geometrical distortion of a plurality of types of protective hoods are evaluated.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a geometric distortion removal and reproduction apparatus that evaluates the influence of a geometric distortion of a protective hood on a captured image by removing or reproducing geometric distortion of an image taken through the protective hood and a camera lens.

  Conventionally, as a technology for recognizing a space three-dimensionally, using the principle of triangulation from parallax when the same object is imaged with a stereo camera composed of two cameras arranged with a predetermined baseline length apart Image processing by a so-called stereo method for obtaining a distance is known.

  In the image processing by the stereo method, two image signals from a stereo camera are sequentially shifted and overlapped to obtain a position where the two image signals coincide with each other. It is desirable that there is only a corresponding positional shift caused by, and if there is a positional shift due to other optical distortion or the like, proper stereo matching cannot be performed, and the accuracy of the distance information decreases.

  Therefore, the present applicant detects parallax by stereo matching processing based on a pair of images including a reference image captured by a stereo camera and a comparison image in Patent Document 1 (Japanese Patent No. 3261115), and uses the parallax data as the parallax data. By correcting using a field angle correction ratio integral value table created by measuring in advance minute actual image data for each predetermined region of the comparison image, variation in the focal length of the lens, geometric distortion of the lens (hereinafter, We proposed a technique that eliminates the influence of “lens distortion”.

Japanese Patent No. 3261115

  By the way, for example, a stereo camera mounted on a moving body such as a vehicle obtains a captured image through a protective hood represented by a front window glass (hereinafter abbreviated as “front glass”) in addition to a lens. , Geometric distortion caused by the protective hood (hereinafter referred to as “glass distortion”) is likely to cause parallax errors, especially when the stereo camera is an in-vehicle camera, the windshield has a relatively wide range of vertical and horizontal directions. Is distorted.

  As a result, even if lens distortion, etc. is corrected using the angle-of-view correction ratio integration value table, the image taken through the protective hood has a problem that the equidistant surface does not become flat due to the effect of glass distortion, resulting in a decrease in distance measurement accuracy. Will be invited.

  By the way, the windshield is different for each vehicle type, and therefore the glass distortion is also different for each vehicle type. As a result, when an attempt is made to evaluate the influence of the glass distortion on the captured image, it is necessary to perform actual traveling and simulation for all vehicle types, and there is an inconvenience that enormous man-hours and costs are required.

  In view of the above circumstances, the present invention can evaluate the influence on the captured image caused by the geometric distortion of a plurality of types of protective hoods without enormous man-hours and costs, and the geometrical shape of the protective hoods. An object of the present invention is to provide a geometric distortion removal and reproduction apparatus that can reduce parallax errors caused by distortion.

  A geometric distortion elimination and reproduction apparatus according to the present invention includes a stereo camera having two cameras arranged with a predetermined baseline length apart, and a stereo camera arranged in front of the stereo camera and centered at a corresponding lattice point of each camera. A grid chart provided with markers, a protective hood interposed between the stereo camera and the grid chart, and a characteristic in which the grid chart is imaged by each camera and coordinates of each grid point of the grid chart are obtained. A geometric table that stores geometric distortion correction data for each grid point based on the difference between the coordinates of each grid point obtained by the detection unit and the characteristic detection unit and the ideal coordinates corresponding thereto is not provided with the protective hood. A geometric table creating unit that creates a state and a state in which the protective hood is interposed for each different type, and the geometric table created by the geometric table creating unit A data storage unit to be stored and a plurality of the geometric tables stored in the data storage unit are combined, and the stereo camera and the protective hoods are combined with the geometric distortion correction data stored in the respective geometric tables. A calibration operation unit that removes the geometric distortion or reproduces the geometric distortion of a different protective hood.

  According to the present invention, a plurality of geometric tables stored in the data storage unit are combined to remove or reproduce the geometric distortion between the stereo camera and each protective hood. The effect of the geometric distortion of the protective hood on the captured image can be evaluated without enormous man-hours and cost, and the parallax error caused by the geometric distortion of the stereo camera and the protective hood Can be reduced.

The positional relationship between a lattice chart board and a stereo camera that images the lattice chart board is shown, (a) is a front view, (b) is a plan view, and (c) is a side view. Circuit block diagram of geometry table creation device Circuit block diagram of geometric distortion removal and reproduction device (A) is explanatory drawing which shows the reference image of the lattice chart board imaged with the reference camera, (b) is explanatory drawing which shows the comparative image of the lattice chart board imaged with the comparison camera. Flow chart showing the geometry table creation routine (A) is explanatory drawing which shows the lattice image imaged through the lens which has lens distortion, (b) is explanatory drawing which shows the lattice image imaged through the lens which has lens distortion, and the windshield which has glass distortion, (c) is Explanatory drawing showing an image after removing glass distortion (A) is explanatory drawing which shows the lattice image imaged through the lens which has lens distortion, (b) is explanatory drawing which shows the lattice image imaged through the lens which has lens distortion, and the windshield which has glass distortion, (c) is An explanatory view showing an image after removing the glass distortion, (d) is an explanatory view showing a lattice image taken through another windshield, (e) is an image of (c) calibrated with an image of (d) Illustration of the image A conventional example is shown, (a) is an explanatory view showing a lattice image taken through a lens having lens distortion, (b) is an explanatory view showing an image after affine correction of the image of (a), (c) is a glass Explanatory drawing showing an image taken through a distorted windshield

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. As shown in FIGS. 1 and 2, the geometric table creation device 1 includes a stereo camera 2, a geometric table creation device 4 connected to the stereo camera 2, a lattice chart board Bm, and a windshield as a protective hood. Gf.

  The stereo camera 2 is the same as that actually mounted on the vehicle. The stereo camera 2 includes a reference camera 2a and a comparison camera 2b, and both the cameras 2a and 2b are fixed to the stay 2c with a predetermined baseline length. . Further, a plurality of windshields Gf are prepared for each vehicle type on which the stereo camera 2 is mounted, and the positional relationship such as distance and inclination with respect to the stereo camera 2 is set to be equivalent to that in actual vehicle mounting.

  The lattice chart board Bm is directly opposed to the stereo camera 2 at a predetermined distance. On the surface of the lattice chart board Bm, a lattice pattern in which lattice points are equally spaced at intervals of an integer of the base line length of the stereo camera 2 is described. The base line of the stereo camera 2 is the lattice chart board Bm. Are arranged at positions spaced apart from each other by a predetermined grid point. Further, center markers Rc and Lc are drawn around the lattice point that coincides with the optical axes of the cameras 2a and 2b in the center of the lattice chart board Bm.

  The center markers Rc and Lc are formed in a shape that does not affect the matching search of the lattice points and can clearly specify the starting position of the matching search with the cameras 2a and 2b in advance. In the storage units of both the cameras 2a and 2b, a reference pattern corresponding to the center markers Rc and Lc and a cross pattern corresponding to the lattice points surrounded by the center markers Rc and Lc are stored in advance.

  As shown in FIG. 2, the geometric table creation device 4 includes preprocessing units 5a and 5b for preprocessing images captured by both cameras 2a and 2b, and an input image memory 6, and each grid point on the image. A stereo image characteristic detecting unit 7 for obtaining the coordinates of the image, a geometric table creating unit 8 for creating a geometric table based on the coordinates of the lattice points obtained by the stereo image characteristic detecting unit 7, and a data storage unit 9 for storing the geometric table. I have.

  The pre-processing units 5a and 5b digitally output analog image signals of the grid chart board Bm captured by the cameras 2a and 2b through the windshield Gf for each vehicle type when the windshield Gf is not mounted. An imaging unit including various circuits such as an A / D converter that converts signals into signals. Further, address control is performed when data is written to the input image memory 15, and left and right cameras are assigned to predetermined addresses in the input image memory 15. Images (digital images) of the lattice chart board Bm captured by 2a and 2b are stored.

  Specifically, the processes in the stereo image characteristic detecting unit 7 and the geometric table creating unit 8 described above are executed according to a geometric table creating routine shown in FIG.

  As an initial setting before the execution of the routine, with the stereo camera 2 mounted on the vehicle, the stereo chart 2 is directly opposed to the grid chart board Bm, and the center markers Rc and Lc drawn on the grid chart board Bm are respectively connected to the cameras 2a and 2b. And set the distance from the grid chart board Bm so that the predetermined area of the grid chart board Bm falls within the angle of view of both cameras 2a and 2b.

  Then, the program is started. First, in step S1, variable values such as specification constant values and various processing variable values of the stereo camera 2 are initialized. Thereafter, the process proceeds to step S2, and the cameras 2a and 2b capture the grid chart board Bm. As shown in FIGS. 4A and 4B, one image of the reference image Ri and the comparison image Li is obtained. Sampling.

  Next, the process proceeds to step S3, in which the center markers Rc and Lc in the sampled reference image Ri and the comparison image Li and the lattice point drawn at the center thereof are stored in advance as reference marks stored in the cameras 2a and 2b. And by pattern matching with the cross pattern. In step S4, as shown in FIGS. 4A and 4B, based on the coordinates Rci and Lci set so that the detected center markers Rc and Lc are substantially at the center of the image, the lattice chart board Bm Set search range for.

  Thereafter, the process proceeds to step S5, where all the grid points displayed in the reference image Ri and the comparison image Li are searched, and the coordinates R11 to Rmm and L11 to Lmm of each grid point are obtained. As a method for obtaining the coordinates R11 to Rmm and L11 to Lmm of each lattice point, for example, the original lattice point stored in advance is compared with the lattice point on the image, and SAD (Sum (Sum)) which is a well-known image processing technique is used. After performing pattern matching by (of Absolute intensity Difference) matching, the coordinates R11 to Rmm and L11 to Lmm of the respective lattice points are obtained as local minimum points by performing matching with subpixel accuracy using equiangular straight line fitting. .

  Then, the process proceeds to step S6, where the coordinates R11 to Rmm and L11 to Lmm at which the lattice points of the reference image Ri and the comparison image Li obtained in step S5 described above are detected, and the ideal image of each lattice point stored in advance are displayed. Geometric distortion correction data for each grid point is obtained from the difference from the coordinates, and stored at a predetermined address in a table composed of horizontal coordinates and vertical coordinates to create the current geometric table.

  Then, it progresses to step S7, this geometric table is memorize | stored in memory, and a routine is complete | finished. Steps S1 to S5 are processing in the stereo image characteristic detecting unit 7 of the geometric table creating apparatus 4, and steps S6 and S7 are processing in the geometric table creating unit 8.

  The geometric table created in accordance with the above-described geometric table creation routine includes a geometric table in which the windshield Gf is not interposed between the stereo camera 2 and the lattice chart board Bm, and a windshield Gf for each vehicle type when the vehicle is actually mounted. And a geometric table set in the same positional relationship (distance, orientation).

  Therefore, the geometric table created based on the images captured by the reference camera 2a and the comparison camera 2b adds the amount of the windshield non-mounting to the number of windshields Gf for each vehicle type in each of the reference image and the comparison image. It becomes a number. In the following, for convenience, the reference image Ri and the comparison image Li will be collectively referred to as a grid image.

  By the way, when the stereo camera 2 images the lattice chart board Bm without passing through the windshield Gf, as shown in FIG. 6A, the lattice image (reference image Ri, comparative image Li) includes distortion aberration. Curves under the influence of lens distortion. Further, in the lattice image obtained by imaging the lattice chart board Bm through the windshield Gf, distortion including lens distortion and distortion of the windshield Gf (glass distortion) appears as shown in FIG. 6B.

  This lattice chart board Bm has already been picked up by the stereo camera 2 mounted on a vehicle on which the windshield Gf before factory shipment is not mounted, and a geometric table has already been created based on the image at that time. In the above-described geometric table creation routine, the geometric table is created with the windshield Gf not attached, and both geometric tables should exhibit the same characteristics. However, the installation environment is different between after factory shipment and before factory shipment. May have different characteristics. Therefore, in this embodiment, the geometric table in a state where the windshield Gf after factory shipment is not mounted is separately created.

  In the following, for ease of explanation, the windshield Gf is made of two windshields α and β, the geometric table created before factory shipment is A, and the windshield Gf after factory shipment is created with the non-mounted state. Let B be the geometric table created, C be the geometric table created based on the grid image captured through the windshield α, and D be the geometric table created based on the grid image captured through the windshield β.

  Accordingly, the geometric tables A and B have a characteristic of lens distortion of the stereo camera, the geometric table C has a characteristic of including the glass distortion of the windshield α in the lens distortion, and the geometric table D is The lens distortion includes the glass distortion of the windshield β.

  Further, the geometric distortion removal and reproduction apparatus shown in FIG. 3 performs a simulation for removing the geometric distortion of the windshields α and β or reproducing the geometric distortion by using the above-described geometric tables A to D. A reference traveling image reproduction unit 10a, a comparative traveling image reproduction unit 10b, an image correction unit 20, and an output image storage unit 30. The image output to the output image storage unit 30 is an image analysis unit as an image analysis unit. It is output to the unit 40.

  The reference traveling image reproduction unit 10a and the comparative traveling image reproduction unit 10b reproduce the simulation traveling image captured by the reference camera 2a and the comparison camera 2b. The traveling image is, for example, the stereo camera 2 through the windshield α. It was taken with. Therefore, this image is captured in a state including the lens distortion of the stereo camera 2 and the glass distortion of the windshield α. In addition, as a traveling image of the reference traveling image reproduction unit 10a and the comparative traveling image reproduction unit 10b, a traveling image captured by a stereo camera from which the windshield Gf is removed and lens distortion is removed may be employed. Since lens distortion has been removed from the beginning, lens distortion characteristics can be obtained for each model by creating a geometric table for a stereo camera for each model.

  The traveling images reproduced by the traveling image reproducing units 10 a and 10 b are transmitted to the image correcting unit 20. The image correction unit 20 receives a geometric table storage unit 21 that stores the geometric tables A to D, a reference traveling image, and a comparative traveling image, and the geometric tables A to D stored in the geometric table storage unit 21 There are provided calibration calculation units 22a and 22b that are selectively input, and filter correction units 23a and 23b to which images from the respective calibration calculation units 22a and 22b are input.

  The calibration calculation units 22a and 22b read the geometric table selected by the external operation from the geometric tables A to D stored in the geometric table storage unit 21, and the grid point coordinates stored in the read geometric table Based on the geometric distortion correction data for each, the corresponding coordinates of the reference traveling image and the comparative traveling image are geometrically calibrated to remove or reproduce the geometric distortion. The filter correction units 23a and 23b remove noise included in each geometrically calibrated travel image (gradation image) using a spatial filter.

  The output image storage unit 30 stores the reference travel image and the comparative travel image output from the image correction unit 20 and outputs them to the image analysis unit 40.

  The image analysis unit 40 reads the reference traveling image data and the comparative traveling image data stored in the output image storage unit 30 and analyzes both traveling image data, thereby giving the image of the glass distortion of the windshield Gf. Assess the impact.

  Next, geometric calibration is performed on the reference traveling image and the comparative traveling image, which are processed by the above-described calibration calculation units 22a and 22b, using a geometric table arbitrarily selected from the geometric tables A to D, thereby obtaining a geometric shape. An example of removing or reproducing the dynamic distortion will be described. In the following expression, a minus sign means geometric calibration, and this simply shows an aspect for removing distortion.

-Calibration using the geometric table A Since the reference running image and the comparative running image are taken by the stereo camera 2 through the windshield α, the glass distortion of the windshield α and the lens distortion of the stereo camera 2 before factory shipment are Included as geometric distortion. When the glass distortion of the windshield α is εα and the lens distortion of both cameras 2a and 2b is εA, when this is calibrated with the geometric table A, the geometric table A is obtained before factory shipment.
(Lens strain εA + glass strain εα) + (− lens strain εA) = glass strain εα
Thus, the reference traveling image having the glass strain εα and the comparative traveling image are output to the output image storage unit 30.

Calibration using the geometric table B A reference traveling image and a comparative traveling image including the glass distortion εα of the windshield α and the lens distortion εA of the stereo camera 2 before factory shipment are used as the lens of the stereo camera 2 after factory shipment. When calibration is performed with the geometric table B having distortion, when the lens distortion of the geometric table B is εB,
(Lens strain εA + glass strain εα) + (− lens strain εB)
= (Lens strain εA-lens strain εB) + glass strain εα
Thus, the difference between the lens distortion εA and the lens distortion εB, and the reference traveling image and the comparative traveling image including the glass distortion εα are output to the output image storage unit 30.

Calibration using the geometric table C A reference running image and a comparative running image including the glass distortion εα of the windshield α and the lens distortion εA of the stereo camera 2 before factory shipment are used as the lens of the stereo camera 2 after factory shipment. When calibrating with a geometric table C having a strain εB and a glass strain εα of the windshield α (lens strain εB + glass strain εα),
(Lens strain εA + glass strain εα) − (lens strain εB + glass strain εα)
= (Lens distortion εA−Lens distortion εB)
Thus, the reference traveling image including the difference between the lens distortion εA and the lens distortion εB and the comparative traveling image are output to the output image storage unit 30.
At this time, when the lens distortions εA and εB have substantially the same characteristics, each traveling image captured by the stereo camera 2 through the windshield α after shipment from the factory has a lens distortion εA (= εB) shown in FIG. ) Includes the glass distortion εα of the windshield α, and an image as shown in FIG. By calibrating this with the geometric table C, it is possible to generate an image in which both the lens distortion εA (= εB) and the glass distortion εα are removed, as shown in FIG.

  As a result, when the characteristics of the lens distortion εA and the lens distortion εB are substantially equal, the horizontal deviation and the vertical deviation between the two cameras 2a and 2b are obtained by calibrating the reference traveling image and the comparative traveling image with the geometric table C. Can be removed.

Calibration using the geometric table D A reference traveling image and a comparative traveling image including the glass distortion εα of the windshield α and the lens distortion εA of the stereo camera 2 before factory shipment are used as the lens of the stereo camera 2 after factory shipment. Calibration is performed with a geometric table D having a strain εB and a glass strain of the windshield β. Note that the glass strain of the windshield β is εβ.

(Lens strain εA + glass strain εα) + (− lens strain εB−glass strain εβ)
= (Lens strain εA−Lens strain εB) + (Glass strain εα−Glass strain εβ)
Thus, the difference between the lens strain εA and the lens strain εB, and the reference travel image and the comparative travel image including the difference between the glass strain εα and the glass strain εβ are output to the output image storage unit 30.

  As described above, according to the present embodiment, the reference travel image and the comparative travel image including the glass distortion εα and the lens distortion εA, which are captured by the stereo camera 2 through the windshield α, are stored in the geometric table storage unit 21. By selectively performing geometric calibration with each of the geometric tables A to D stored in the table, it is possible to output a reference traveling image and a comparative traveling image having different distortion characteristics.

  Accordingly, for example, the lens distortion εA and the glass distortion εα included in the reference traveling image and the comparative traveling image are selectively read in the geometric tables A to D stored in the geometric table storage unit 21 to perform geometric calibration. Can be removed.

-Removal of lens strain εA and glass strain εα As described above, when the lens strains εA and εB have substantially the same characteristics, the lens strain εA and the glass strain εα can be removed by using the geometric table C. it can. However, when the lens distortion εB shows a characteristic different from the lens distortion εA, it is difficult to remove the lens distortion εA with the geometric table C alone, which hinders improvement in distance measurement accuracy.

Therefore, it is necessary to remove the lens distortion εA in order to improve the distance measurement accuracy. In this case, first, the geometric table A and the geometric table B are read, and a new geometric table E is created from this difference, that is, the difference between the lens distortion εA before factory shipment and the lens distortion εB after factory shipment.
Geometric table E = Geometric table A-Geometric table B
It is stored in the geometric table storage unit 21.

Next, this geometric table E is added to the geometric table C to create a new geometric table F.
Geometric table F = Geometric table C + Geometric table E
It is stored in the geometric table storage unit 21.

The geometric table C has a characteristic in which the glass strain εα is added to the lens distortion εB, and the geometric table E has a difference characteristic between the lens distortion εA and the lens distortion εB.
Geometric table F = (lens strain εB + glass strain εα) + (lens strain εA−lens strain εB)
= Lens distortion εA + glass distortion εα
Thus, the glass distortion εα and the lens distortion εA before factory shipment included in the reference traveling image and the comparative traveling image captured by the stereo camera 2 through the windshield α are obtained by geometric calibration using the geometric table F. The reference running image and the comparative running image from which these can be easily removed are output to the output image storage unit 30. As a result, the parallax error generated by the geometric distortion of the windshield α between the reference traveling image and the comparative traveling image can be reduced, and the distance measurement accuracy can be improved.

  Furthermore, by creating the geometric table F, an image from which both the lens distortion εA and the glass distortion εα can be removed can be generated, so that the performance of the reference running image, the comparative running image, and the stereo matching can be improved. Can do. Further, since distortion in the horizontal direction is also removed, variation in parallax offset calibration is reduced, and distance measurement accuracy can be improved.

  Next, the reference traveling image and the comparative traveling image captured by the stereo camera 2 through the windshield α are output from the reference traveling image reproducing unit 10a and the comparative traveling image reproducing unit 10b to the image correcting unit 20, and the image correcting unit 20 outputs the front traveling image. A mode in which a reference traveling image and a comparative traveling image captured through glass β are output in a simulated manner will be described.

  In this case, if a geometric table of (lens strain εA + glass strain εα−glass strain εβ) is created, the glass strain εα can be removed and the characteristics of the glass strain εβ can be output from the image correction unit 20.

First, a geometry table G is created from the difference between the geometry tables B and D stored in the geometry table storage unit 21;
Geometric table G = Geometric table B-Geometric table D
= Lens strain εB- (lens strain εB + glass strain εβ)
= -Glass strain εβ
Then, the geometry table F stored in the geometry table storage unit 21 is read, the geometry table G is added, and the geometry table H is created.
Geometric table H = Geometric table F + Geometric table G
= Lens strain εA + glass strain εα−glass strain εβ
It is stored in the geometric table storage unit 21. With this geometric table H, if the running images output from the reference running image reproduction unit 10a and the comparative running image reproduction unit 10b are geometrically calibrated,
(Lens strain εA + glass strain εα) − (lens strain εA + glass strain εα−glass strain εβ)
= Glass strain εβ
Thus, it is possible to simulate the travel image captured through the windshield β with the travel image captured by the stereo camera 2 through the windshield α, and analyze the image through the windshield β by this simulation. .

  That is, an image obtained by imaging the lattice chart board Bm through the windshield α is obtained by adding the glass distortion εα of the windshield α to the difference between the lens distortion εA before factory shipment and the lens distortion εB after factory shipment shown in FIG. The image is as shown in FIG. This is geometrically calibrated with the geometric table F as shown in FIG. 5C, thereby removing the vertical shift between the two cameras 2a and 2b and the distortion in the horizontal direction. ), The geometric table G is added, and the geometrical calibration is performed with the geometric table H as shown in FIG. 5E, whereby the glass distortion εβ of the windshield β can be reproduced.

  Therefore, as in the conventional example shown in FIG. 8, the lens distortion εA obtained by imaging the lattice chart board Bm with the stereo camera 2 before shipment from the factory is geometrically converted by affine transformation as shown in FIG. After the correction, the glass distortion equivalent to the glass distortion εβ generated when the image is taken through the windshield β can be reproduced without replacing the windshield α with the windshield β. It becomes possible.

  As a result, by programming the data of the geometric table H in the stereo camera 2, it is possible to obtain a running image as if the stereo camera 2 was captured through the windshield β in the actual vehicle running with the windshield α.

  Thus, according to the present embodiment, the stereo chart 2 after factory shipment is used to image the lattice chart board Bm in a state where there is no windshield Gf and in which a windshield Gf for each vehicle type is mounted, By generating in advance a geometric table having the characteristics of strain εB and glass strains εα, εβ, etc., lens distortion and glass distortion can be removed without wearing a windshield for each vehicle type and driving a real vehicle. Alternatively, it is possible to acquire travel image data that reproduces glass distortion. As a result, it is possible to easily evaluate the influence on the captured image caused by the geometric distortion of the plural types of windshields Gf without enormous man-hours and costs. Furthermore, it is possible to reduce parallax errors caused by the geometric distortion of the windshield Gf.

  Note that the present invention is not limited to the above-described embodiment. For example, a traveling image captured by the stereo camera 2 through the front glass α can be used to simulate a traveling image captured through the laminated glass. is there. Since the laminated glass has a structure integrated by adhering a pair of glass plates with an intermediate film interposed therebetween, each glass plate and each of the intermediate films have specific distortions.

  Therefore, first, the geometric table J is created based on the lattice image captured by the stereo camera 2 through the laminated glass. Then, by applying this geometric table J in place of the above-described geometric table D, it is possible to obtain simulated driving image data captured by the stereo camera 2 through the laminated glass.

  Further, by acquiring the lens distortion of the stereo camera 2 for each model, a combination of the stereo camera 2 of a different model and the windshield Gf for each vehicle type can be obtained without changing the stereo camera 2 and the windshield Gf one by one. With the stereo camera 2 and the windshield Gf, it is possible to simulate and evaluate geometric distortion correction (removal) and reproduction for all combinations.

  Further, the stereo camera 2 is not limited to the one mounted on the vehicle, but may be an aircraft, a pneumatic vehicle, or the like. In this case, the protective hood is not limited to the windshield Gf but may be a windshield.

1 ... Geometric table creation device,
2 ... Stereo camera,
2a ... Reference camera,
2b ... Comparison camera
2c ... Stay,
4 ... Geometric table creation device,
5a, 5b ... pre-processing unit,
6 ... Input image memory,
7: Stereo image characteristic detection unit,
8: Geometric table creation part,
9: Data storage unit,
10a: reference running image reproduction unit,
10b ... comparative running image reproduction unit,
15 ... Input image memory,
20: Image correction unit,
21 ... Geometric table storage unit,
22a, 22b ... calibration operation unit,
23a, 23b ... filter correction unit,
30: Output image storage unit,
40. Image analysis unit,
A to H ... geometric table,
Gf, α, β ... windshield,
Li ... Comparison image,
R11-Rmm, L11-Lmm ... coordinates,
Rc, Lc ... Center marker,
Ri: Reference image,
εA, εB ... lens distortion,
εα, εβ ... Glass strain

Claims (4)

A stereo camera having two cameras arranged with a predetermined baseline length apart;
A grid chart that is arranged in front of the stereo camera and has a center marker disposed at a grid point corresponding to each camera;
A protective hood interposed between the stereo camera and the grid chart;
A characteristic detection unit that images the lattice chart with each camera and obtains coordinates of each lattice point of the lattice chart;
A geometric table that stores geometric distortion correction data for each lattice point from the difference between the coordinates of each lattice point determined by the characteristic detection unit and the ideal coordinates corresponding thereto, the state where the protective hood is not interposed, and A geometric table creation unit that creates a protective hood for each different type,
A data storage unit for storing the geometry table created by the geometry table creation unit;
A plurality of geometric tables stored in the data storage unit are combined, and geometric distortion between the stereo camera and each protective hood is removed by the geometric distortion correction data stored in each geometric table. Alternatively, a geometric distortion removing and reproducing apparatus comprising a calibration calculation unit that reproduces geometric distortion of different protective hoods. The calibration calculation unit is configured to calculate a geometric distortion included in a traveling image captured by interposing the specific protective hood in front of the stereo camera, and to calculate the geometric table for the stereo camera and the specific protective hood. 2. The geometric distortion removal and reproduction apparatus according to claim 1, wherein the geometric distortion removal data stored in the geometric table is removed. The calibration calculation unit is configured to calculate a geometric distortion included in a traveling image captured by interposing the specific protective hood in front of the stereo camera, and to calculate the geometric table for the stereo camera and the specific protective hood. The geometric distortion correction data stored in the geometric table is removed, and the corresponding geometric distortion of the stereo camera or the protective hood is reproduced with the geometric distortion correction data stored in the different geometric table. The geometric distortion removing and reproducing apparatus according to claim 1. The geometric distortion according to claim 3, further comprising an image analysis unit that evaluates an influence of the geometric distortion of the stereo camera or the protective hood corresponding to the different geometric table in the reproduced traveling image. Removal reproduction device.

Images