JP2011228857A - Calibration device for on-vehicle camera - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a calibration device for on-vehicle cameras, which solves the problem that a reference marker, which is taken for performing camera calibration, is not imaged in high contract when the light environment of the background is in a bad condition, resulting in inaccurate camera calibration.SOLUTION: Region specifying means 30 specifies a region including a reference marker 10 which is in an image taken by imaging means 20. Contrast calculation means 40 calculates a value corresponding to the contrast in the specified region. If the calculated value corresponding to the contrast is smaller than a predetermined value, exposure changing means 60 changes the exposure of the imaging means 20 and the imaging means 20 takes images in which the value corresponding to the contrast in the specified region is larger than the predetermined value. Then, calibration performing means 70 performs camera calibration with the taken image.

Description

  The present invention relates to a camera internal parameter such as a focal length by associating a three-dimensional position of a point in space with a two-dimensional position on the image when the point is imaged by the camera, More particularly, the present invention relates to an in-vehicle camera calibration device that can perform camera calibration operation in a short time. .

  2. Description of the Related Art In recent years, a system in which a camera is installed in a vehicle and an image around the vehicle that is likely to be a blind spot from a driver is captured and displayed.

  In particular, recently, a plurality of cameras are installed in a vehicle, coordinate conversion is performed as if looking down from directly above, and an overhead image is generated, and the generated overhead images are synthesized to look around 360 ° around the vehicle. A system for displaying a captured image has also been put into practical use.

  In order to realize such a system, it is necessary to combine and synthesize images captured by a plurality of cameras installed in a vehicle into one image. At that time, since the internal parameters such as the focal length of the camera and the values of external parameters related to the installation position and direction of the camera are generally unknown, the camera calibration operation is performed at the time of camera installation. A conversion table for coordinate conversion is created by clarifying the value of, and coordinate conversion and image synthesis are performed based on the generated coordinate conversion table. For this reason, various camera calibration methods have been proposed (Patent Document 1, Patent Document 2, and Patent Document 3).

JP 2003-78925 A JP 2003-244521 A JP 2005-250628 A

  Various camera calibration methods have been disclosed, including Patent Documents 1 to 3, but all of these inventions are based on an image of a reference marker that has been imaged through a camera. The camera is calibrated by measuring a plurality of coordinates of characteristic points (feature points) such as vertices in the center and associating them with the coordinate values of the corresponding three-dimensional space. Furthermore, these inventions are based on the premise that an image showing a reference marker used for calibration is photographed with good contrast so that the coordinates of the feature points in the image can be easily measured.

  In general, the in-vehicle camera is mounted on a factory assembly line. The factory assembly line can be equipped with a dedicated environment for camera calibration, so images including reference markers can be taken with high contrast under good lighting conditions. Can be measured reliably, and calibration can be executed reliably.

  On the other hand, when a vehicle equipped with a camera is damaged due to an accident or the like and the camera is damaged, the camera is generally replaced at a repair shop. In such a repair shop, there is generally no dedicated environment for camera calibration, and a reference marker is set on the floor or the ground, and the reference marker is photographed by the camera for calibration.

  At this time, there may be a case where the reference marker cannot be imaged with high contrast due to the influence of illumination light or sunlight. At that time, trial and error are repeated until the reference marker is imaged with high contrast by moving the position where the reference marker is pasted or moving the installation location of the vehicle.

  Alternatively, the calibration of the camera is performed using an image in which the reference marker is unclear, so that the coordinate value of the feature point is not accurately measured, thereby combining a plurality of images into one image. In this case, there is a possibility that the image may be displaced, or the captured image may not be sufficiently corrected for distortion.

  More recently, in-vehicle cameras that can be mounted even after shipment from the factory have been developed, and the opportunity to perform camera calibration is increasing outside the assembly line of the factory.

  Therefore, it is desired to develop a technique that can easily and reliably carry out camera calibration even outside the assembly line of a factory.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an in-vehicle camera calibration device that can reliably perform camera calibration even when a reference marker is not clearly imaged.

  The in-vehicle camera calibration device according to the present invention images a region including a reference marker arranged so that a three-dimensional position can be specified so that a value corresponding to the contrast of the region is larger than a predetermined value. The camera is calibrated using the image imaged in this way.

  That is, a first on-vehicle camera calibration device according to the present invention includes a reference marker having a specific light and dark pattern arranged so that a three-dimensional position can be specified, and an imaging unit that captures an image including the reference marker. A region specifying unit for specifying a region in which the reference marker is captured from an image photographed by the photographing unit; and a region identified by the region identifying unit from an image photographed by the photographing unit Contrast calculating means for calculating a value corresponding to the contrast in the image, contrast determining means for determining whether the value corresponding to the contrast calculated by the contrast calculating means is greater than a predetermined value, and the contrast First exposure changing means for changing the exposure of the photographing means so that the corresponding value is larger than a predetermined value. The first exposure changing means changes the exposure of the photographing means, and then the first photographing instruction means causes the photographing means to take an image, and the photographing means is used to calibrate the photographing means. Calibration execution means for performing the above.

  According to the first on-vehicle camera calibration device according to the present invention configured as described above, the region specifying unit specifies the region including the reference marker for the image shot by the shooting unit, and the contrast calculating unit. Calculates a value corresponding to the contrast in the specified region, and the contrast determination means determines whether the calculated value is larger than a predetermined value, and if the calculated value is larger than the predetermined value, When it is small, after the exposure of the photographing means is changed by the first exposure control means, the photographing means takes a picture again according to the instruction from the first photographing instruction means, and the contrast calculating means for the photographed image, When a value corresponding to the contrast in the specified area is calculated, and the contrast determination unit determines that the calculated value is greater than a predetermined value, the captured image And calibrating the camera with the calibration execution means, it is possible to accurately measure the coordinates of the feature points in the reference marker in the image, so that the camera calibration can be ensured regardless of the lighting environment. Can be done.

  Further, a second on-vehicle camera calibration device according to the present invention includes a reference marker having a specific light and dark pattern arranged so that a three-dimensional position can be specified, and an imaging unit that captures an image including the reference marker. A region specifying unit that specifies a region in which the reference marker is captured from an image shot by the shooting unit; a second exposure changing unit that changes the exposure of the shooting unit over a plurality of stages a predetermined number of times; The contrast calculating means for calculating a value corresponding to the contrast in the area specified by the area specifying means from the image shot by the shooting means, and the exposure of the shooting means by the second exposure changing means. Each time the camera is changed a predetermined number of times, the second photographing instruction means for causing the photographing means to take an image, and the photographing by the instruction of the photographing instruction means An image selecting means for selecting an image having a value corresponding to the contrast larger than a predetermined value and having a maximum value corresponding to the contrast from the plurality of images selected by the image selecting means; And a calibration execution means for calibrating the photographing means using an image.

  According to the second on-vehicle camera calibration device according to the present invention configured as described above, every time the exposure of the photographing unit is changed a predetermined number of times by the second exposure control unit, the instruction from the second photographing instruction unit Accordingly, a value corresponding to the contrast of the area including the reference marker specified by the area specifying means is calculated by the contrast calculating means for the plurality of images taken by the imaging means, and the calculated contrast is calculated by the image selecting means. An image having a value that is greater than a predetermined value and having a maximum value corresponding to the contrast is selected, and the camera is calibrated by the calibration execution unit using the selected image. The coordinates of the feature points in the captured reference marker can be measured accurately, so the camera can be used regardless of the lighting environment. It is possible to perform the calibration reliably.

  In addition, a third on-vehicle camera calibration device according to the present invention includes a reference marker having a specific light and dark pattern arranged so that a three-dimensional position can be specified, and an imaging unit that captures an image including the reference marker. A region specifying unit for specifying a region in which the reference marker is captured from an image photographed by the photographing unit; and a region identified by the region identifying unit from an image photographed by the photographing unit The contrast calculation means for calculating the value corresponding to the contrast in the image, the contrast determination means for determining that the value corresponding to the contrast calculated by the contrast calculation means is greater than a predetermined value, and the contrast determination means , When it is determined that a value corresponding to the contrast is smaller than the predetermined value, And a contrast correction unit that corrects a value corresponding to the contrast to be larger than the predetermined value, and performs calibration of the photographing unit using an image corrected by the contrast correction unit Means.

  According to the third on-vehicle camera calibration device according to the present invention configured as described above, an area including the reference marker specified by the area specifying means by the contrast calculating means from the image taken by the photographing means. A value corresponding to the contrast is calculated, and when the contrast determining unit determines that the value corresponding to the calculated contrast is smaller than a predetermined value, the value corresponding to the contrast is larger than the predetermined value by the contrast correcting unit. By using the image corrected in this way and calibrating the camera by the calibration execution means, the coordinates of the feature points in the reference marker appearing in the image can be accurately measured. Ensure that the camera is calibrated regardless of the lighting environment. It can be.

  According to the on-vehicle camera calibration device according to the present invention, the reference marker imaged by the imaging unit is imaged so that the value corresponding to the contrast of the region including the reference marker is larger than a predetermined value. Since the camera is calibrated using the image thus made, the coordinates of the feature points in the reference marker appearing in the image can be accurately measured, so regardless of the light environment for calibration, An effect is obtained that the camera can be reliably calibrated.

1 is a block diagram illustrating a schematic configuration of a vehicle surrounding monitoring device to which a vehicle-mounted camera calibration device according to a first embodiment of the present invention is applied. It is a top view which shows the state by which the reference marker and imaging | photography means which concern on 1st Embodiment of this invention were installed in the vehicle. It is a figure explaining the installation position of the reference marker which concerns on 1st Embodiment of this invention. It is a figure which shows the example of the image observed when performing calibration operation in 1st Embodiment of this invention. It is a flowchart (1) of the calibration operation of the camera which concerns on 1st Embodiment of this invention. It is a flowchart (2) of the calibration operation of the camera which concerns on 1st Embodiment of this invention. It is a figure which shows the cursor utilized when performing calibration operation in 1st Embodiment of this invention. 3 is a flowchart of image conversion and image composition processing according to the first embodiment of the present invention. It is a figure which shows the example of the display image which concerns on 1st Embodiment of this invention. It is a block diagram which shows schematic structure of the vehicle periphery monitoring apparatus 6 to which the vehicle-mounted camera calibration apparatus which concerns on 2nd Embodiment of this invention is applied. It is a flowchart of the calibration operation of the camera which concerns on 2nd Embodiment of this invention. It is a block diagram which shows schematic structure of the vehicle periphery monitoring apparatus to which the vehicle-mounted camera calibration apparatus which concerns on 3rd Embodiment of this invention is applied. It is a flowchart of the calibration operation of the camera which concerns on 3rd Embodiment of this invention. It is a figure which shows the contrast correction process of the image which concerns on 3rd Embodiment of this invention.

  Hereinafter, an embodiment of a calibration device for an in-vehicle camera according to the present invention will be described with reference to the drawings.

  In this embodiment, the first vehicle-mounted camera calibration device (hereinafter referred to as vehicle-mounted camera calibration device 2) according to the present invention is easy for the driver to view images around the vehicle captured by a plurality of cameras installed in the vehicle. The present invention is applied to the vehicle surrounding monitoring device 5 that displays in the form.

  FIG. 1 is a block diagram showing a schematic configuration of a vehicle surroundings monitoring device 5 according to an embodiment of the present invention. The vehicle surrounding monitoring apparatus 5 according to the present invention is provided on a reference marker 10 having a black-and-white light / dark pattern installed on a road surface and a vehicle 150 (see FIG. 2), and photographs images around the vehicle 150 in different directions. A plurality of cameras, a region specifying unit 30 for identifying the position of the region including the reference marker 10 from the images captured by the photographing unit 20, and a reference marker specified by the region specifying unit 30 Contrast calculation means 40 for calculating a value C corresponding to the contrast of the region including 10, contrast determination means 50 for determining whether or not the value C calculated by the contrast calculation means 40 is larger than a predetermined value, and the value C First exposure changing means 60 for changing the exposure of the camera constituting the photographing means 20 so that the value C is larger than the predetermined value when the value is smaller than the predetermined value; 1 After the exposure is changed by the exposure changing means 60, the first photographing instruction means 35 for issuing a photographing instruction to the photographing means 20, the calibration executing means 70 for executing camera calibration, and the result of the calibration. Based on the coordinate conversion table creating means 80 for creating the coordinate conversion table, the coordinate conversion means 90 for converting the image photographed by the photographing means 20 into a bird's eye image, and the coordinate-converted image in the vehicle 150 The image display means 100 to be displayed, the operation switch 110 for instructing the start and end of the vehicle surrounding monitoring device 5 and the contents of the camera calibration operation, and the shift position of the vehicle 150 are detected to be in the reverse position. And a shift position detecting means 120.

  Specifically, the photographing means 20 is connected to four cameras (first camera 22, second camera 24, third camera 26, fourth camera 28) and each camera (22, 24, 26, 28). 4 decoders (a first decoder 23 connected to the first camera 22 and a second decoder 25 connected to the second camera 24) that sample and quantize the captured images and convert them into digital images. , A third decoder 27 connected to the third camera 26, and a fourth decoder 29) connected to the fourth camera 28.

  As shown in FIG. 2, the first camera 22 is provided on the front bumper of the vehicle 150, the second camera 24 is provided on the left door mirror, the third camera 26 is provided on the rear bumper, and the fourth camera 28 is provided on the right door mirror. , Each is arranged.

  Further, as shown in FIG. 2, the adjacent camera has a shooting range of the second camera 24 and a third range so that the shooting range of the first camera 22 and the shooting range of the second camera 24 form a first overlapping area E1. Shooting by the fourth camera 28 so that the shooting range of the third camera 26 and the shooting range of the fourth camera 28 form a third overlapping region E3 so that the shooting range of the camera 26 forms a second overlapping region E2. The range and the shooting range of the first camera 22 are installed so as to form a fourth overlapping region E4. It is assumed that the three-dimensional coordinates of the installation positions of the cameras (22, 24, 26, 28) are known in advance.

  Here, in detail, as shown in FIG. 2, the reference marker 10 has four black and white checkered patterns, all of which have the same size and the same shape (first reference marker 12, second reference marker). The marker 14, the third reference marker 16, and the fourth reference marker 18).

  In detail, the calibration execution means 70 includes a corresponding point instructing unit 72 for instructing the position of the vertex of the reference marker 10 in the photographed image by the operator, and the vertex of the instructed reference marker 10. Based on the correspondence between the position coordinates and the three-dimensional coordinates of the apex of the reference marker 10, the camera parameter calculation unit 74 calculates internal parameters and external parameters of the camera.

  Further, the operation switch 110, in detail, when performing the calibration of the camera, the start switch 112 that instructs the start of the vehicle surroundings monitoring device 5, the end switch 114 that instructs the end of the vehicle surroundings monitoring device 5, An image monitor switch 115 for observing the captured image, a camera selection switch 116 for selecting a camera for capturing the image to be observed, and a corresponding point for instructing start of an operation for associating the vertex captured in the image with the reference marker An instruction start switch 117 and a cursor movement switch 118 for moving a cursor K used to instruct corresponding points on the image.

  Among these, the reference marker 10, the imaging unit 20, the region specifying unit 30, the first imaging instruction unit 35, the contrast calculating unit 40, the contrast determining unit 50, the first exposure changing unit 60, and the calibration execution. The means 70 and a part of the operation switch 110 (the image monitor switch 115, the camera selection switch 116, the corresponding point instruction start switch 117, and the cursor movement switch 118) constitute the in-vehicle camera calibration device 2 of the present invention.

  The vehicle surrounding monitoring device 5 calibrates each camera (22, 24, 26, 28) installed in the vehicle 150, and then uses the result to calibrate each camera (22, 24, 26, 28). ) Is converted into an image viewed from above the vehicle 150, and the images converted into the overhead image are combined into a single image and presented to the driver.

  Hereinafter, the operation of the present embodiment will be described by dividing into camera calibration and image conversion / synthesis.

  First, the flow of the calibration operation of the cameras (22, 24, 26, 28) in the vehicle surroundings monitoring apparatus 5 according to the present embodiment will be described based on the flowcharts of FIGS.

  First, the vehicle 150 is placed at a predetermined place for calibration (S2 in FIG. 5). The predetermined place is preferably a plane having no undulation or inclination. At this time, the place where the vehicle 150 is placed is, as shown in FIG. 3, the point on the road surface at the left front corner of the vehicle is the origin (0, 0, 0), and from there on the road surface on the left side of the vehicle A three-dimensional orthogonal coordinate system (Xw, Yw, Zw) is formed so as to form a right-handed system with the Xw axis in the direction toward the rear of the vehicle along the plane.

  The operator who performs the calibration sets the reference markers 10 having checkered patterns whose dimensions have been measured in advance (S1 in FIG. 5) at the four corners of the vehicle 150 (S3 in FIG. 5), and then the image monitor. The switch 115 is operated so that the image captured by the imaging unit 20 can be observed by the image display unit 100, and the camera selection switch 116 is operated to display an image from the first camera 22 to the fourth camera 28. A camera to be observed is selected (S4 in FIG. 5).

  First, the operator observes an image captured by the first camera 22 and confirms that both the first reference marker 12 and the second reference marker 14 are shown in the image (FIG. 5). S5). At this time, when one of the reference markers (12 or 14), or both of the reference markers (12 and 14) are not shown, or when a part of the reference marker (12 or 14) is interrupted, The installation position of the reference marker (12 or 14) is finely adjusted (S6 in FIG. 5) so that the first reference marker 12 and the second reference marker 14 can be seen without interruption.

  Next, images taken by the second camera 24, the third camera 26, and the fourth camera 28 are observed in order, and two reference markers are shown in the image without interruption in the same manner as in the above procedure. Confirmed (S7 in FIG. 5). If the reference marker 10 is not shown in the image or is interrupted, fine adjustment of the installation position of the reference marker 10 is performed.

  After the fine adjustment of the installation position of the reference marker 10 is completed, the three-dimensional coordinates of the checkerboard apexes (9 places) of each reference marker 10 are measured by the following procedure.

  First, the mutual installation interval of each reference marker (12, 14, 16, 18) and the installation direction (angle) of each reference marker (12, 14, 16, 18) are measured.

  Next, the relative relationship between the vehicle 150 and the installation position of each reference marker (12, 14, 16, 18) is measured. Here, using the origin of the three-dimensional orthogonal coordinate system (Xw, Yw, Zw) as a reference, each installation of the first reference marker 12, the second reference marker 14, the third reference marker 16, and the fourth reference marker 18 is performed therefrom. The distance in the Xw direction and the distance in the Yw direction to the position are each measured. Based on the above measurement results, the three-dimensional coordinates of each vertex of each reference marker are calculated as shown in FIG. 3 (S8 in FIG. 5), and the calculated three-dimensional coordinates are stored in the camera parameter calculation unit 74. Is done.

  Next, the operator specifies the approximate position of the area in which the reference marker 10 is captured from the captured image by the action of the area specifying unit 30. This specification is performed by the operator specifying a rectangular area including the reference marker 10 while observing the captured image with the image display means 100.

Here, the image captured by the first camera 22 is converted from the composite signal to the component signal by the first decoder 23, and the luminance signal of the converted component signal is sampled (for example, horizontal 512 pixels × 480 pixels) and quantization (for example, 8 bits), and converted into a digital image I ₁ (x, y) (hereinafter simply referred to as image I ₁ (x, y)). Similarly, for the images taken by the second camera 24, the third camera 26, and the fourth camera 28, the image I ₂ (x, y), the image I ₃ (x, y), and I ₄ (x, y). Shall be converted to Hereinafter, when the images I ₁ (x, y) to I ₄ (x, y) are collectively referred to as images I _i (x, y). Here, i = 1, 2, 3, and 4.

The area specifying unit 30 has a function of drawing a rectangular area by superimposing it on the image I _i (x, y), and the position and size of the rectangular area can be arbitrarily changed according to an instruction from the operator. It has become.

Due to the action of the area specifying means 30, for example, as shown in FIG. 4, two rectangular areas in which the first reference marker 12 and the second reference marker 14 are shown in the image I ₁ (x, y), respectively. It is specified (S9 in FIG. 5).

Here, in the image I ₁ (x, y) photographed by the first camera, the rectangular area including the reference marker 12 shown on the left side is the area number R ₁₁ , and the rectangle including the reference marker 14 shown on the right side is displayed. to represent the region in the area number R _12. Furthermore, the coordinates of the upper left vertex of the rectangular area R ₁₁ are represented by (x ₁₁ , y ₁₁ ), the coordinates of the lower right vertex are represented by (x ₁₂ , y ₁₂ ), and the coordinates of the upper left vertex of the rectangular area R _12. (X ₁₃ , y ₁₃ ) and the coordinates of the lower right vertex are represented by (x ₁₄ , y ₁₄ ).

Further, by generalizing this, a rectangular area including the reference marker 10 shown on the left side in the image I _i (x, y) taken by the i-th camera (i = 1, 2, 3, 4) is obtained. area number R _i1, will be represented by the area number R _i2 a rectangular area including the reference marker 10 that is reflected on the right. Furthermore, the coordinates of the upper left vertex of the rectangular area R _i1 are represented by (x _i1 , y _i1 ), the coordinates of the lower right vertex are represented by (x _i2 , y _i2 ), and the coordinates of the upper left vertex of the rectangular area R _{i2 Is} represented by (x _i3 , y _i3 ) and the coordinates of the lower right vertex are represented by (x _i4 , y _i4 ).

The area number R ₁₁ of the rectangular area, the coordinates of the upper left vertex (x ₁₁ , y ₁₁ ), and the coordinates of the lower right vertex specified in the image I ₁ (x, y) specified by the action of the area specifying means 30. (X ₁₂ , y ₁₂ ), the area number R ₁₂ of the rectangular area, the coordinates of the upper left vertex (x ₁₃ , y ₁₃ ), and the coordinates of the lower right vertex (x ₁₄ , y ₁₄ ) are respectively contrast calculation means. 40.

A similar procedure is performed for the image I _i (x, y) captured by the remaining cameras (24, 26, 28) (S10 in FIG. 5), and the area number R _{ij of the} identified rectangular area is determined. And the coordinates of the upper left vertex (x _i1 , y _i1 ) and the coordinates of the lower right vertex (x _i4 , y _i4 ) are respectively stored in the contrast calculation means 40.

Next, for each image I _i (x, y) taken by the four cameras (22, 24, 26, 28) by the contrast calculating means 40, each rectangle specified by the area specifying means 30 is used. A value C corresponding to the contrast of the image I _i (x, y) in the region R _ij (i = 1, 2, 3, 4. j = 1, 2) is calculated (S11 in FIG. 5). The digitized value C, and can be considered various methods, here, the sum SUM of the gray values in the rectangular region R _ij, the total number of pixels N in the rectangular region R _ij determined, (Equation 1 ).

In (Expression 1), the range in which the sum is taken is inside the rectangular region R _ij in which the reference marker 10 is shown, and the value C takes a larger value as the reference marker 10 is clearly shown. This was taken when the environment in which the vehicle 150 was placed was dim, and the image I _i (x, y) to be photographed would be dark overall, and when strong sunlight was coming from a specific direction In the image I _i (x, y), a portion exposed to sunlight becomes bright, and in either case, the contrast of the reference marker 10 in the photographed image I _i (x, y) decreases. It is to do.

  Next, in the contrast determining means 50, it is determined whether or not the value C calculated by the contrast calculating means 40 is larger than a predetermined value Cth preset by a prior experiment or the like (S12 in FIG. 5).

Hereinafter, taking the image I ₁ (x, y) as an example, the flow of determination processing in the contrast determination means 50 will be described.

When the magnitude relationship between the value C calculated for the two rectangular areas R ₁₁ and R _{12 in the} photographed image I ₁ (x, y) and the predetermined value Cth is compared, the two rectangular areas R ₁₁ are compared. And the value C calculated for R ₁₂ are both greater than the predetermined value Cth (case 1), the values C calculated for the two rectangular regions R ₁₁ and R ₁₂ are both the predetermined value Cth. The value C is larger than the predetermined value Cth in one rectangular area and the value C is smaller than the predetermined value Cth in the other rectangular area (Case 3). . Hereinafter, the flow of processing will be described for each case.

In case 1 (when S12 in FIG. 5 is Yes), the region number R _ij and the captured image I ₁ (x, y) are sent to the corresponding point instruction unit 72 and stored (S13 in FIG. 5). ).

  In case 2 (when S17 in FIG. 5 is No), the contrast determination unit 50 changes the exposure of the corresponding first camera 22 to the first photographing instruction unit 35 and the first exposure change unit 60. An instruction is issued (S14 in FIG. 5). Further, the value C is transmitted to the first exposure changing means 60.

  When the first exposure changing unit 60 receives an instruction from the contrast determining unit 50, the first exposure changing unit 60 acquires the current exposure setting value of the first camera 22. Next, the first exposure changing means 60 calculates an exposure change amount for increasing the contrast based on the acquired exposure setting value and the received value C. That is, when it is determined that the exposure is insufficient, the diaphragm is opened to increase the exposure amount, and when it is determined that the exposure is excessive, the diaphragm is closed and the exposure amount is decreased. Specifically, the exposure correction amount is determined based on the magnitude of the difference between the calculated value C and the predetermined value Cth (S15 in FIG. 5).

  The determined exposure correction amount is sent to the corresponding first camera 22. The first camera 22 that has received the exposure correction instruction corrects the specified exposure amount. At the same time, an instruction to start shooting is issued from the first shooting instruction means 35 to the first camera 22, and shooting is performed with the corrected exposure (S16 in FIG. 5).

After the value C is calculated again for the two rectangular regions R _ij by the contrast calculation unit 40 for the image I ₁ (x, y) photographed with the exposure corrected, the contrast determination unit by 50, is compared with the value C and the predetermined value Cth, 2 two rectangular region R _ij value C calculated from and when both greater than the predetermined value Cth, it is determined that good images were photographed, the image I ₁ (x, y) is sent to the corresponding point instruction unit 72 and stored (S13 in FIG. 5). On the other hand, when the value C is smaller than the predetermined value Cth, exposure correction and photographing, and calculation and determination of the value C are performed again according to the procedure described above.

In case 3 (when S17 in FIG. 5 is Yes), the area number R _{ij of the} rectangular area where the value C is larger than the predetermined value Cth and the captured image I ₁ (x, y) indicate corresponding points. The data is sent to the unit 72 and stored (S18 in FIG. 5). Then, exposure correction is performed only for the rectangular area where the value C is smaller than the predetermined value Cth, and photographing is performed again (S16 in FIG. 5). When the value C calculated again after exposure correction is larger than the predetermined value Cth, the region number R _ij of the rectangular region that satisfies the condition and the image I ₁ (x, y) photographed at that time Is sent to the corresponding point instruction unit 72 and stored therein. On the other hand, when the value C is smaller than the predetermined value Cth despite the exposure correction, the exposure correction and photographing, and the calculation and determination of the value C are performed again according to the procedure described above.

Further, in order to capture an image I ₁ (x, y) in which the value C is larger than the predetermined value Cth, a positive exposure correction is necessary for the rectangular region R _i1 , but for the rectangular region R _i2 . A negative exposure correction may be required (or vice versa). In this case, first, the exposure is positively corrected, and an image in which the value C calculated from the region R _i1 is larger than the predetermined value Cth is taken. Next, the exposure is negatively corrected, and an image in which the value C calculated from the region R _i2 is larger than the predetermined value Cth is taken. Of course, the order of plus correction and minus correction may be reversed.

When the above procedure is repeated and the values C calculated from the two rectangular areas including the reference marker 10 in the photographed image I ₁ (x, y) are both greater than the predetermined value Cth, the next camera Repeat the same procedure for. In this way, the four cameras (22, 24, 26, 28) capture an image in which the reference marker 10 is captured with high contrast (S19 in FIG. 5).

  Next, the calibration of the cameras (22, 24, 26, 28) is performed by the calibration execution means 70. Various methods of camera calibration have been proposed, and any of these methods may be applied. A typical method will be described below. Here, a procedure for calibrating the first camera 22 will be described.

  For the sake of explanation, the three-dimensional orthogonal coordinate system is represented by (Xw, Yw, Zw), and the screen coordinate system on the image of the first camera 22 is represented by (x, y). At this time, if a point (Xw, Yw, Zw) in the three-dimensional orthogonal coordinate system is projected onto a point (x, y) in the screen coordinate system, a relational expression of (Expression 2) holds between the two. .

When (Expression 2) is transformed, (Expression 3) and (Expression 4) are obtained.
_{(P 11 X w + P 12} Y w + P 13 Z w + P 14) / (P 31 X w + P 32 Y w + P 33 Z w + P 34) = x ( Equation 3)
_{(P 21 X w + P 22} Y w + P 23 Z w + P 24) / (P 31 X w + P 32 Y w + P 33 Z w + P 34) = y ( Equation 4)
Further, by transforming (Equation 3) and (Equation 4), (Equation 5) is obtained.

Here, the three-dimensional coordinates (X _wi , Y _wi , Z _wi ) of the vertices constituting the checkered pattern of the first reference marker 12 and the second reference marker 14 and the vertices are represented by the image I ₁ (x, y ) The coordinates (x _i , y _i ) of the point projected on (hereinafter referred to as the corresponding point) are obtained. By substituting the result into (Equation 5), 12 simultaneous equations are derived. By solving these simultaneous equations for P ₁₁ to P ₃₄ , it is possible to obtain information on the internal parameters and the external parameters of the first camera 22.

  At this time, in order to improve the accuracy of the calibration, all the corresponding points to be obtained are not obtained from one reference marker (12 or 14), but, for example, the first reference marker 12 and the second reference marker 14 to 3 It is desirable to select from both reference markers, such as selecting point by point.

In the present embodiment, since the first reference marker 12 and the second reference marker 14 are shown in the image I ₁ (x, y), a maximum of 18 corresponding points can be searched. it can. This is more than the six sets required for calibration. Therefore, as described above, (Equation 5) may be solved from only 6 pairs of corresponding points, or (Equation 5) may be solved by searching for all 18 pairs of corresponding points to improve the accuracy of the solution. Good.

As a method for identifying the corresponding points, the operator may visually identify the coordinates of the vertices of the first reference marker 12 and the second reference marker 14 shown in the image I ₁ (x, y). The positions of the vertices of the first reference marker 12 and the second reference marker 14 shown in the image I ₁ (x, y) may be automatically detected.

Hereinafter, an example in which the operator visually identifies the nine vertex positions of the first reference marker 12 and the nine vertex positions of the second reference marker 14 will be described. When the operator operates the corresponding point instruction start switch 117 (S1 in FIG. 6), as shown in FIG. 7, the first reference marker 12 and the second reference marker 14 stored in the corresponding point instruction unit 72 are clear. The cursor K is superimposed on the image I ₁ (x, y) shown in FIG. 6 and displayed on the image display means 100 (S2 in FIG. 6). The cursor K is configured such that its display position can be moved in four directions indicated by arrows in FIG.

The operator operates the cursor moving switch 118 while moving the cursor K while confirming the image on the image display means 100 (S3 in FIG. 6), and the first reference shown in the image I ₁ (x, y). After confirming that the position of the vertex of the marker 12 and the position of the intersection of the cursor K are matched (S4 in FIG. 6), camera parameters are calculated using the matched position coordinates together with the vertex number of the first reference marker 12. This is stored in the unit 74 (S5 in FIG. 6). Here, vertex numbers 1 to 9 are assigned to the vertex positions of the respective reference markers 10. That is, from left to right, top to bottom, the top left vertex is numbered 1, the top right vertex is 3, the bottom left vertex is 7, the bottom right vertex is 9, and so on. And

  The operation of specifying the coordinates of the vertexes of the reference marker 12 is performed on all nine vertices of the first reference marker 12 (S6 in FIG. 6), and the same applies to the second reference marker 14. The coordinates of the nine vertices are identified, and the coordinates of the identified vertices are stored in the camera parameter calculation unit 74 (S7 in FIG. 6).

  When the coordinates of the 18 vertices are specified, the camera parameter calculation unit 74 stores the coordinate values of the stored vertices and the three-dimensional of the first reference marker 12 and the second reference marker 14 corresponding to the respective coordinate values. The coordinate values are substituted into (Expression 5), and 18 simultaneous equations are derived for one reference marker (S8 in FIG. 6).

The simultaneous equations _and solving for _{P 34} from _{P 11} (S9 in FIG. 6), the calibration of the first camera 22 is completed. In this embodiment, since 36 simultaneous equations are derived for 12 unknowns, any 12 of them may be selected and solved, or the most error that satisfies 18 simultaneous equations. it may be calculated _{P 34} from a small _{P 11.} The calculated values of P ₁₁ to P ₃₄ are stored in the coordinate conversion table creating unit 80.

The above procedure is similarly applied to the image I _i (x, y) photographed by the other camera (24, 26, 28) (S10 in FIG. 6), and the camera (22, 24, 26, 28) Complete the calibration.

  Note that the image monitor switch 115, the camera selection switch 116, the corresponding point instruction start switch 117, and the cursor movement switch 118 used in the series of calibration operations are switches used only for the calibration operation. It is desirable to remove it when the operation is completed and to be able to reconnect when it becomes necessary to perform the calibration operation again.

  Next, the flow of image conversion and image composition in the vehicle surroundings monitoring device 5 according to the present embodiment will be described based on the flowchart of FIG.

In the coordinate conversion table creating means 80, each image I _i (x, y) photographed by the four cameras (22, 24, 26, 28) is converted into a bird's-eye view seen from directly above the vehicle 150 and converted. A coordinate conversion table for synthesizing the four overhead images thus obtained into one image is created (S1 in FIG. 8).

Specifically, as a coordinate conversion table for the first camera 22, a first x table T _x1 (x, y) representing an x coordinate before conversion / combination and a first y table T representing a y coordinate before conversion / combination. Two tables of _y1 (x, y) are created.

The creation of these coordinate conversion table, obtained in the calibration operation of the camera, the 12 parameter values P ₃₄ from P ₁₁ is used in Equation (5), pre-converted, or the image after the synthesis It is created by calculating the coordinate value of the image before conversion or before composition, which is stored in arbitrary coordinates. The created coordinate conversion table is stored in the coordinate conversion table creating means 80 (S2 in FIG. 8).

A coordinate conversion table is similarly created for cameras other than the first camera 22. That is, for the second camera 24, two sheets, a second x table T _x2 (x, y) representing the converted x coordinate and a second y table T _y2 (x, y) representing the converted y coordinate. For the third camera 26, the third x table T _x3 (x, y) representing the converted x coordinate and the third y table T _y3 (x, y) representing the y coordinate after conversion. Two sheets are created, and for the fourth camera 28, a fourth x table T _X4 (x, y) representing the converted x coordinate and a fourth y table T _y4 (x, y) representing the converted y coordinate. ) Are created.

Next, the action of the created coordinate conversion table will be specifically described. The photographed image I ₁ (x, y) is converted into an overhead image, and each of the images I _i (x, y) photographed by four cameras (22, 24, 26, 28) is overhead image. And a combined image generated by combining the four overhead images into one image is L (x, y).

At this time, assuming that the value of T _x1 (x, y) = α is stored in the first x table and the value of T _y1 (x, y) = β is stored in the first y table, the pixel of the composite image L (x, y) ( By storing the gray value I ₁ (α, β) in x, y), image conversion and composition are executed.

  Whether the shift position detecting means 120 detects that the shift position of the vehicle 150 is in the reverse position (S3 in FIG. 8) or that the start switch 112 has been operated (S4 in FIG. 8). Then, the display contents displayed on the image display means 100 at that time are stored (S5 in FIG. 8), and the first decoder 23, the second decoder 25, the third decoder 27, and the fourth decoder 29 are triggered. Thus, image input is performed simultaneously from the four cameras (22, 24, 26, 28) (S6 in FIG. 8).

The image captured by the first camera 22 is converted from the composite signal to the component signal by the first decoder 23, and the luminance signal of the converted component signals is converted into a digital signal, and the image I ₁ (x , Y) is generated. Similarly, the images taken by the remaining cameras (24, 26, 28) are converted into images I ₂ (x, y), I ₃ (x, y), and I ₄ (x, y) through the decoder, respectively. The

The input four images I _i (x, y) are coordinated according to the rules described above with reference to the coordinate conversion table stored in the coordinate conversion table storage unit 84 by the action of the coordinate conversion means 90. Replacement is performed, and a composite image L (x, y) is generated (S7 in FIG. 8).

  After performing the above operation using the coordinate conversion table, the fixed pattern 160 representing the vehicle 150 is superimposed on the coordinate-converted image, and the image shown in FIG. 9 is generated. This image is displayed on the image display means 100 (S8 in FIG. 8). Thereafter, S6 to S8 in FIG. 8 are repeated.

  When it is detected by the shift position detecting means 120 that the shift position is other than the reverse position (S9 in FIG. 8) or the end switch 114 is detected (S10 in FIG. 8), the image is displayed. The image representing the vehicle surroundings displayed on the display means 100 is erased, and the display content before the vehicle surrounding monitoring device 5 is operated is displayed (S11 in FIG. 8).

  According to the first on-vehicle camera calibration device according to the present invention configured as described above, the region specifying unit specifies the region including the reference marker for the image shot by the shooting unit, and the contrast calculating unit. Calculates a value corresponding to the contrast in the specified region, and the contrast determination means determines whether the calculated value is larger than a predetermined value, and if the calculated value is larger than the predetermined value, When it is small, after changing the exposure of the photographing means by the first exposure control means, the photographing means takes a picture again according to the instruction from the first photographing instruction means, and the contrast calculating means for the photographed image, A value corresponding to the contrast in the specified area is calculated, and when the contrast determination unit determines that the calculated value is greater than a predetermined value, the captured image is By calibrating the camera with the calibration execution means, the coordinates of the feature points in the reference marker appearing in the image can be accurately measured, so that the camera calibration can be reliably performed regardless of the lighting environment. It can be carried out.

  In this embodiment, a checkered pattern is used as the reference marker, but the checkered pattern is not limited to this. That is, the positions of feature points such as checkerboard vertices can be specified, and the coordinates of at least six feature points and the three-dimensional coordinates of those feature points from an image photographed by one camera are used. As long as it can be associated with, another type of reference marker may be used.

  In this embodiment, a vehicle surrounding image captured by a plurality of cameras installed in a vehicle with the second on-vehicle camera calibration device (hereinafter referred to as an on-vehicle camera calibration device 3) according to the present invention is shown to the driver. The present invention is applied to the vehicle surrounding monitoring device 6 that displays in an easy-to-view form.

  In particular, in this embodiment, in order to calibrate the camera, when photographing a reference marker arranged outside the vehicle, each time the camera exposure is changed, a plurality of exposures changed from the same camera. An image is photographed, an image in which a region including the reference marker is photographed most clearly is selected from the photographed images, and the camera is calibrated using the selected image.

  FIG. 10 is a block diagram showing a configuration of the vehicle surrounding monitoring device 6 according to the embodiment of the present invention. The vehicle surrounding monitoring apparatus 6 according to the present invention is installed in a reference marker 10 having a black and white light and dark pattern installed on a road surface and a vehicle 150 (see FIG. 2), and photographs images in different directions around the vehicle. An imaging unit 20 configured by a plurality of cameras and performing imaging in accordance with an instruction from the second imaging instruction unit 36, and an area specifying unit for specifying the position of an area including the reference marker 10 among images captured by the imaging unit 20 30 and a contrast calculating means 40 for calculating a value C corresponding to the contrast of the area including the reference marker 10 specified by the area specifying means 30, and the value C is larger than a predetermined value among the captured images, and An image selecting means 55 for selecting an image having the maximum value C, and a second exposure changing means 62 for changing the exposure of the camera constituting the photographing means 20 a predetermined number of times over a plurality of stages. In response to an instruction from the second exposure changing means 62, every time the exposure is changed a predetermined number of times, the second photographing instruction means 36 for giving a photographing instruction to the photographing means 20 and calibration for executing camera calibration An execution means 70, a coordinate conversion table creation means 80 for creating a coordinate conversion table based on the result of calibration, a coordinate conversion means 90 for converting an image photographed by the photographing means 20 into a bird's eye image, and The image display means 100 for displaying the coordinate-converted image, the operation switch 110 for instructing the start and end of the vehicle surrounding monitoring device 6 and the contents of the camera calibration operation, and the shift position of the vehicle 150 are set to the reverse position. It comprises shift position detecting means 120 for detecting the presence.

  Among these, the reference marker 10, the imaging unit 20, the region specifying unit 30, the second imaging instruction unit 36, the contrast calculation unit 40, the image selection unit 55, the second exposure change unit 62, and the calibration execution. The means 70 and a part of the operation switch 110 (image monitor switch 115, camera selection switch 116, corresponding point instruction start switch 117, cursor movement switch 118) constitute the in-vehicle camera calibration device 3 of the present invention.

  The detailed configurations of the reference marker 10, the photographing unit 20, the calibration execution unit 70, and the operation switch 110 are the same as those in the first embodiment.

  The vehicle surrounding monitoring device 6 calibrates each camera (22, 24, 26, 28) installed in the vehicle 150, and then uses the result to calibrate each camera (22, 24, 26, 28). 28), the image captured in 28) is converted into an image viewed from above the vehicle 150, and the images converted into the overhead image are combined into a single image and presented to the driver. Of these, image conversion and image composition are the same as those in the first embodiment, and therefore, only the camera calibration operation portion will be described with reference to the flowchart of FIG.

  Assume that four cameras from the first camera 22 to the fourth camera 28 are installed in the vehicle 150 as in the first embodiment.

  First, the vehicle 150 is placed at a predetermined place for calibration (S2 in FIG. 11).

  The operator who performs the calibration installs the reference markers 10 having checkered patterns whose dimensions have been measured in advance (S1 in FIG. 11) at the four corners of the vehicle 150 (S3 in FIG. 11), and then the image monitor. The switch 115 is operated so that the image captured by the imaging unit 20 can be observed by the image display unit 100, and the camera selection switch 116 is operated to display an image from the first camera 22 to the fourth camera 28. Select the camera to be observed.

  Here, the operator first follows the procedure described in the first embodiment so that the reference markers (12, 14, 16, 18) can be seen in each camera (22, 24, 26, 28) without interruption. Then, the installation position is finely adjusted (S4 in FIG. 11).

  After the fine adjustment of the installation position of the reference marker 10 is completed, the three-dimensional coordinates of the vertex of each reference marker 10 are calculated as shown in FIG. 3 by the procedure described in the first embodiment (S5 in FIG. 11). ).

  Next, the operator specifies the approximate position of the area in which the reference marker 10 is captured from the captured image according to the procedure described in the first embodiment by the action of the area specifying unit 30 (see FIG. 11 S6).

By this operation, as shown in FIG. 4, in the image I ₁ (x, y) input from the first camera 22, the rectangular region R _{11 in} which the first reference marker 12 is reflected and the second reference marker 14. rectangular area R ₁₂ is identified that is captured. Further, the coordinates of the upper left vertex and the coordinates of the lower right vertex of the specified rectangular regions R ₁₁ and R ₁₂ are stored in the contrast calculation means 40.

  The procedure for specifying a rectangular area including the reference marker 10 is sequentially performed in the same manner for images taken by the second camera 24, the third camera 26, and the fourth camera 28.

Next, the exposure amount of the photographing unit 20 is changed a predetermined number of times over a plurality of stages by the action of the second exposure changing unit 62. At this time, every time the exposure amount is changed in accordance with an instruction from the second imaging instruction unit 36, the imaging unit 20 captures an image a predetermined number of times (S7 in FIG. 11). Here, it is assumed that an image taken b-th by the a-th camera is represented by I _ab (x, y).

  The method of changing the exposure may be determined according to the specifications of the camera to be used. For example, first, after photographing with the exposure amount determined according to the brightness of the external environment at that time, , -2EV, -1EV, + 1EV, + 2EV in this order. In this case, a total of five images are taken with different exposure amounts. The image thus shot is stored in the image selection means 55.

After image capture with the four cameras (22, 24, 26, 28) is completed (S8 in FIG. 11), the contrast calculation means 40 causes each of the four cameras (22, 24, 26, 28) to For each image I _ab (x, y) photographed a predetermined number of times, each rectangular area R _ij (i = 1, 2, 3, 4, j = 1, 2) specified by the area specifying means 30. A value C corresponding to the contrast of the image I _ab (x, y) in the middle is calculated (S9 in FIG. 11). Various methods can be considered for digitizing the value C. Here, it is assumed that the value C is calculated by (Equation 1) described in the first embodiment.

Next, each rectangular region including the reference marker 10 is selected from the plurality of images I _ab (x, y) photographed by each of the four cameras (22, 24, 26, 28) by the image selection means 55. (S11 in FIG. 11), the contrast of the image I _ab (x, y) in each rectangular area R _ij (i = 1, 2, 3, 4, j = 1, 2) calculated previously is set. An image in which the corresponding value C is larger than the predetermined value Cth that has been determined in advance and maximizes is selected (S10 in FIG. 11).

When an image whose value C is larger than a predetermined value Cth and is maximized is selected (when S12 in FIG. 11 is Yes), the region number R _ij and the corresponding image I _ab (x , Y) is sent to the corresponding point instruction section 72 (S13 in FIG. 11).

  If an image whose value C is larger than the predetermined value Cth determined in advance is not selected (when S12 in FIG. 11 is No), the exposure amount of the photographing means 20 is again over a plurality of stages. Changed a predetermined number of times. The exposure change amount at this time is set larger than the first exposure change amount. Further, every time the exposure amount is changed in accordance with an instruction from the second imaging instruction unit 36, the imaging unit 20 captures an image again a predetermined number of times (S14 in FIG. 11).

  After shooting a predetermined number of times, the process proceeds to S10 in FIG. 11 again, and the value C corresponding to the contrast is calculated and the image satisfying the predetermined condition is selected as before.

  The above procedure is sequentially performed on the four cameras (22, 24, 26, and 28) (S15 in FIG. 11), and an image in which all the reference markers 10 are captured with high contrast is acquired.

  Thereafter, using the image sent to the corresponding point instruction unit 72, the camera is calibrated according to the flowchart of FIG. Since the function of calibration is as described in the first embodiment, the description is omitted here.

After the calibration is performed, a coordinate conversion table is created by the coordinate conversion table creating means 80 according to the flowchart of FIG. 8, and based on the created coordinate conversion table, images I _i (x , Y) is converted into an image obtained by looking down the vehicle from directly above by the coordinate conversion means 90, further synthesized into one image, and displayed on the image display means 100.

  According to the on-vehicle camera calibration device 3 according to the second embodiment configured as described above, every time the exposure of the photographing unit is changed a predetermined number of times by the second exposure control unit, the second photographing instruction unit In accordance with the instruction, a value corresponding to the contrast of the area including the reference marker specified by the area specifying means is calculated by the contrast calculating means for the plurality of images taken by the imaging means, and calculated by the image selecting means. An image having a value corresponding to contrast larger than a predetermined value and having a maximum value corresponding to the contrast is selected, and the camera is calibrated by the calibration execution means using the selected image. Can accurately measure the coordinates of the feature points in the reference marker reflected in the Calibration of the camera can be reliably performed.

  In this embodiment, a vehicle surrounding image obtained by photographing a third on-vehicle camera calibration device (hereinafter referred to as an on-vehicle camera calibration device 4) according to the present invention with a plurality of cameras installed on the vehicle is displayed to the driver. The present invention is applied to the vehicle surrounding monitoring device 7 that displays in an easy-to-view form.

  In particular, in this embodiment, in order to calibrate the camera, after imaging a reference marker arranged outside the vehicle, a region including the reference marker in the captured image has a contrast that satisfies a predetermined condition. Thus, the contrast of the captured image is corrected, and the camera is calibrated using the image with the corrected contrast.

  FIG. 12 is a block diagram showing a configuration of the vehicle surrounding monitoring device 7 according to the embodiment of the present invention. The vehicle surrounding monitoring apparatus 7 according to the present invention is provided on a reference marker 10 having a black and white light and dark pattern installed on a road surface, and on a vehicle 150 (see FIG. 2), and takes images in different directions around the vehicle 150. A plurality of cameras, a region specifying unit 30 for identifying the position of the region including the reference marker 10 from the images captured by the photographing unit 20, and a reference marker specified by the region specifying unit 30 10, a contrast calculating unit 40 that calculates a value C corresponding to the contrast of the region including 10, a contrast correcting unit 56 that corrects the value C when the value C calculated by the contrast calculating unit 40 is smaller than a predetermined value, Based on the calibration execution means 70 for executing calibration and the result of the calibration, a coordinate conversion table is created. A coordinate conversion table creating means 80; a coordinate conversion means 90 for converting an image photographed by the photographing means 20 into a bird's eye image; an image display means 100 for displaying the coordinate-converted image in the vehicle 150; It comprises an operation switch 110 for instructing the start and end of the vehicle surroundings monitoring device 7 and the contents of the camera calibration operation, and a shift position detecting means 120 for detecting that the shift position of the vehicle 150 is in the reverse position.

  Among these, the reference marker 10, the photographing unit 20, the region specifying unit 30, the contrast correcting unit 56, the calibration executing unit 70, and a part of the operation switch 110 (image monitor switch 115, camera selection switch 116, corresponding The point indication start switch 117 and the cursor movement switch 118) constitute the in-vehicle camera calibration device 4 of the present invention.

  The detailed configurations of the reference marker 10, the photographing unit 20, the calibration execution unit 70, and the operation switch 110 are the same as those in the first embodiment.

  The vehicle surrounding monitoring device 7 calibrates each camera (22, 24, 26, 28) installed in the vehicle 150, and uses the result to calibrate each camera (22, 24, 26, 28). 28), the image captured in 28) is converted into an image viewed from above the vehicle 150, and the images converted into the overhead image are combined into a single image and presented to the driver. Of these, image conversion and image composition are the same as those in the first embodiment. Therefore, only the camera calibration operation will be described based on the flowchart of FIG.

  Assume that four cameras from the first camera 22 to the fourth camera 28 are installed in the vehicle 150 as in the first embodiment.

  First, the vehicle 150 is placed at a predetermined location for calibration (S2 in FIG. 13).

  The operator who performs the calibration sets the reference markers 10 having checkered patterns whose dimensions have been measured in advance (S1 in FIG. 13) at the four corners of the vehicle 150 (S3 in FIG. 13), and then the image monitor. The switch 115 is operated so that the image captured by the imaging unit 20 can be observed by the image display unit 100, and the camera selection switch 116 is operated to display an image from the first camera 22 to the fourth camera 28. Select the camera to be observed.

  Here, the operator first follows the procedure described in the first embodiment so that the reference markers (12, 14, 16, 18) can be seen in each camera (22, 24, 26, 28) without interruption. Then, the installation position is finely adjusted (S4 in FIG. 13).

  After the fine adjustment of the installation position of the reference marker 10 is completed, the three-dimensional coordinates of the vertex of each reference marker 10 are calculated as shown in FIG. 3 by the procedure described in the first embodiment (S5 in FIG. 13). ).

  Next, the operator specifies the approximate position of the area in which the reference marker 10 is captured from the captured image according to the procedure described in the first embodiment by the action of the area specifying unit 30 (see FIG. 13 S6).

By this operation, as shown in FIG. 4, in the image I ₁ (x, y) input from the first camera 22, the rectangular region R _{11 in} which the first reference marker 12 is reflected and the second reference marker 14. rectangular area R ₁₂ is identified that is captured. Further, the coordinates of the upper left vertex and the coordinates of the lower right vertex of the specified rectangular regions R ₁₁ and R ₁₂ are stored in the contrast correction unit 56.

  The procedure for specifying a rectangular area including the reference marker 10 is sequentially performed in the same manner for images taken by the second camera 24, the third camera 26, and the fourth camera 28.

Next, for the four images I _i (x, y) photographed by each camera, the value C corresponding to the contrast in the rectangular area specified by the area specifying means 30 by the contrast correcting means 56 is obtained. , (Equation 1) (S7 in FIG. 13).

When values C calculated from two rectangular areas including the reference marker 10 in the image I _i (x, y) are both larger than a predetermined value Cth (S8 in FIG. 13) The number R _ij and the captured image I _i (x, y) are sent to the corresponding point instruction unit 72 (S9 in FIG. 13).

When the calculated value C is larger than the predetermined value Cth in only one of the two rectangular areas including the reference marker 10 (when S10 in FIG. 13 is Yes), the value C is higher than the predetermined value Cth. Area number R _ij having a larger value and the captured image I _i (x, y) are sent to the corresponding point instruction unit 72 (S11 in FIG. 13).

For a rectangular area where the value C is smaller than the predetermined value Cth, the contrast of the image I _i (x, y) is corrected by the contrast correcting means 56 (S12 in FIG. 13). Various methods have been proposed for correcting the contrast of an image, and any of these methods may be used, but here, a method using a density histogram of an image shown in FIG. 14 is used.

FIG. 14 shows an example of the density histogram H ₁ of the rectangular region R ₁₁ in the image I ₁ (x, y) of FIG. The grayscale histogram is a frequency distribution table in which the horizontal axis represents the grayscale values that can be represented by the image, and the vertical axis represents the total number of pixels having each grayscale value included in the image. If the image I ₁ (x, y) is quantized to 8 bits, the horizontal axis of the density histogram H ₁ can take a value from 0 to 255.

From the density histogram H _{1 in} FIG. 14, the image I ₁ (x, y) has only a gray value between the minimum value Tmin and the maximum value Tmax in the image I ₁ , and from 0, which is a range of values that the image can take. It has only a narrow range of values with respect to 255, and it can be seen that the contrast of the image is low.

  Here, correction for increasing the contrast of the image is performed according to (Expression 6). That is, assuming that the gray value before correction of a pixel is z and the gray value after correction is z ′, z ′ is as follows.

Correction according to (Equation 6) is called histogram expansion, and is often used when image contrast enhancement is performed. Based on the equation (6), by correcting the gray values of the rectangular area R _11, it is emphasized contrast of the rectangular region R ₁₁ and the value C is increased. For example, density histogram H ₁ shown in FIG. 14 is corrected as density histogram H _2. Here, to be expressed image _I i (x, y) an image contrast correction is performed on the _{I i} '(x, y) at.

For the image I _i ′ (x, y) whose contrast has been corrected, the contrast calculation means 40 calculates the value C again (S13 in FIG. 13), and determines that the value C is greater than the predetermined value Cth. (When S14 in FIG. 13 is Yes), the region number R _ij and the contrast-corrected image I ₁ ′ (x, y) are sent to the corresponding point instruction unit 72 (S15 in FIG. 13).

Despite the contrast correction, when the value C is smaller than the predetermined value Cth (when S14 in FIG. 13 is No), Tmin = Tmin + k, Tmax = Tmax−k (k = 1, 2, 3, ..) As new Tmin and Tmax, contrast correction is performed again on the first image by (Equation 6) (S12 in FIG. 13). This is to increase the contrast correction effect by increasing the degree of expansion of the histogram. The value of k is updated, the correction for the original image is repeated, the value C is checked each time, and if an image I _i ′ (x, y) having a value C greater than the predetermined value Cth is obtained, the region The number R _ij and the contrast-corrected image I _i ′ (x, y) are sent to the corresponding point instruction unit 72 (S15 in FIG. 13).

When the value C is smaller than the predetermined value Cth in the two areas including the reference marker 10 (when S10 in FIG. 13 is No), the contrast correction means 56 corrects the contrast in both of the two rectangular areas. (S12 in FIG. 13). As a result, when it is determined that the value C is larger than the predetermined value Cth, the region number R _ij and the contrast-corrected image I _i ′ (x, y) are sent to the corresponding point instruction unit 72 (S15 in FIG. 13). ).

  When the value C is smaller than the predetermined value Cth in spite of the contrast correction (No in S14 in FIG. 13), the values of Tmin and Tmax are updated as described above to correct the contrast (see FIG. 13 S12) is repeated.

  The above procedure is sequentially performed from the first camera 22 to the fourth camera 28 (S16 in FIG. 13), and an image in which all the reference markers 10 are captured with high contrast is acquired.

  Thereafter, using the image sent to the corresponding point instruction unit 72, the camera is calibrated according to the flowchart of FIG. Since the function of calibration is as described in the first embodiment, the description is omitted here.

After the calibration is performed, a coordinate conversion table is created by the coordinate conversion table creating means 80 according to the flowchart of FIG. 8, and based on the created coordinate conversion table, images I _i (x , Y) is converted into an image obtained by looking down the vehicle from directly above by the coordinate conversion means 90, further synthesized into one image, and displayed on the image display means 100.

  According to the on-vehicle camera calibration device 4 according to the third embodiment configured as described above, an area including the reference marker specified by the area specifying means by the contrast calculating means from the image taken by the photographing means. When the value corresponding to the contrast is calculated and the contrast determining means determines that the value corresponding to the calculated contrast is smaller than the predetermined value, the value corresponding to the contrast is lower than the predetermined value by the contrast correcting means. By correcting the image so that it becomes larger and calibrating the camera using the calibration execution means, the coordinates of the feature points in the reference marker appearing in the image can be accurately measured. Surely calibrate the camera regardless of the lighting environment Door can be.

2 Car-mounted camera calibration device 5 Vehicle surroundings monitoring device 10 Reference marker 20 Imaging means 30 Area specifying means 35 First imaging instruction means 40 Contrast calculation means 50 Contrast judgment means 60 First exposure change means 70 Calibration execution means 80 Coordinates Conversion table creation means 90 Coordinate conversion means 100 Image display means 110 Operation switch 120 Shift position detection means

Claims (3)

A reference marker having a specific light and dark pattern arranged so that a three-dimensional position can be specified;
Photographing means for photographing an image including the reference marker;
An area specifying means for specifying an area in which the reference marker is captured from the image shot by the shooting means;
Contrast calculating means for calculating a value corresponding to the contrast in the area specified by the area specifying means from among the images shot by the shooting means;
Contrast determination means for determining whether a value corresponding to the contrast calculated by the contrast calculation means is larger than a predetermined value;
First exposure changing means for changing the exposure of the photographing means so that a value corresponding to the contrast is larger than a predetermined value;
After the exposure of the photographing means is changed by the first exposure changing means, the first photographing instruction means for causing the photographing means to take an image and the photographed image are used to calibrate the photographing means. A vehicle-mounted camera calibration apparatus, comprising: a calibration execution unit for performing the calibration. A reference marker having a specific light and dark pattern arranged so that a three-dimensional position can be specified;
Photographing means for photographing an image including the reference marker;
An area specifying means for specifying an area in which the reference marker is captured from the image shot by the shooting means;
Second exposure changing means for changing the exposure of the photographing means a predetermined number of times over a plurality of stages;
Contrast calculating means for calculating a value corresponding to the contrast in the area specified by the area specifying means from among the images shot by the shooting means;
A second photographing instruction means for causing the photographing means to photograph an image each time the exposure of the photographing means is changed a predetermined number of times by the second exposure changing means;
Image selecting means for selecting an image having a value corresponding to the contrast larger than a predetermined value and having a maximum value corresponding to the contrast from among a plurality of images photographed in accordance with an instruction from the second photographing instruction means. When,
An in-vehicle camera calibration apparatus comprising calibration execution means for calibrating the photographing means using an image selected by the image selection means. A reference marker having a specific light and dark pattern arranged so that a three-dimensional position can be specified;
Photographing means for photographing an image including the reference marker;
An area designating unit for designating an area in which the reference marker is captured from an image photographed by the photographing unit;
Contrast calculating means for calculating a value corresponding to the contrast in the area specified by the area specifying means from among the images shot by the shooting means;
Contrast determination means for determining that a value corresponding to the contrast calculated by the contrast calculation means is greater than a predetermined value;
Contrast correction means for correcting the value corresponding to the contrast to be larger than the predetermined value when the contrast determination means determines that the value corresponding to the contrast is smaller than the predetermined value; Have
An in-vehicle camera calibration apparatus comprising calibration execution means for calibrating the photographing means using the image corrected by the contrast correction means.