JP2015186026A - Camera calibration device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a camera calibration without the influence of ambient brightness.SOLUTION: An imaging unit 30 having an imaging element 32 comprising near-infrared light receiving elements 34 (34a, 34b, 34c, ...) sensitive to near-infrared light images plural targets 24(24a, 24b, 24c, 24d) transmitting therethrough or reflecting near-infrared light. A target position measurement unit 48 measures the respective positions of the targets 24 from inside an observed image I'. An image correction unit 52 geometrically corrects the image I' so that the targets 24 may be observed at positions where they should be observed originally.

Description

  The present invention relates to the internal distance of a camera such as a focal length by associating a three-dimensional position of a point in space with a two-dimensional position of a point on the image when the point is imaged by the camera. The present invention relates to a camera calibration apparatus that determines parameters and external parameters related to the installation position and installation direction of the camera.

  In recent years, an ITS system that installs a camera on a vehicle, detects a road area and a preceding vehicle from the observed images, analyzes the relationship between the vehicle and the road environment, and assists safe driving has become widespread. is there.

  In order to realize such a system and exhibit stable performance, in general, external parameters such as the position of the camera installed in the vehicle and its mounting direction, and internal parameters such as the camera focal length and lens distortion are used. It needs to be properly calibrated. For this reason, various calibration apparatuses have been proposed.

JP 2010-107348 A

  However, for example, the calibration device described in Patent Document 1 is configured to detect the position of an installed target (marker) and calculate the center of gravity position as the marker installation position.

  In such a conventional calibration apparatus, in order to measure the positions of a plurality of markers (targets) observed by the camera, the markers reflected in the image are detected by image processing, and the positions are detected. I was measuring. At this time, if the ambient brightness fluctuates, there is a possibility that the marker detection accuracy may be lowered. Therefore, there is a problem that the accuracy of the calibration is deteriorated depending on the use environment.

  The present invention has been made in view of the above-described problems, and can perform calibration by stably measuring the position of a marker (target) without being affected by the brightness of surroundings to be calibrated. An object is to provide a calibration device.

  The camera calibration apparatus according to the present invention reliably measures the position of a target used for calibration, regardless of fluctuations in ambient brightness.

  That is, the camera calibration apparatus according to the present invention includes an imaging device including a near-infrared light receiving element having sensitivity to near-infrared light, an imaging unit that generates an image using the imaging device, and the near-infrared light. A target position measuring unit that measures each position of the plurality of targets out of an image including a plurality of targets that are transmitted or reflected and the plurality of targets captured by the imaging unit, and the image An image correction unit that performs geometric correction of the image so that the plurality of targets are reflected at positions where they are originally observed based on the positions of the plurality of targets in the image. To do.

  According to the camera calibration device according to the present invention configured as described above, since the imaging element constituting the imaging unit includes a near-infrared light receiving element having sensitivity to near-infrared light, When a plurality of targets that transmit or reflect infrared light are photographed, a plurality of targets are observed with high contrast in the observed image. Therefore, the positions of a plurality of targets can be reliably and accurately measured by the target position measuring unit. And since the image correction unit corrects the position on the image of the plurality of targets measured by the target position measurement unit, even if the ambient brightness fluctuates, without using an illumination device or the like, The imaging unit can be calibrated reliably without being affected by ambient brightness.

  According to the camera calibration apparatus of the present invention, the camera can be calibrated stably without being affected by the brightness of the surroundings for calibration.

It is an external view which shows the state which mounted the camera calibration apparatus which concerns on Example 1 which is 1 embodiment of this invention. (A) is a figure which shows the external appearance which looked at the front windshield of the vehicle from the front, when the camera calibration apparatus which concerns on Example 1 is mounted in the vehicle. (B) is the side view which looked at Fig.5 (a) from the direction of arrow E. FIG. 1 is a functional block diagram illustrating a functional configuration of a camera calibration device according to Embodiment 1. FIG. BRIEF DESCRIPTION OF THE DRAWINGS It is a figure explaining the structure of the image pick-up element in Example 1, (a) is a top view of an image pick-up element. (B) is sectional drawing which cut the image pick-up element by cutting line AA. FIG. 3 is a diagram illustrating a schematic pattern of spectral sensitivity of each light receiving element that constitutes the imaging element according to the first embodiment. (A) is an example of the image which shows the state image | photographed in the position which should have a visual target in Example 1. FIG. (B) is an example of an image showing a state in which the visual target is actually photographed in the first embodiment. (C) is a figure which shows the specific example of the optotype used in Example 1. FIG. It is a figure which shows a mode that the image shown to Fig.6 (a) and the image shown to FIG.6 (b) were piled up. (A) is a figure showing the amount of position shift of the horizontal direction of the observation position of a target computed based on the result of Drawing 7. (B) is a figure showing the amount of position shift of the observation position of a target in the vertical direction calculated based on the result of FIG. 3 is a flowchart illustrating an overall flow of processing performed in the first embodiment. 10 is a flowchart showing a flow of calibration performed in the flowchart of FIG. 9.

  Hereinafter, specific embodiments of a camera calibration apparatus according to the present invention will be described with reference to the drawings.

In this embodiment, the present invention is applied to a camera calibration apparatus that calibrates a camera installed in a vehicle.
(Description of the configuration of the camera calibration device)

  First, the configuration of a camera calibration apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating a state in which a camera calibration device 10 according to the present embodiment is mounted on a vehicle V.

  The camera calibration device 10 is a visual target attached to an outer surface of a windshield 20 and an imaging unit 30 installed toward the outside of the vehicle V in the vicinity of the rear side of the room mirror 12 of the vehicle V. A transparent film 22 on which 24a, 24b, 24c, and 24d are printed or pasted is provided.

  The imaging unit 30 has an imaging element 32 (FIG. 3) such as a CCD or CMOS. The specific configuration of the image sensor 32 will be described later.

  The transparent film 22 has the same spectral transmission characteristics as the windshield 20 (windshield), and four visual targets 24a, 24b, 24c, and 24d are drawn on the surface thereof.

  Although the four visual targets 24a, 24b, 24c, and 24d do not have to have the same size, the transmission characteristics of near-infrared light (for example, light of 750 to 850 nm) is near-infrared of the windshield 20 (windshield). It has characteristics different from the light transmission characteristics. That is, when the near-infrared light transmittance of the windshield 20 is 100%, the targets 24a, 24b, 24c, and 24d all have spectral transmission characteristics that block (do not transmit) near-infrared light. It shall be. The four visual targets 24 a, 24 b, 24 c and 24 d are all within the photographing field of view of the imaging unit 30. Hereinafter, the four visual targets 24a, 24b, 24c, and 24d are collectively referred to simply as the visual target 24. The specific shape of the visual target 24 will be described later.

  Specifically, the imaging unit 30 detects the position of the lane marker in front of the vehicle V by detecting the position of the imaging unit of the drive recorder that records the state of the outside of the vehicle V or the vehicle V, and deviates from the lane. Used for various applications such as alarm devices. However, the content of the present invention does not limit the specific application of the imaging unit 30.

  2A and 2B are diagrams illustrating a detailed mounting state of the imaging unit 30. FIG. FIG. 2A is a view of the vehicle V as viewed from the front. A transparent film 22 is attached to the outer surface of the windshield 20 (windshield). The transparent film 22 is affixed at a position including the imaging field of view of the imaging unit 30. A target 24 (24a, 24b, 24c, 24d) is printed or pasted on the transparent film 22. In addition, since the transparent film 22 has the same spectral transmission characteristics as the windshield 20 as described above, there is no influence on the image picked up by the image pickup unit 30 by attaching the transparent film 22. .

  The target 24 (24a, 24b, 24c, 24d) is different from the windshield 20 (windshield) only in the near-infrared light transmission characteristics, so that its presence can be confirmed as far as the driver sees it. Can not. Therefore, when the windshield 20 is manufactured, the visual target 24 may be embedded in the glass. In this case, the near-infrared light transmittance of the windshield 20 may be substantially 0, and the near-infrared light transmittance of the visual target 24 may be substantially 100%.

  The imaging field of view of the imaging unit 30 is set on the surface of the windshield 20 (windshield) so as to be within a region R1 shown in FIG. The application using the imaging unit 30 is set so as to utilize image information in the range of the region R2 inside the region R1.

  All the visual targets 24 (24a, 24b, 24c, 24d) are installed at positions that are reflected in the region R1 and in the region outside the outer periphery of the region R2.

2B is a side view of FIG. 2A viewed from the direction of arrow E. FIG. As shown in FIG. 2B, the imaging unit 30 is installed on the back side of the windshield 20 (windshield) of the vehicle V via the camera fixing unit 26. The imaging unit 30 is fixed so as to be positioned in the vicinity of the back side of the room mirror 12 and is installed so as not to enter the field of view of the driver of the vehicle V.
(Description of functional configuration of camera calibration device)

  FIG. 3 is a functional block diagram illustrating a functional configuration of the camera calibration apparatus 10 according to the present embodiment.

  The camera calibration device 10 is mounted on a vehicle V and, as shown in FIG. 3, an imaging unit 30 that monitors the outside of the vehicle V from the inside of the windshield 20 (windshield), an image processor 40, and imaging The correction data storage unit 60 for storing the position where the visual target 24 is originally observed in the obtained image and the correction data obtained by the calibration, the calibration start instruction, and the image pickup unit 30 And a microcomputer 70 that executes a specific application using an image.

  As described above, the target 24 (24a, 24b, 24c, 24d) is embedded in the windshield 20 (windshield), and the windshield 20 has a spectral transmission characteristic that cuts near infrared light. It is assumed that the visual target 24 has a spectral transmission characteristic that transmits near-infrared light.

  The imaging unit 30 includes an imaging element 32 and a plurality of near-infrared light receiving elements 34a, 34b, 34c,. Hereinafter, the near-infrared light receiving elements 34a, 34b, 34c,... Are collectively referred to as the near-infrared light receiving element 34 unless otherwise specified. The detailed structure of the image sensor 32 will be described later.

  The image processor 40 detects all the targets 24 (24a, 24b, 24c, 24d) from the captured images and measures their positions. In addition, when the target 24 is observed to deviate from the originally observed position in the captured image, geometric correction of the image is performed, and an image in which the target 24 is observed at a predetermined position is obtained. Generate. Specifically, the image processor 40 is configured by an ASIC (Application Specific Integrated Circuit) that is an integrated circuit.

  The image processor 40 includes an image input unit 42, a near-infrared image selection unit 44, a color image selection unit 46, a target position measurement unit 48, a target position comparison unit 50, an image correction unit 52, Have

  The image input unit 42 instructs the imaging unit 30 to perform imaging, separates the near-infrared signal from the output signal captured by the imaging unit 30, and further generates a color signal.

  The near infrared image selection unit 44 selects and outputs the near infrared signal generated by the image input unit 42.

  The color image selection unit 46 selects and outputs the color signal generated by the image input unit 42.

  The target position measurement unit 48 detects all the targets 24 (24a, 24b, 24c, 24d) from the near-infrared signal output from the near-infrared image selection unit 44, and determines the center position thereof. measure.

  The target position comparison unit 50 compares the center positions of all the targets 24 measured by the target position measurement unit 48 with the positions where the target 24 is originally observed. The position where the visual target 24 is originally observed is stored in advance in the correction data storage unit 60.

  The image correction unit 52 uses the correction data stored in the correction data storage unit 60 when the position of the target 24 reflected in the captured image is deviated from the position of the target 24 that is originally observed. The image is geometrically corrected so that the visual target 24 is reflected at the position where it is originally observed. A detailed correction method will be described later.

  The microcomputer 70 includes a calibration start instruction unit 72 and an application execution unit 74.

  The calibration start instruction unit 72 outputs an instruction of an operator or a program incorporated in the application and instructs the image input unit 42 to start calibration.

The application execution unit 74 executes the various applications described above that use the image captured by the imaging unit 30. Note that the image processor 40 is used when complex image processing needs to be performed depending on the contents of the application, or particularly when high-speed processing is required.
(Description of the structure of the image sensor)

  Next, a specific structure of the image sensor 32 will be described with reference to FIGS. FIG. 4A is a diagram illustrating an example of the arrangement of color filters installed on the top of the light receiving element corresponding to each pixel of the image sensor 32. As shown in FIG. 4 (a), the red filter R, the green filter G, and the blue filter B, which are color filters, are arranged in accordance with the rules of the so-called Bayer arrangement at the upper part of each light receiving element constituting the imaging element 32. Is provided. A part of the green filter G is replaced with a near-infrared transmission filter C that transmits visible light to near-infrared light (for example, 850 nm). Among these, the pixel provided with the near-infrared transmission filter C is the above-described near-infrared light receiving element 34 (34a, 34b, 34c,...), And FIG. The light receiving elements 34a to 34d are shown.

FIG. 4B shows a cross-sectional view of FIG. 4A taken along the cutting line AA. As shown in FIG. 4B, the imaging element 32 is provided with the above-described filters (R, G, B, C) on the upper part of the light receiving element 36 provided on the bottom thereof, and further on the upper part thereof. In addition, an IR cut filter 38 that blocks infrared light having a predetermined wavelength or more (for example, 900 nm or more) and between the visible light region and the near infrared region (for example, 650 to 830 nm) is provided.
(Explanation of spectral sensitivity characteristics of filter)

  Next, spectral sensitivity characteristics of each filter provided in the image sensor 32 will be described with reference to FIG. FIG. 5 shows the spectral sensitivity S of each filter shown in FIG. The red filter R, the green filter G, and the blue filter B have a characteristic of transmitting only light having a wavelength corresponding to each color.

  The maximum sensitivity values of the color filters are actually different from each other, but in order to make the explanation easy to understand, the maximum sensitivity values of the red filter R, the green filter G, and the blue filter B are equal in FIG. It draws like so.

Here, the near infrared transmission filter C has sensitivity to near infrared light in addition to visible light. Therefore, the near-infrared light receiving element 34 (34a, 34b, 34c,...) Provided with the near-infrared transmission filter C is also capable of detecting near-infrared light that cannot be detected by the red filter R, the green filter G, and the blue filter B. Has sensitivity.
(Explanation of calibration method)

  Next, a specific calibration method will be described with reference to FIGS. FIG. 6A is an example of an image I showing a state where the image capturing unit 30 has photographed the target 24 (24a, 24b, 24c, 24d) at a position where the target should be. In FIG. 6A, the center position of the target 24a is observed at the point C1, the center position of the target 24b is observed at the point C2, and the center position of the target 24c is observed at the point C3. This shows that the center position of the mark 24d is observed at the point C4.

  FIG. 6B is an example of an image I ′ showing a state in which the visual target 24 (24a, 24b, 24c, 24d) is actually captured by the imaging unit 30. As described above, the windshield 20 (windshield) (FIG. 1) and the visual target 24 are significantly different from each other in the transmittance of near-infrared light. Therefore, the visual target 24 is higher in the image I ′ than the surroundings. It is imaged with contrast. At this time, whether the visual target 24 is observed brighter or darker than the surroundings depends on the spectral transmittance of the near-infrared light in the windshield 20 and the near-infrared light in the visual target 24. It depends on the magnitude relationship of spectral transmittance.

  The image is picked up due to an attachment error of the image pickup unit 30, a position shift of the image pickup unit 30 due to vibration during traveling of the vehicle V, a distortion of a lens constituting an optical system provided in the image pickup unit 30, or the like. The position of the visual target 24 shown in the image I ′ is deviated from the originally observed position. In FIG. 6B, the center position of the target 24a is observed at the point D1, the center position of the target 24b is observed at the point D2, and the center position of the target 24c is observed at the point D3. This shows that the center position of the mark 24d is observed at the point D4.

  For the visual target 24, for example, a marker having a shape shown in FIG. That is, the visual target 24 has an orthogonal cross shape so that the two rectangular regions bisect each other. Each rectangular region has a width a, and a portion of length b projects vertically and horizontally with the two rectangular regions orthogonal to each other. Here, as for the width a of the rectangular area, the length of the portion corresponding to the width a of the rectangular area is observed with an odd number of pixels on the image when the target 24 is imaged by the imaging unit 30. It is desirable to set as follows. This is because the center position of the image of the visual target 24 shown on the image needs to be measured as accurately as possible in order to perform accurate calibration.

  Since the center position of the visual target 24 is measured, it is not necessary that all the visual targets 24 to be used have the same size. That is, for example, the target 24 (for example, the target 24c in FIG. 2B) arranged at a position far from the imaging unit 30 is the target 24 (for example, FIG. 2) arranged at a position close to the imaging unit 30. It is desirable that the size be larger than the visual target 24a) in (b).

  The target 24 (24a, 24b, 24c, 24d) is detected from the image I ′ picked up by the image pickup unit 30 by combining existing image processing techniques. Specifically, the edge component is emphasized with respect to the difference area between the area R1 and the area R2 of the captured image I ′, and the area surrounded by the emphasized edge component is detected. Alternatively, the difference area between the area R1 and the area R2 of the image I ′ is binarized with a predetermined threshold value to detect the area. Only regions having a shape similar to the cross shape are selected from the regions thus detected by a method such as template matching. The region selected in this way is a region corresponding to the visual target 24.

  Note that the method for detecting the region corresponding to the visual target 24 is not limited to the method described above, and an appropriate image processing method may be applied in combination as appropriate.

  Next, the center position of each selected area is measured. Various methods are conceivable for measuring the center position of the region corresponding to the detected visual target 24 (24a, 24b, 24c, 24d), any of which may be applied. For example, for each selected region, the area centroid position can be calculated and used as the center position of the region.

  FIG. 7 shows a state in which the result of measuring the center position of the target 24 from the image I and the result of measuring the center position of the target 24 from the image I ′ are combined into one image. .

  That is, in FIG. 7, the target 24a to be observed at the position of the point C1 (C1x, C1y) is actually observed at the position of the point D1 (D1x, D1y), and the positional deviation indicated by the arrow P1 occurs. It is shown that. The magnitude and direction of this positional deviation is represented by (Δx1, Δy1) which is a vector component of the arrow P1. Here, Δx1 is the amount of lateral displacement, and in the case of FIG. 7, Δx1 = D1x−C1x. Δy1 is the amount of positional deviation in the vertical direction, and in the case of FIG. 7, Δy1 = D1y−C1y.

  Similarly, in FIG. 7, the target 24b to be observed at the position of the point C2 (C2x, C2y) is actually observed at the position of the point D2 (D2x, D2y), and the positional deviation indicated by the arrow P2 is shifted. The target 24c to be observed at the position of the point C3 (C3x, C3y) is actually observed at the position of the point D3 (D3x, D3y), and the positional shift indicated by the arrow P3 occurs. The target 24d to be observed at the position of the point C4 (C4x, C4y) is actually observed at the position of the point D4 (D4x, D4y), and the positional deviation indicated by the arrow P4 has occurred. It is shown that.

  The magnitude and direction of the positional deviation at the point D2 are (Δx2, Δy2), the magnitude and direction of the positional deviation at the point D3 are (Δx3, Δy3), and the magnitude and direction of the positional deviation at the point D4 are (Δx4). , Δy4). Each is calculated in the same manner as (Δx1, Δy1) described above.

  At this time, if the magnitudes of the positional deviations of the points D1, D2, D3, and D4 are all smaller than a predetermined value, it is determined that the image that should be observed is observed and calibration is not performed. May be. On the other hand, when the magnitude of the positional deviation of the points D1, D2, D3, and D4 is larger than a predetermined value even at one location, calibration described below is performed.

  Note that the predetermined value of the amount of positional deviation for determining whether or not to perform calibration may be set as appropriate according to the contents of the application executed by the application execution unit 74.

  FIG. 8A is a diagram illustrating a lateral displacement Δx of the observation position of the visual target 24 calculated based on the result of FIG.

  That is, FIG. 8A shows the points C1, C2, C3, C4 (FIG. 7) and the actually observed points D1 obtained as a result of FIG. , D2, D3, D4 (FIG. 7) are shown as lateral displacements Δx.

  Specifically, in FIG. 8A, for each of the observed points D1, D2, D3, and D4, the horizontal coordinate x of each point is taken on the horizontal axis, and the positional deviation amount Δx in the horizontal direction is set to the vertical direction. Plotted for the axis.

  Further, FIG. 8A shows a regression line Lx that minimizes the sum of residuals with each plotted point. Based on the regression line Lx, the amount of lateral displacement Δxi assumed in an arbitrary lateral position Dix can be read as described in FIG.

  FIG. 8B is a diagram illustrating the amount of displacement Δy in the vertical direction of the observation position of the visual target 24 calculated based on the result of FIG.

  That is, FIG. 8B shows points C1, C2, C3, and C4 (FIG. 7) where each target 24 is assumed to be observed and points D1, D2, D3, and D4 actually observed (FIG. 8). 7) shows the positional deviation amount Δy in the vertical direction.

  Specifically, in FIG. 8B, for each of the observed points D1, D2, D3, and D4, the vertical coordinate y of each point is taken on the horizontal axis, and the amount of positional deviation Δy in the vertical direction is expressed in the vertical direction. Plotted for the axis.

  Further, FIG. 8B shows a regression line Ly that minimizes the sum of residuals with each plotted point. Based on this regression line Ly, the vertical displacement amount Δyi assumed at an arbitrary vertical position Diy can be read as shown in FIG.

Although it has been described in the description of FIGS. 8A and 8B that the regression lines Lx and Ly are obtained, a regression curve is generally calculated. This is because the imaging unit 30 includes an optical element having nonlinearity such as a lens, and generally its imaging position has nonlinear characteristics. What kind of curve should be adapted may be determined by the user as appropriate. As described above, the imaging unit 30 is calibrated by the series of operations described above. The obtained data of the regression lines Lx and Ly are stored as correction data in the correction data storage unit 60 (FIG. 3).
(Description of the flow of processing in the first embodiment)

  Next, the overall flow of processing performed in the first embodiment will be described with reference to FIGS. FIG. 9 is a flowchart showing the overall processing flow of the first embodiment, and FIG. 10 is a flowchart showing the calibration flow. First, the flowchart of FIG. 9 will be described.

  (Step S10) It is determined whether or not an ignition switch (not shown in FIG. 3) of the vehicle V is ON. When the ignition switch is ON, the process proceeds to step S12, and otherwise, step S10 is repeated. This is a process performed to determine whether or not the vehicle V is in a travelable state.

  (Step S12) It is determined whether or not the vehicle V is traveling. When the vehicle V is traveling, the process proceeds to step S16, and when the vehicle V is stopped, the process proceeds to step S14.

  (Step S14) The processing of FIG. 10 is activated, and the imaging unit 30 (FIG. 3) is calibrated. Details of the processing will be described later.

  (Step S16) In the application execution unit 74 (FIG. 3), it is determined whether or not the application is ready to be executed. When the application is ready to be executed, the process proceeds to step S18. Otherwise, the process proceeds to step S14.

  (Step S18) The processing of FIG. 10 is activated, and the imaging unit 30 (FIG. 3) is calibrated. Details of the processing will be described later.

  (Step S20) A predetermined application is executed in accordance with an instruction from the application execution unit 74 (FIG. 3). Note that the result of the calibration executed in step S14 or step S18 is reflected in the operation of the application.

  (Step S22) During the execution of the application, the process of FIG. 10 is activated at a predetermined timing, and the imaging unit 30 (FIG. 3) is calibrated. Details of the processing will be described later. Note that the result of the calibration executed in step S22 is sequentially reflected in the operation of the application.

  (Step S24) It is determined whether an ignition switch of the vehicle V (not shown in FIG. 3) is ON. When the ignition switch is ON, the process proceeds to step S26, and otherwise, the process of FIG. 9 is terminated. This is a process performed to determine whether or not the vehicle V has finished driving.

  (Step S26) It is determined whether an instruction to end the execution of the application is issued from the application execution unit 74 (FIG. 3). When an instruction to end the execution of the application is issued, the process of FIG. 9 is ended. Otherwise, the process returns to step S22.

  Next, the flowchart of FIG. 10 will be described.

  (Step S40) It is determined whether or not a calibration start instruction is issued from the calibration start instruction section 72 (FIG. 3). When a calibration start instruction is issued, the process proceeds to step S42, and otherwise, step S40 is repeated.

  (Step S42) A near-infrared image is imaged by the imaging unit 30 (FIG. 3), and is input to the target position measurement unit 48 (FIG. 3) via the near-infrared image selection unit 44 (FIG. 3).

  (Step S44) The target position measurement unit 48 (FIG. 3) detects all the targets 24 (FIG. 3) shown in the image and measures the center position thereof.

  (Step S46) It is determined whether there is correction data already generated in the correction data storage unit 60 (FIG. 3). When there is correction data, the process proceeds to step S48, and when there is no correction data, the process proceeds to step S52.

  (Step S48) The correction data is read from the correction data storage unit 60 (FIG. 3).

  (Step S50) The image correction unit 52 (FIG. 3) corrects the image based on the correction data read out in step S48 on the color image captured by the imaging unit 30 (FIG. 3).

  (Step S52) In the target position comparison unit 50 (FIG. 3), it is determined whether or not the target position measured in step S44 is deviated by a predetermined value or more from the position that should be originally observed. If it is deviated by a predetermined value or more, it is determined that calibration is necessary, and the process proceeds to step S54. Otherwise, it is determined that calibration is not necessary, and the process returns to the main routine (FIG. 9).

  (Step S54) The target position comparison unit 50 (FIG. 3) calculates the amount of displacement Δx, Δy between the target position measured in step S44 and the target position to be originally observed.

  (Step S56) The positional deviation amounts Δx and Δy calculated for all the visual targets 24 (24a, 24b, 24c, 24d) (FIG. 3) are subjected to regression analysis. Then, a regression line or a regression curve is obtained.

  (Step S58) The image correction unit 52 (FIG. 3) corrects the image based on the result of the regression analysis performed in step S56 on the color image captured by the imaging unit 30 (FIG. 3).

  (Step S60) The result of the regression analysis performed in Step S56 is stored in the correction data storage unit 60 (FIG. 3) as correction data, and the process returns to the main routine (FIG. 9).

  As described above, according to the camera calibration device 10 according to the present invention configured as described above, the image sensor 32 configuring the image capturing unit 30 has the near-infrared light receiving element 34 having sensitivity to near-infrared light. (34a, 34b, 34c,...), An image that is observed when a plurality of targets 24 (24a, 24b, 24c, 24d) that transmit or reflect near-infrared light are captured by the imaging unit 30. In I ′, a plurality of visual targets 24 are observed with high contrast. Therefore, the positions of the plurality of targets 24 can be reliably and accurately measured by the target position measuring unit 48. Then, the image correction unit 52 geometrically corrects the positions of the measured visual targets 24. Therefore, it is possible to reliably perform the calibration of the imaging unit 30 without using special illumination or the like and without being affected by ambient brightness.

  Further, according to the camera calibration device 10 according to the present invention configured as described above, a plurality of targets 24 (24a, 24b, 24c, 24d) are included in the image I ′ captured by the imaging unit 30. Since it is arranged so as to appear in an area outside the outer periphery of the area R2, which is an image area used for a specific application, the detection positions of a plurality of targets 24 to be detected for calibration are limited. be able to. Therefore, the imaging unit 30 can be calibrated more reliably.

  And according to the camera calibration device 10 according to the present invention configured as described above, the imaging unit 30 is installed so as to photograph the outside of the vehicle V through the windshield 20 (windshield) of the vehicle V. Since the plurality of targets 24 (24a, 24b, 24c, 24d) are installed on the inside, on the outside surface, or on the inside surface of the windshield 20 of the vehicle V, the calibration of the imaging unit 30 is performed. It is not necessary to prepare a wide space for installing a plurality of visual targets 24 (24a, 24b, 24c, 24d).

  Furthermore, according to the camera calibration device 10 according to the present invention configured as described above, the imaging unit 30 is installed so as to photograph the outside of the vehicle V through the windshield 20 (windshield) of the vehicle V. Since the plurality of targets 24 (24a, 24b, 24c, 24d) are embedded in the windshield 20 of the vehicle V, the calibration of the imaging unit 30 is performed even when the vehicle V is traveling. Can be performed.

  Further, according to the camera calibration device 10 according to the present invention configured as described above, any one of the positions of the plurality of targets 24 (24a, 24b, 24c, 24d) corrected by the image correction unit 52 is selected. When the position of the visual target 24 deviates from the originally observed position by a predetermined value or more, the image correction unit 52 performs geometric correction of the image I ′. The imaging unit 30 can be calibrated without any influence on the detection of the travel lane). Therefore, since the imaging unit 30 can be calibrated at the same time while using the imaging unit 30 for a specific application, the performance degradation of the specific application due to the unintended occurrence of the imaging unit 30 is prevented. can do.

  As described above, in the first embodiment, the example in which the calibration is performed using the four targets 24a, 24b, 24c, and 24d has been described. However, the number of targets used for the calibration is not limited to four. That is, the calibration accuracy can be further improved by performing calibration using a larger number of targets.

  In the first embodiment, the layout for photographing the outside of the vehicle V through the windshield 20 (windshield) of the vehicle V has been described. However, this is not limited to the windshield 20, and the rear glass of the vehicle V is described. In addition, the layout can be similarly applied to a layout in which the outside of the vehicle V is photographed through the side glass.

  As mentioned above, although the Example of this invention was explained in full detail with drawing, since an Example is only an illustration of this invention, this invention is not limited only to the structure of an Example. Of course, changes in design and the like within a range not departing from the gist are included in the present invention.

10 Camera calibration device 20 Windshield (windshield)
22 Transparent film 24, 24a, 24b, 24c, 24d Target 30 Imaging unit 32 Imaging device 34, 34a, 34b, 34c Near-infrared light receiving element 40 Image processor 42 Image input unit 44 Near-infrared image selection unit 46 Color image Selection unit 48 Target position measurement unit 50 Target position comparison unit 52 Image correction unit 60 Correction data storage unit 70 Microcomputer 72 Calibration start instruction unit 74 Application execution unit V Vehicle

Claims (5)

An image sensor including a near-infrared light-receiving element having sensitivity to near-infrared light;
An imaging unit for generating an image by the imaging element;
A plurality of targets that transmit or reflect the near-infrared light; and
A target position measuring unit that measures the position of each of the plurality of targets from among the images including the plurality of targets captured by the imaging unit;
An image correction unit that performs geometric correction of the image based on the positions of the plurality of targets in the image so that the plurality of targets are reflected in positions originally observed. A camera calibration device.   The plurality of targets are arranged so as to appear in an area outside an outer periphery of an image area used for a specific application in an image photographed by the imaging unit. The camera calibration apparatus according to 1.   The imaging unit is installed so as to photograph the outside of the vehicle through a windshield of the vehicle, and the plurality of targets are installed on an outer surface or an inner surface of the windshield. The camera calibration device according to claim 1 or 2.   The image pickup unit is installed so as to photograph the outside of the vehicle through a windshield of the vehicle, and the plurality of targets are embedded in the windshield. The camera calibration apparatus according to claim 2.   When the position of any target among the positions of the plurality of targets corrected by the image correction unit is deviated by a predetermined value or more from the originally observed position, the image correction unit The camera calibration apparatus according to claim 1, wherein a geometric correction is performed.