JP2009267602A - Vehicle periphery monitoring apparatus, and setting correcting method of camera installing position and posture information which is applicable thereto - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide "a vehicle periphery monitoring apparatus and a setting correcting method of camera installing position and posture information" capable of inexpensively displaying a vehicle periphery image excellent in visibility and having little deviation between viewpoint converted images. <P>SOLUTION: An installing position and posture information setting correcting means 51 corrects the setting of an on-vehicle camera installing position and posture information on the basis of a vehicle periphery image 1 subjected to distortion correction processing which includes an image 59 of a first image photographing object. At this time, for the seam portion, an error of a position of a feature point 60 in the image 59 of the first image photographing object is detected, and the proper installing position and posture information is actually measured by correction for eliminating the detected error, and correction by linear interpolation is enough for a region other than the seam portion in the viewpoint converted image 1. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring apparatus and a camera attachment position / posture information setting correction method applied to the vehicle periphery monitoring apparatus, and more particularly, to correct a setting of an in-vehicle camera attachment position / posture information used for generating a vehicle peripheral image. The present invention relates to a vehicle periphery monitoring apparatus suitable for the above and a method for correcting setting of camera mounting position / posture information applied thereto.

  In recent years, with the development of in-vehicle camera technology and image processing technology, as a system that supports safe and smooth driving operation of the host vehicle in parking lots, etc., photographing the surroundings of the host vehicle with a plurality of in-vehicle cameras installed in the host vehicle A vehicle periphery monitoring device is used that generates a vehicle periphery image that is an image as if the periphery of the host vehicle is viewed from above the host vehicle based on the video, and displays the generated vehicle periphery image on a display in the vehicle. It became so.

  FIG. 14 shows a vehicle periphery image 1 displayed in a parking lot as an example of the vehicle periphery image 1 displayed on the display by such a vehicle periphery monitoring device.

  As shown in FIG. 14, the vehicle peripheral image 1 includes a substantially rectangular illustration image 2 indicating the host vehicle, a front viewpoint conversion image 3, a left side viewpoint conversion image 5, and a right side viewpoint conversion surrounding the four sides of the illustration image 2. The image 6 and the rear viewpoint conversion image 7 are formed so as to be connected.

  Here, the front viewpoint conversion image 3 is generated on the basis of a photographed image of a front camera attached to the front part (for example, an emblem part) of the host vehicle as an in-vehicle camera. According to the image 3, it is possible to monitor the front of the host vehicle as viewed from above the host vehicle. The front viewpoint conversion image 3 has a front end part of the front end side of the vehicle peripheral image 1 and a rear end connected to the front end side of the illustration image 2. It has a trapezoidal shape with the front edge being a long upper base and the rear edge being a short bottom.

  Further, the left-side viewpoint converted image 5 is generated based on a photographed image of the left side camera attached to the left side part (for example, the left door mirror part) of the host vehicle as the in-vehicle camera. It is possible to monitor the left side of the host vehicle as viewed from the upper side of the host vehicle by the viewpoint conversion image 5. This left-side viewpoint converted image 5 has its right edge connected to the left edge of the illustration image 2, and at the first joint 8 corresponding to the edge connected to the front edge of the right edge, the front viewpoint converted image. 3 is connected to the left end side. The first joint 8 has a linear shape extending from the left front corner in the illustration image 2 toward the left front. The front end portion of the first joint 8 reaches the front end side of the vehicle peripheral image 1.

  Further, the right-side viewpoint converted image 6 is generated based on a photographed image of a right side camera attached to a right side portion (for example, a right door mirror portion) of the host vehicle as an in-vehicle camera. It is possible to monitor the right side of the host vehicle as viewed from the upper side of the host vehicle by the direction viewpoint conversion image 6. The right-side viewpoint converted image 6 has its left edge connected to the right edge of the illustration image 2 and a front viewpoint converted image at the second joint 10 corresponding to the edge connected to the front edge of the left edge. 3 is connected to the right edge of 3. The second joint 10 has a linear shape extending from the right front corner in the illustration image 2 toward the right front. The front end portion of the second joint 10 reaches the front end side of the vehicle peripheral image 1.

  Furthermore, the rear viewpoint conversion image 7 is generated based on a photographed image of the back camera attached to the rear part (for example, the attaching part of the license plate) of the host vehicle as the in-vehicle camera. The viewpoint conversion image 7 can monitor the rear of the host vehicle as viewed from above the host vehicle. This rear viewpoint converted image 7 has its front end connected to the rear end of the illustration image 2 and left side viewpoint conversion at the third joint 11 corresponding to the end connected to the left end of the front end. It is connected to the rear edge of the image 5. The third joint 11 has a linear shape extending from the left rear corner in the illustration image 2 toward the left rear. Further, the rear end portion of the third joint 11 reaches the left end side of the vehicle peripheral image 1. Further, the rear viewpoint converted image 7 is connected to the rear end portion of the right side viewpoint converted image 6 at a fourth joint 12 corresponding to an end side connected to the right end of the front end side. The fourth joint 12 has a linear shape extending from the right rear corner in the illustration image 2 toward the right rear. Further, the rear end portion of the fourth joint 12 reaches the right end side of the vehicle peripheral image 1.

  Such a vehicle periphery monitoring device that displays the vehicle periphery image 1 uses a wide-angle camera having a wide-angle lens such as a fisheye lens as an in-vehicle camera from the viewpoint of being able to monitor a 360 ° direction around the host vehicle. There were many.

  When the vehicle periphery image 1 is generated using such an in-vehicle camera, as shown in FIG. 15, an image is formed on the imaging surface 14 of the in-vehicle camera such as a CCD (in FIG. 15, the imaging surface of the back camera). Of the captured video 15, only the video in a predetermined usage video area 16 used for generating the vehicle surrounding image 1 is cut out, and the cut-out video is converted into viewpoint-converted images 3, 5, 5 corresponding to the in-vehicle camera. The viewpoint was changed to 6,7. The viewpoint conversion images 3, 5, 6, and 7 generated based on the captured images of the respective in-vehicle cameras are combined (connected) at the same time as the generation to form the vehicle peripheral image 1.

  However, since the video in the use video area 16 has distortion due to the distortion of the lens of the in-vehicle camera, distortion correction for the video in the use video area 16 is performed at the time of viewpoint conversion. It was necessary to eliminate distortion by performing processing.

In this distortion correction processing, internal parameters of the in-vehicle camera are used as distortion correction values. As internal parameters for performing distortion correction processing, for example, image principal points (Cx, Cy) and distortion correction parameters (k ₁ , k ₂ , k ₃ , k ₄ , k ₅ ) as shown in FIG. 16 are used. It was done.

  Here, FIG. 16 shows the relationship between the three-dimensional world coordinate system and the coordinate system (hereinafter referred to as the image coordinate system) of the imaging surface 14 (digital image plane) of the in-vehicle camera in the two-dimensional coordinate system. Yes. As shown in FIG. 16, the origin in the three-dimensional world coordinate system and the origin in the image coordinate system are both located on the Z-axis, and this Z-axis is the optical axis of the fish-eye lens 18 as a lens of the in-vehicle camera. It matches.

  Then, the point P in the three-dimensional space is projected as a point p on the imaging surface 14 through the fisheye lens 18.

  At this time, the image height r (θ) is expressed by the following equation using an internal parameter, an incident angle (angle of view) θ [°] of light from the point P to the Z axis, and a focal length [mm] of the fisheye lens 18. It will be expressed as

r (θ) = 2 ftan (θ / 2)
= K ₁ θ + k ₂ θ ³ + k ₃ θ ⁵ + k ₄ θ ⁷ + k ₅ θ ⁹

  In the vehicle periphery monitoring device, after grasping that a predetermined distortion has occurred in the photographed image due to the image height characteristic related to such an internal parameter, the image in the use video area 16 is obtained by using the internal parameter. The distortion correction processing is performed so that the distortion of the image is eliminated in the state of the viewpoint conversion images 3, 5, 6, and 7 (that is, the vehicle peripheral image 1).

  Furthermore, in the vehicle periphery monitoring device, not only the distortion is removed by such distortion correction processing, but also the vehicle periphery in which the viewpoint conversion images 3, 5, 6, and 7 are connected as continuous images. It was important to display image 1.

  For this reason, in the vehicle periphery monitoring device, when converting the video in the use video area 16 into the viewpoint conversion images 3, 5, 6, and 7, the viewpoint conversion image is obtained by using the external parameters of each in-vehicle camera. The viewpoint conversion is performed so that 3, 5, 6, and 7 are appropriately connected at joints 8, 10, 11, and 12.

  Here, the external parameter is information on the mounting position and mounting posture of the in-vehicle camera in the host vehicle, and as such an external parameter, the X coordinate, the Y coordinate, and the Z coordinate of the position of the in-vehicle camera in the three-dimensional world coordinate system. (Tx, ty, tz) and rotation angles (rx, ry, rz) around the coordinate axes of the X coordinate axis, the Y coordinate axis, and the Z coordinate axis regarding the direction of the optical axis of the in-vehicle camera in the three-dimensional world coordinate system are known. .

JP 2007-158695 A JP 2004-200819 A JP 2001-116515 A

  By the way, the in-vehicle camera is not necessarily attached to the host vehicle at an appropriate attachment position and attachment posture, and some attachment errors such as displacement and rotation may occur.

  When only design values are used as external parameters at the time of viewpoint conversion in spite of such attachment errors, the viewpoint conversion images 3, 5, 6 and 7 have a deviation such as an inclination. As a result, a poor vehicle peripheral image 1 in which the viewpoint conversion images 3, 5, 6, and 7 are not properly connected to each other is generated.

  Therefore, conventionally, in the vehicle periphery monitoring device, for example, external parameters are measured by using a measuring tool in which a regular pattern 21 called a target board 20 as shown in FIG. External parameters are used for generation (viewpoint conversion) of the vehicle periphery image 1.

  In order to actually measure external parameters using such a target board 20, first, as shown in FIG. 18, the host vehicle 23 to which a plurality of in-vehicle cameras are attached is positioned on a flat work surface 24, and The target board 20 is set at a predetermined position corresponding to the center of the viewing angle of each in-vehicle camera on the work surface 24, and then the target board 20 is photographed by the in-vehicle camera.

  By photographing the target board 20, as shown in FIG. 19, an image 20 'of the target board 20 is formed at the center of the used image area 16 of each in-vehicle camera.

  Then, the image formation position of each pattern 21 ′ on the image 20 ′ of the target board 20 is the normal image formation position (the image formation position of the pattern 21 ′ when the current external parameters are assumed to be as designed values. ) Is detected.

  Based on the detection result, an error amount from the design value of the external parameter is calculated, and the calculated error amount is added to the design value to obtain the actual measurement value of the current external parameter. .

  However, in the actual measurement of the external parameters using such a target board 20, since each pattern 21 'on the video 20' of the target board 20 is greatly distorted particularly in a wide angle of view area, a wide use video area. When the viewpoint conversion is performed based on the captured image in the image 16, the viewpoint conversion images 3, 5, 6, and 7 are appropriately connected to each other due to a subtle measurement error particularly in a wide-angle region. There was a gap.

  Therefore, the present invention has been made in view of such problems, and a vehicle periphery monitoring device capable of displaying a vehicle periphery image excellent in visibility with little deviation between viewpoint-converted images at low cost, and the same It is an object of the present invention to provide a setting correction method for camera mounting position / posture information applied to the camera.

  In order to achieve the above-described object, a vehicle periphery monitoring device according to the present invention is provided by a plurality of in-vehicle cameras having a wide-angle field of view mounted on the host vehicle so as to be attached to a predetermined position in the host vehicle. A vehicle periphery monitoring device that generates a vehicle periphery image that is an image obtained by looking down the periphery of the host vehicle from above the host vehicle based on a video image of the periphery of the vehicle, and displays the vehicle periphery image on a display unit. The peripheral monitoring device main body is a video cutout unit that cuts out a video in a predetermined use video area for each vehicle-mounted camera used for generating the vehicle peripheral image from each captured video of the plurality of vehicle-mounted cameras, Predetermined distortion correction for each in-vehicle camera that has been set with respect to the video in the use video area for each in-vehicle camera cut out by this video cutout means Distortion correction means for performing distortion correction processing for each vehicle-mounted camera using the camera, and distortion correction value setting means for setting the distortion correction value for each vehicle-mounted camera used for the distortion correction processing by the distortion correction device, respectively. The area indicated by the image using the distortion correction processing for each in-vehicle camera by the distortion correction means is set to a predetermined area above the vehicle using the set mounting position / posture information of each in-vehicle camera. Viewpoint conversion means for generating the vehicle peripheral image in which the viewpoint conversion images of the respective in-vehicle cameras are connected to each other by performing viewpoint conversion into viewpoint conversion images that are images viewed from the viewpoint of Mounting position for setting the mounting position / posture information of each on-vehicle camera used for the viewpoint conversion by the converting means And a mounting position / posture information setting correction unit capable of correcting the setting of the mounting position / posture information by the mounting position / posture information setting unit. The correcting means is provided under a work environment in which a first object to be photographed having a known shape from which a plurality of feature points arranged in a straight line can be extracted is installed at a predetermined position around the vehicle. Then, based on the captured video of each in-vehicle camera on which the first imaging object is reflected, the viewpoint-converted images for each in-vehicle camera each including the image of the first imaging object are joined together. A feature point extracting means for extracting the feature point from the image of the first object to be photographed after the vehicle periphery image is generated, and any one viewpoint based on the extraction result of the feature point extracting means Strange By comparing the arrangement state of the feature points in the joint portion with another viewpoint conversion image of the replacement image and the regular shape of the first object to be photographed in the joint portion, the joint portion An error detection means for detecting an error in the position of the feature point in the first correction of the first attachment position / posture information corresponding to the joint portion capable of eliminating the error detected by the error detection means A first correction amount calculating means for calculating an amount; and the first correction for each of the pair of joint portions formed so as to provide an interval in a normal arrangement direction of the feature points in the viewpoint conversion image. The feature in the region other than the joint portion sandwiched between the pair of joint portions in the viewpoint conversion image by linear interpolation using the calculation result of the amount calculation means. A second correction amount calculating means for calculating a correction amount of second attachment position / posture information corresponding to a region other than the joint portion capable of eliminating the point position error; and the first attachment. First correction means for correcting information corresponding to the joint portion in the attachment position / posture information currently set by the attachment position / posture information setting means based on a correction amount of position / posture information; Based on the correction amount of the second attachment position / posture information, information corresponding to an area other than the joint portion in the attachment position / posture information currently set by the attachment position / posture information setting unit is corrected. And a second correction means.

  According to such a configuration, the mounting position / posture information setting correcting means corrects the setting of the mounting position information and mounting posture information (hereinafter referred to as mounting position / posture information) of the in-vehicle camera. This can be performed based on the vehicle peripheral image after the distortion correction processing including the image of the shooting target object. At this time, the error of the position of the feature point in the image of the first shooting target is obtained for the joint portion. While measuring the appropriate attachment position / orientation information by correcting the detected error, the area other than the joint part in the viewpoint conversion image can be corrected by linear interpolation. Therefore, it is possible to display a vehicle periphery image with excellent visibility with little shift between viewpoint conversion images at a low cost.

  Further, the first object to be photographed is a grid installed around the flat work surface on which the own vehicle is placed so that a pattern surface thereof is parallel to the work surface. Preferably, the feature points are lattice points in the lattice pattern.

  According to such a configuration, by using a simple grid pattern as the first object to be photographed, feature points (grid points) can be easily and reliably detected on the viewpoint conversion image. Can be further reduced.

  Further, the attachment position / posture information currently set by the attachment position / posture information setting means may be a design value of the attachment position / posture information.

  According to such a configuration, the design value of the mounting position / posture information that does not cause an error from the actual mounting position and mounting position of the in-vehicle camera can be continuously used for viewpoint conversion. Can be further reduced.

  Furthermore, it is preferable that the attachment position / posture information is an external parameter of the in-vehicle camera.

  And according to such a structure, viewpoint conversion can be performed more appropriately by using the external parameter of a vehicle-mounted camera.

  In addition, the image processing apparatus includes distortion correction value setting correction means that can correct the setting of the distortion correction value by the distortion correction value setting means, and the distortion correction value setting correction means includes a second imaging object having a known shape. In the work environment installed around the host vehicle, after the video in the use video area in which the second object to be photographed is cut out from the video captured by the in-vehicle camera, By controlling the distortion correction means, the first distortion correction value that is currently set by the distortion correction value setting means is used as the distortion correction process for setting and correcting the distortion correction value. Distortion correction control means for performing distortion correction processing on the video in the use video area, and on the video obtained by the distortion correction processing for the setting correction work or the viewpoint conversion method for this video By comparing the shape of the second object to be photographed on the image obtained by the viewpoint conversion with the normal shape of the second object to be photographed, by the distortion correction process for the setting correction work From the obtained video or the image obtained by the viewpoint conversion for this video, a distortion region that is a region in the video or the image in which distortion remains without the distortion correction processing being accurately performed, The strain area detecting means for detecting together with the remaining strain amount in the strain area, and an additional correction amount capable of correcting the distortion in the strain area detected by the strain area detecting means as the remaining strain amount. And calculating the second distortion correction value by adding the calculated additional correction amount to the first distortion correction value. By controlling the value calculation means and the distortion correction value setting means, the second distortion correction value calculated by the distortion correction value calculation means is transmitted to the distortion correction value setting means in the use video area. And a new correction value used as the distortion correction value used in the distortion correction processing for a video in a specific video area corresponding to the distortion area, and a video area other than the specific video area in the used video area Comprises a setting control means for continuing to set the first distortion correction value as the distortion correction value used in the distortion correction processing, so that the distortion correction value corresponding to the specific video region is set. The mounting position / posture information setting correction unit is configured to correct only the setting, and the distortion correction value setting correction unit adjusts the distortion correction value by the distortion correction value setting correction unit. It is preferable that the setting of the attachment position / posture information is corrected based on the vehicle peripheral image generated through the distortion correction processing after the setting correction work.

  According to such a configuration, even when an error has occurred in the currently set distortion correction value, the distortion correction value setting correction unit corrects the distortion correction value setting at a low cost and is attached. The position / posture information setting correction means further appropriately corrects the setting of the mounting position / posture information of the in-vehicle camera based on the vehicle peripheral image obtained through the distortion correction processing after the setting correction work of the distortion correction value. be able to.

  Furthermore, it is preferable that the second imaging object is the first imaging object.

  And according to such a structure, since the correction of the setting of the mounting position / posture information of the in-vehicle camera and the correction of the setting of the distortion correction value can be performed using the same photographing object, the cost can be reduced. Further reduction can be achieved.

  Furthermore, it is preferable that the first distortion correction value is a design value of the distortion correction value.

  According to such a configuration, the design value of the distortion correction value can be used effectively, and the cost can be further reduced.

  The distortion correction value is preferably an internal parameter of the in-vehicle camera.

  And according to such a structure, an appropriate distortion correction process can be performed by using the internal parameter of a vehicle-mounted camera as a distortion correction value.

  The camera attachment position / posture information setting correction method according to the present invention includes a plurality of in-vehicle cameras having a wide-angle field of view attached to a predetermined position in the own vehicle, Each of the images in the predetermined use video area for each in-vehicle camera is cut out, and the predetermined distortion correction value for each set in-vehicle camera is set for the cut out video in the use video area for each in-vehicle camera. Each of the in-vehicle cameras used is subjected to distortion correction processing, and the images obtained by the distortion correction processing for each of the in-vehicle cameras are shown by using the set attachment position / posture information of each in-vehicle camera. Each in-vehicle camera is converted into a viewpoint conversion image that is an image as seen from a predetermined viewpoint above the vehicle. Generating a vehicle peripheral image that is an image as if the surroundings of the own vehicle in which the respective viewpoint conversion images are connected to each other are looked down from above the own vehicle, and displaying the generated vehicle peripheral image on the display unit In the vehicle periphery monitoring apparatus, a camera mounting position / posture information setting correction method for correcting the setting of the mounting position / posture information used for the viewpoint conversion, and a plurality of feature points arranged in a straight line can be extracted. A first shooting object having a known shape is set as a work environment installed at a predetermined position around the host vehicle, and the first shooting is performed under the set work environment. A vehicle in which the viewpoint-converted images for the respective in-vehicle cameras each including the image of the first object to be photographed are connected based on the captured images of the in-vehicle cameras on which the object is reflected After generating the side image, the feature point is extracted from the image of the first object to be photographed, and based on the extraction result of the feature point, another viewpoint converted image for any one viewpoint converted image and An error in the position of the feature point in the joint portion is detected by comparing the arrangement state of the feature points in the joint portion with the regular shape of the first object to be photographed in the joint portion. Then, a correction amount of the first attachment position / posture information corresponding to the joint portion capable of eliminating the detected error is calculated, and a normal arrangement direction of the feature points in the viewpoint conversion image In the viewpoint conversion image, linear interpolation using a calculation result of the first attachment position / posture information for each of the pair of joint portions formed so as to be spaced apart from each other is used. Second attachment position / posture information corresponding to a region other than the joint portion that can eliminate an error in the position of the feature point in a region other than the joint portion sandwiched between the pair of joint portions. And correcting the information corresponding to the joint portion in the currently set attachment position / posture information based on the correction amount of the first attachment position / posture information, On the basis of the correction amount of the attachment position / posture information, the information corresponding to the region other than the joint portion in the currently set attachment position / posture information is corrected.

  And according to such a method, the setting of the mounting position / posture information of the in-vehicle camera can be corrected based on the vehicle peripheral image after the distortion correction processing including the image of the first photographing object. In this case, an area in the viewpoint conversion image in which the error of the position of the feature point in the image of the first object to be photographed is detected, and the appropriate attachment position / orientation information is actually measured by the correction for eliminating the detected error. Can be limited to the joint portion, and the region other than the joint portion sandwiched by the joint portion in the viewpoint conversion image can be corrected by linear interpolation. It is possible to display a vehicle peripheral image with little deviation and excellent visibility at a low cost.

  Further, as the first object to be photographed, a lattice pattern is provided around the vehicle on a flat work surface on which the vehicle is placed so that a pattern surface thereof is parallel to the work surface. In addition, it is preferable to use lattice points in the lattice pattern as the feature points.

  And according to such a method, since a feature point can be easily and reliably detected on a viewpoint conversion image by using a simple lattice pattern as the first object to be photographed, the cost can be further reduced. be able to.

  Furthermore, a design value of attachment position / posture information may be used as the currently set attachment position / posture information.

  According to such a method, the design value of the mounting position / posture information that does not cause an error from the actual mounting position and mounting posture of the in-vehicle camera can be continuously used for viewpoint conversion. Can be further reduced.

  Furthermore, it is preferable to use external parameters of the in-vehicle camera as the attachment position / posture information.

  And according to such a method, viewpoint conversion can be performed more appropriately by using the external parameter of the vehicle-mounted camera.

  In addition, the second object to be photographed is reflected from a photographed image of the in-vehicle camera under a work environment in which a second object to be photographed having a known shape is installed around the host vehicle. After the video in the used video area is cut out, distortion correction for the video in the used video area using the currently set first distortion correction value is performed as distortion correction processing for setting and correcting the distortion correction value. And the shape of the second object to be photographed on the image obtained by the distortion correction processing for the setting correction work or on the image obtained by the viewpoint conversion for the image, and the second photographing By comparing with the normal shape of the object, the image obtained by the distortion correction processing for the setting correction work, or the image obtained by the viewpoint conversion for this image, A distortion region that is an area in the video or the image in which distortion remains without being accurately corrected only with the remaining amount of distortion in the distortion area, and the detected distortion area An additional correction amount capable of correcting the distortion is obtained based on the residual amount of distortion, and the obtained additional correction amount is added to the first distortion correction value to obtain a second correction amount. A distortion correction value is calculated, and the calculated second distortion correction value is used in the distortion correction processing for the image in a specific video area corresponding to the distortion area of the used video area. As a new value, the first distortion correction value is continuously used for the distortion correction processing for the video area other than the specific video area in the use video area. Only the distortion correction value setting corresponding to the specific video area is corrected, and the vehicle peripheral image generated through the distortion correction processing after the correction is performed is continued. Based on this, it is preferable to correct the setting of the attachment position / posture information.

  According to such a method, even when an error has occurred in the currently set distortion correction value, the distortion correction value setting is corrected at low cost, and the distortion correction value setting is corrected. Based on the vehicle periphery image obtained through the distortion correction processing in, the setting of the mounting position / posture information of the in-vehicle camera can be corrected more appropriately.

  Furthermore, it is preferable to use the first imaging object as the second imaging object.

  And according to such a method, since the correction of the setting of the mounting position / posture information of the in-vehicle camera and the correction of the setting of the distortion correction value can be performed using the same photographing object, the cost can be reduced. Further reduction can be achieved.

  Furthermore, it is preferable to use a design value of the distortion correction value as the first distortion correction value.

  According to such a method, the design value of the distortion correction value can be used effectively, and the cost can be further reduced.

  Moreover, it is preferable to use an internal parameter of the in-vehicle camera as the distortion correction value.

  And according to such a method, an appropriate distortion correction process can be performed by using the internal parameter of a vehicle-mounted camera as a distortion correction value.

  ADVANTAGE OF THE INVENTION According to this invention, the vehicle periphery image excellent in the visibility with few shift | offset | differences of viewpoint conversion images can be displayed at low cost.

  Hereinafter, an embodiment of a vehicle periphery monitoring device according to the present invention will be described with reference to FIGS. 1 to 11.

  Note that portions having the same or similar basic configuration as those in the related art will be described using the same reference numerals.

  As shown in FIG. 1, the vehicle periphery monitoring apparatus (vehicle periphery monitoring apparatus main body) 32 in this embodiment has four vehicle-mounted cameras 34, 35, 36, and 37 mounted on the host vehicle. Each of the in-vehicle cameras 34, 35, 36, and 37 is a camera having a wide-angle field of view with a wide-angle lens such as a fisheye lens, and a solid-state image sensor (imaging surface 14) such as a CCD or CMOS. It is said to be a digital camera.

  One of these in-vehicle cameras 34, 35, 36, and 37 is a front camera 34 attached to the front portion of the host vehicle. The front camera 34 is located in front of the host vehicle around the host vehicle. A predetermined image area centered on the camera is imaged. One of the other three in-vehicle cameras 35, 36, and 37 is a left side camera 35 attached to the left side of the own vehicle. The left side camera 35 is a self-portrait around the own vehicle. A predetermined shooting area centered on the left side of the vehicle is shot. One of the other two in-vehicle cameras 36 and 37 is a right side camera 36 attached to the right side of the own vehicle, and this right side camera 36 is a vehicle in the vicinity of the own vehicle. A predetermined image area centered on the right side is imaged. The remaining one in-vehicle camera 37 is a back camera 37 attached to the rear part of the own vehicle, and this back camera 37 has a predetermined shooting area around the back of the own vehicle around the own vehicle. It is supposed to shoot.

  A camera video acquisition unit 38 is connected to each of the in-vehicle cameras 34, 35, 36, 37, and the camera video acquisition unit 38 can acquire a captured video from each of the in-vehicle cameras 34, 35, 36, 37. ing.

  The camera video acquisition unit 38 is connected with a video cutout unit 40 as a video cutout unit, and the video cutout unit 40 captures images of the in-vehicle cameras 34, 35, 36, and 37 from the camera video acquisition unit 38. Video is input. The video cutout unit 40 then uses the in-vehicle cameras 34, 35, and 36 used to generate the vehicle peripheral image 1 (see FIG. 14) from each of the input video images of the in-vehicle cameras 34, 35, 36, and 37. , 37, the video in the predetermined use video area 16 is cut out. Note that the video in the usage video area 16 in the video captured by the front camera 34 is used to generate the front viewpoint converted image 3 in the vehicle peripheral image 1. In addition, the video in the use video area 16 in the video shot by the left side camera 35 is used to generate the left-side viewpoint conversion image 5 in the vehicle peripheral image 1. Further, the video in the usage video area 16 in the video shot by the right side camera 36 is used to generate the right-side viewpoint converted image 6 in the vehicle peripheral image 1. Furthermore, the video in the usage video area 16 in the video captured by the back camera 37 is used to generate the rear viewpoint converted image 7 in the vehicle peripheral image 1.

  A distortion correction unit 42 serving as a distortion correction unit is connected to the video cutout unit 40, and each of the in-vehicle cameras 34, 35, 36, and 37 cut out by the video cutout unit 40 is connected to the distortion correction unit 42. The video in the corresponding use video area 16 is input. Then, the distortion correction unit 42 sets the distortion set in the distortion correction unit 42 for the video in the use video area 16 for each of the in-vehicle cameras 34, 35, 36, and 37 input from the video cutout unit 40. Distortion correction processing using internal parameters for each of the in-vehicle cameras 34, 35, 36, and 37 as correction values is performed. That is, for the video in the use video area 16 corresponding to any one on-vehicle camera 34, 35, 36, 37 (the clipped video), the one set in the distortion correction unit 42. Distortion correction processing using internal parameters of the in-vehicle cameras 34, 35, 36, and 37 is performed. In this distortion correction processing, a so-called relationship in which coordinates (XY coordinates) between an arbitrary pixel in an image in the use video area 16 before the distortion correction processing and a pixel corresponding to this pixel after the distortion correction processing is described. You may make it carry out by using a mapping table. In this case, the correspondence relationship of the coordinates in the mapping table is defined using internal parameters set in the distortion correction unit 42.

  An internal parameter setting unit 43 as a distortion correction value setting unit is connected to the distortion correction unit 42, and the internal parameter setting unit 43 is connected to each in-vehicle camera used for distortion correction processing with respect to the distortion correction unit 42. Internal parameters for each of 34, 35, 36, and 37 are set.

  An internal parameter design value storage unit 44 is connected to the internal parameter setting unit 43 and the in-vehicle cameras 34, 35, 36, and 37. The internal parameter design value storage unit 44 includes the in-vehicle cameras 34, 35, The design values of the internal parameters of the in-vehicle cameras 34, 35, 36, 37 acquired from 36, 37 are stored.

  The internal parameter setting unit 43 performs distortion correction on the design value of the internal parameter for each of the in-vehicle cameras 34, 35, 36, 37 stored in the internal parameter design value storage unit 44 at the beginning of manufacture of the vehicle periphery monitoring device 32. Each is set in the section 42.

  It should be noted that the internal parameters set in the distortion correction unit 42 by the internal parameter setting unit 43 can be changed by an internal parameter setting / correcting operation described later.

  A viewpoint conversion unit 46 as a viewpoint conversion unit is connected to the distortion correction unit 42, and internal parameters for the respective in-vehicle cameras 34, 35, 36, and 37 by the distortion correction unit 42 are connected to the viewpoint conversion unit 46. Images respectively obtained by the used distortion correction processing are input. Then, the viewpoint conversion unit 46 uses each parameter input from the distortion correction unit 42 as an external parameter as attachment position / posture information of each on-vehicle camera 34, 35, 36, 37 set in the viewpoint conversion unit 46. By using, the viewpoint of the area indicated by each video is converted into an image as seen from a predetermined viewpoint above the host vehicle.

  As a result, the image after distortion correction processing corresponding to the front camera 34 is viewpoint-converted to the front viewpoint conversion image 3. In addition, the image after distortion correction processing corresponding to the left side camera 35 is subjected to viewpoint conversion into the left side viewpoint conversion image 5. Further, the video corresponding to the right side camera 36 is subjected to viewpoint conversion into the right side viewpoint converted image 6. Furthermore, the video corresponding to the back camera 37 is subjected to viewpoint conversion into the rear viewpoint conversion image 7. In this way, the viewpoint images 3, 5, 6, and 7 for each of the in-vehicle cameras 34, 35, 36, and 37 obtained by the viewpoint conversion by the viewpoint conversion unit 46 are combined (connected) with the viewpoint conversion. In addition, the vehicle peripheral image 1 is formed by being combined with the illustration image 2 uniquely generated by the viewpoint conversion unit 46.

  This viewpoint conversion is performed by using a so-called mapping table in which a correspondence relationship between an arbitrary pixel in the video before the viewpoint conversion and a pixel corresponding to the pixel after the viewpoint conversion (XY coordinate) is described. May be.

  An external parameter setting unit 48 as an attachment position / posture information setting unit is connected to the viewpoint conversion unit 46, and the external parameter setting unit 48 is connected to the viewpoint conversion unit 46 for each in-vehicle used for viewpoint conversion. External parameters for each of the cameras 34, 35, 36, and 37 are set.

  An external parameter design value storage unit 50 is connected to the external parameter setting unit 48 and each of the in-vehicle cameras 34, 35, 36, and 37. The design values of the external parameters of the in-vehicle cameras 34, 35, 36, 37 acquired from 36, 37 are stored.

  The external parameter setting unit 48 converts the design values of the external parameters for the respective in-vehicle cameras 34, 35, 36, and 37 stored in the external parameter design value storage unit 50 into viewpoints at the beginning of manufacture of the vehicle periphery monitoring device 32. Each is set in the section 46.

  The external parameters set in the viewpoint conversion unit 46 by the external parameter setting unit 48 can be changed by correcting the external parameter settings described later.

  A display 47 as a display unit is connected to the viewpoint conversion unit 46, and the vehicle peripheral image 1 generated by the viewpoint conversion by the viewpoint conversion unit 46 is displayed on the display 47.

  The vehicle periphery monitoring device 32 in this embodiment includes an external parameter setting correction unit 51 as an attachment position / posture information setting correction unit. The external parameter setting correction unit 51 is a predetermined type as shown in FIG. The external parameter setting state by the external parameter setting unit 48 (that is, the external parameter set in the viewpoint conversion unit 46) can be corrected under the above work environment.

  Here, the work environment shown in FIG. 2 is installed such that a lattice pattern 53 as a first object to be photographed having a known shape surrounds the host vehicle at a predetermined position around the host vehicle. Working environment. As shown in FIG. 2, the lattice pattern 53 is installed around the own vehicle 54 on the flat work surface 55 on which the own vehicle 54 is placed so that the pattern surface is parallel to the work surface 55. ing. More specifically, the lattice pattern 53 in FIG. 2 is formed on the work surface 55 with a plurality of vertical line patterns 57 elongated in the vertical direction in FIG. 2 at equal intervals in the horizontal direction, and 2, a plurality of horizontal line patterns 58 that are long in the horizontal direction are formed at equal intervals in the vertical direction so as to intersect the vertical line pattern 57 at right angles. It is formed in a square. Each of the line patterns 57 and 58 may be formed on the work surface 55 by painting or pasting. The own vehicle 54 is installed such that its full length direction is parallel to the longitudinal direction of the vertical line pattern 57.

  The external parameter setting correction unit 51 will be described in further detail. As shown in FIG. 1, the external parameter setting correction unit 51 has a feature point extraction unit 52 as a feature point extraction unit. A viewpoint conversion unit 46 is connected to 52.

  As shown in FIG. 3, the feature point extraction unit 52 of each of the in-vehicle cameras 34, 35, 36, and 37 in which the grid pattern 53 is reflected in the use video area 16 under the work environment shown in FIG. 4. Vehicle periphery in which viewpoint converted images 3, 5, 6, and 7 each including an image of a grid pattern 53 (hereinafter referred to as a grid pattern image 59) as shown in FIG. When the image 1 is generated, the vehicle peripheral image 1 is acquired from the viewpoint conversion unit 46.

  Note that the vehicle peripheral image 1 shown in FIG. 4 has no error in the internal parameters currently set by the internal parameter setting unit 43 and the distortion is completely corrected by the distortion correction processing by the distortion correction unit 42. Since there is an error in the external parameter currently set by the setting unit 48, it is assumed that there is a connection shift due to an inclination or the like between the viewpoint converted images 3, 5, 6, and 7.

  Then, the feature point extraction unit 52 extracts a grid point 60 in the grid pattern image 59 as a feature point from the vehicle peripheral image 1 including the grid pattern image 59 acquired from the viewpoint conversion unit 46, as shown in FIG. It is supposed to be. The grid point 60 is a point corresponding to the intersection of the vertical line pattern 57 and the horizontal line pattern 58 in the grid pattern 53 shown in FIG. In FIG. 5, one grid point 60 </ b> A shown at substantially the center of each viewpoint conversion image 3, 5, 6, 7 is a grid point of each viewpoint conversion image 3, 5, 6, 7. The grid point 60A is specially recognized as a reference point for grasping the coordinates of the point 60, and the grid point 60A is set in a color different from that of the other grid points 60. It is set so that one or more points are detected. Each lattice point 60A is set at a known position, and each lattice belonging to each joint portion 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b, which will be described later, is based on these lattice points 60A. Each position of the point 60 can be grasped.

  In FIG. 5, the normal arrangement direction of the lattice points 60 is two directions, that is, the vertical direction and the horizontal direction in FIG. 5, but the arrangement state of each lattice point 60 shown in FIG. It is in a state having an inclination from the arrangement direction.

  In addition to the above configuration, in the present embodiment, the external parameter setting correction unit 51 includes an error detection unit 62 as error detection means, as shown in FIG. The feature point extraction unit 52 is connected. The error detector 62 is connected to a lattice pattern database 63 that stores data relating to the shape and coordinates of the lattice pattern 53.

  When the grid point 60 is extracted by the feature point extraction unit 52, the error detection unit 62, based on this extraction result and the data stored in the grid pattern database 63, selects one arbitrary viewpoint conversion image 3, The arrangement state of the grid points 60 in the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, and 7b with the other viewpoint conversion images 3, 5, 6, and 7 for 5, 6, and 7, and the joints The regular shape of the lattice pattern 53 in the portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b (that is, the regular shape of the lattice pattern image 59) is compared.

  Here, the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, and 7b are the other viewpoint converted images 3, 5, 6, and 7 in one viewpoint converted image 3, 5, 6, and 7. The part (edge part) of the predetermined range which continues to the joints 8, 10, 11, and 12 with.

  For example, as shown in FIG. 5, in the front viewpoint converted image 3, a part of the predetermined range (left edge) that is continuous with the first joint 8 is a joint part 3 a with the left side viewpoint converted image 5. In addition, a predetermined range portion (right end edge portion) that is continuous with the second joint 10 is a joint portion 3 b with the right-side viewpoint converted image 6. The pair of joint portions 3a and 3b in the front viewpoint converted image 3 are formed with a gap in the horizontal direction in FIG. 5 as the normal alignment direction of the lattice points 60.

  Further, in the left-side viewpoint converted image 5, a predetermined range portion (edge portion) continuous with the first joint 8 is a joint portion 5 a with the front viewpoint converted image 3, and the third joint A portion (rear edge portion) in a predetermined range connected to the eye 11 is a joint portion 5 b with the rear viewpoint converted image 7. The pair of joint portions 5a and 5b in the left-side viewpoint converted image 5 are formed at intervals in the vertical direction in FIG. 5 as the normal alignment direction of the lattice points 60.

  Further, in the right-side viewpoint converted image 6, a portion (edge portion) in a predetermined range connected to the second joint 10 is a joint portion 6 a with the front viewpoint converted image 3, and the fourth joint A portion (rear edge portion) in a predetermined range connected to the eye 12 is a joint portion 6 b with the rear viewpoint converted image 7. The pair of joint portions 6a and 6b in the right-side viewpoint converted image 6 are formed with a gap in the vertical direction in FIG. 5 as the normal alignment direction of the lattice points 60.

  Furthermore, in the rear viewpoint converted image 7, a predetermined range portion (edge portion) continuous with the third joint 11 is a joint portion 7 a with the left-side viewpoint converted image 5, A portion (edge portion) in a predetermined range connected to the joint 12 is a joint portion 7 b with the right-side viewpoint converted image 6. The pair of joint portions 7a and 7b in the rear viewpoint converted image 7 are formed with a gap in the horizontal direction in FIG. 5 as the normal alignment direction of the lattice points 60.

  Now, the error detector 62 determines the joint portions 3a, 3b, 5a, 5b, 6a based on the comparison result for the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b. , 6b, 7a, 7b, the error of the position of the grid point 60 is detected.

  Here, FIG. 6 shows a joint portion 3a with the left-side viewpoint converted image 5 in the front viewpoint converted image 3 and a joint portion 5a with the front viewpoint converted image 3 in the left-side viewpoint converted image 5. A comparison between the arrangement state of the lattice points 60 and the regular shape of the lattice pattern 53 (lattice pattern image 59) and an error detection state are shown.

  In FIG. 6, the position error of the grid point 60 is calculated for each grid point 60 in the joint portions 3a and 5a. The unit of error magnitude may be a pixel.

  In addition to the above configuration, in the present embodiment, the external parameter setting correction unit 51 further includes a first correction amount calculation unit 64 as a first correction amount calculation unit, as shown in FIG. An error detection unit 62 is connected to the first correction amount calculation unit 64.

  The first correction amount calculation unit 64 acquires the detection result of the position error of the grid point 60 by the error detection unit 62. Then, based on the detection result of the error detector 62, the first correction amount calculator 64 eliminates the error of the position of the grid point 60 in the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b. The first correction amount is calculated as the correction amount of the first attachment position / posture information corresponding to the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b that can be performed. ing.

  In addition to the above configuration, in the present embodiment, the external parameter setting correction unit 51 further includes a second correction amount calculation unit 65 serving as a second correction amount calculation unit, and the second correction amount calculation unit. A first correction amount calculation unit 64 is connected to 65.

  The second correction amount calculation unit 65 acquires the calculation result of the first correction amount by the first correction amount calculation unit 64.

  Then, the second correction amount calculation unit 65 includes a pair of joint portions 3 a, 3 b, 5 a, and a pair of joint portions 3 a, 3 b, 5 a, The second correction amount as the correction amount of the second attachment position / posture information is calculated using the calculation result of the first correction amount calculation unit 64 for each of 5b, 6a, 6b, 7a, 7b. It has become.

  This second correction amount is other than the joint portion sandwiched between the pair of joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b in the viewpoint conversion images 3, 5, 6, 7. The correction amount of the second attachment position / posture information corresponding to the non-joint region that can eliminate the error in the position of the grid point 60 in the region (hereinafter, referred to as a non-joint region).

  This second correction amount is obtained by linear interpolation using the calculation result of the first correction amount calculation unit 64 for each of the pair of joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b. It is calculated by.

  As a specific example, as illustrated in FIG. 7, the first correction amount calculation unit 64 causes a pair of joint portions 5 a of the left-side viewpoint converted image 5 sandwiching the non-joint region 5 c in the left-side viewpoint converted image 5. , 5b, the first correction amount is calculated.

Here, as shown in FIG. 7, when attention is paid to the leftmost grid line 60 arranged in a straight line on the alternate long and short dash line in FIG. As the first correction amount corresponding to the first grid point 60 (P ₁ ) in the grid point 60 row located at the joint portion 5 a with the forward viewpoint converted image 3, the first grid point 60 (P a correction amount that can be modified by [Delta] x ₁ the position in the + x direction in FIG. 7 in order to eliminate the error of ₁₎ is calculated.

At this time, the first correction amount corresponding to the _second grid point 60 (P ₂ ) in the grid point 60 row located at the joint portion 5b between the left-side viewpoint converted image 5 and the rear viewpoint converted image 7 is provided. Assuming that a correction amount is calculated that can correct the position of the _second grid point 60 (P ₂ ) by Δx ₂ in the −x direction in FIG. 7 in order to eliminate the error.

In such a case, the first grid point 60 (P ₁ ) and the second grid point 60 (P ₂ ) are obtained by linear interpolation with respect to the non-joint region 5 c by the second correction amount calculation unit 65. A second correction amount that can correct the position of the third grid point 60 (P ₃ ) located at the midpoint between the two points by (Δx ₁ −Δx ₂ ) / 2 may be calculated. . Such linear interpolation may be effective even when the correction amount of one of the pair of joint portions 5a and 5b is 0 (that is, there is no error).

  Note that the reason for calculating the correction amount of the non-joint region by such linear interpolation is that the viewpoint conversion images 3, 5, 6, and 7 in which the distortion correction processing has been appropriately performed are linear due to an error of an external parameter. It can be considered that only a certain misalignment remains, and this misalignment is regarded as a normal arrangement direction (for example, the direction of the dashed-dotted line in FIG. 7) of 60 rows of lattice points arranged in a straight line. This is because it can be grasped as an inclination with respect to the vertical direction in FIG.

  In addition to the above-described configuration, in the present embodiment, the external parameter setting correction unit 51 includes a first correction unit 67 as a first correction unit as shown in FIG. A first correction amount calculation unit 64 and an external parameter setting unit 48 are connected to the unit 67.

  The first correction unit 67 acquires the first correction amount calculated by the first correction amount calculation unit 64 from the first correction amount calculation unit 64. Then, the first correction unit 67, based on the first correction amount acquired from the first correction amount calculation unit 64, the joint portion 3a in FIG. 5 in the external parameter currently set by the external parameter setting unit 48. , 3b, 5a, 5b, 6a, 6b, 7a, 7b, external parameters corresponding to the external parameters are corrected.

  This correction is the first for the external parameters (for example, design values) corresponding to the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, and 7b in the currently set external parameters (for example, design values). The correction amount may be added.

  In addition to the above configuration, in the present embodiment, the external parameter setting correction unit 51 includes a second correction unit 68 as a second correction unit. The second correction unit 68 includes a second correction unit 68. A correction amount calculation unit 65 and an external parameter setting unit 48 are connected.

  The second correction unit 68 acquires the second correction amount calculated by the second correction amount calculation unit 65 from the second correction amount calculation unit 65. Then, based on the second correction amount acquired from the second correction amount calculation unit 65, the second correction unit 68 uses a non-joint region (5c, etc.) in the external parameter currently set by the external parameter setting unit 48. ) Is corrected. This correction may be addition of a second correction amount to an external parameter (for example, a design value) corresponding to a non-joint region in the currently set external parameter (for example, a design value).

  According to such a configuration, the external parameter setting correction unit 51 can correct the setting of the external parameter based on the vehicle peripheral image 1 after the distortion correction processing including the lattice pattern image 59. At the same time, for the joint portions 8, 10, 11, and 12, an error in the position of the lattice point 60 is detected, and an appropriate external parameter is measured by correction to eliminate the detected error, while the joint portion is measured. With respect to the sandwiched non-joint region, the external parameter can be estimated only by correction by linear interpolation. As a result, it is possible to display the vehicle peripheral image 1 excellent in visibility with little deviation between the viewpoint conversion images 3, 5, 6, and 7 at low cost.

  In addition to the above configuration, the vehicle periphery monitoring device 32 in the present embodiment further includes an internal parameter setting correction unit 70 as a distortion correction value setting correction unit, as shown in FIG. The unit 70 sets the internal parameters set by the internal parameter setting unit 43 (that is, the distortion correction unit 42) under the same working environment as that of FIG. 2 in which the above-described lattice pattern 53 as the second imaging target is installed. (Internal parameters set to) can be modified.

  However, in the present embodiment, the correction of the internal parameter setting state needs to be already completed in the initial stage of the correction of the external parameter setting state by the external parameter setting correction unit 51 described above.

  The internal parameter setting correction unit 70 will be described in detail. As shown in FIG. 1, the internal parameter setting correction unit 70 includes a distortion correction control unit 71 as a distortion correction control unit. Are connected to the video cutout unit 40 and the distortion correction unit 42, respectively.

  Under the work environment shown in FIG. 2, the distortion correction control unit 71 uses the video 15 in the use video area 16 in which the grid pattern 53 is reflected from the video shots of the four in-vehicle cameras 34, 35, 36, and 37. It is detected from the processing result of the video cutout unit 40 that each (see FIG. 3) has been cut out.

  The distortion correction control unit 71 controls the distortion correction unit 42 after a captured image (hereinafter referred to as a lattice pattern image) in the use image area 16 in which the lattice pattern 53 is reflected is cut out. Thus, the distortion correction unit 42 is caused to perform distortion correction processing for internal parameter setting correction work.

  The distortion correction processing for setting correction work uses the internal parameters (that is, the first distortion correction value) for each of the in-vehicle cameras 34, 35, 36, and 37 that are currently set by the internal parameter setting unit 43. Distortion correction processing is performed for each grid pattern image corresponding to each of the in-vehicle cameras 34, 35, 36, and 37.

  Here, as described above, at the beginning of manufacture of the vehicle periphery monitoring device 32, the internal parameter setting unit 43 sets the design value of the internal parameter in the distortion correction unit 42.

  Therefore, for example, the distortion correction processing for setting correction work that is performed for the first time after the manufacture of the vehicle periphery monitoring device 32 is performed using the design values of the internal parameters of the in-vehicle cameras 34, 35, 36, and 37.

  FIG. 8 schematically shows a distortion correction process for setting correction work on a lattice pattern image. However, since the lattice pattern image after the distortion correction processing in FIG. 8 has an error in the internal parameter (for example, design value) currently set by the internal parameter setting unit 43, the distortion is not completely corrected by the distortion correction processing. It is not a video.

  In the present embodiment, a video obtained by such distortion correction processing for setting correction work is subjected to viewpoint conversion by the viewpoint conversion unit 46, whereby a vehicle including a grid pattern image 59 as shown in FIG. A peripheral image 1 is formed.

  Note that the vehicle peripheral image 1 in FIG. 9 is an image in which distortion is not completely removed because there is an error in the internal parameters currently set by the internal parameter setting unit 43, as in FIG. This can also be seen from the fact that the lattice pattern image 59 is distorted.

  Furthermore, the vehicle peripheral image 1 in FIG. 9 is an image in which the viewpoint conversion images 3, 5, 6, and 7 are not appropriately connected because there is an error in the external parameters currently set by the external parameter setting unit 48. ing.

  In addition to the above configuration, in the present embodiment, the internal parameter setting correction unit 70 includes a distortion region detection unit 72 as a distortion region detection unit, as shown in FIG. A viewpoint conversion unit 46 is connected to 72.

  The distortion region detection unit 72 includes an image obtained by viewpoint conversion by the viewpoint conversion unit 46 with respect to a video obtained by distortion correction processing for setting correction work, that is, a vehicle peripheral image 1 including a lattice pattern image 59. (May be viewpoint-converted images 3, 5, 6, and 7).

  In addition, the lattice pattern database 63 described above is connected to the strain region detection unit 72.

  When the vehicle peripheral image 1 including the lattice pattern image 59 is input from the viewpoint conversion unit 46 to the distortion region detection unit 72, the distortion region detection unit 72 generates the lattice pattern 53 (that is, the lattice pattern image 59) on the vehicle peripheral image 1. The shape is compared with the regular shape of the lattice pattern 53 indicated by the data in the lattice pattern database 63. Then, by performing the comparison, the distortion region detection unit 72 performs the comparison from the vehicle peripheral image 1 including the lattice pattern image 59, so that the distortion correction processing is not accurately performed and the vehicle peripheral image 1 in which distortion remains. The distortion area which is the inner area is detected together with the remaining amount of distortion in the distortion area. As a unit of the remaining amount, for example, a pixel may be used.

  At this time, as shown in FIG. 10, the strain area detection unit 72 extracts and extracts the above-described grid points 60 as feature points from the vehicle surrounding image 1 including the grid pattern image 59 by image recognition. Based on the arrangement state of the lattice points 60, the shape of the lattice pattern image 59 may be compared with the regular shape of the lattice pattern 53.

  In FIG. 10, a region on the vehicle peripheral image 1 (viewpoint conversion images 3, 5, 6, and 7) surrounded by a broken-line frame is a distortion region 75.

  In addition to the above configuration, in the present embodiment, the internal parameter setting correction unit 70 has an internal parameter calculation unit 76 as a distortion correction value calculation unit as shown in FIG. A distortion region detection unit 72 is connected to the unit 76.

  The internal parameter calculation unit 76 is input with the detection result of the distortion region detection unit 72. Then, the internal parameter calculation unit 76 corrects distortion in the distortion region 75 detected by the distortion region detection unit 72 as an additional correction amount based on the input detection result (compensates for an insufficient amount of distortion correction processing). A distortion component correction amount (for example, an amount by which the image principal point or internal parameter should be changed from a current value) is obtained. In determining the correction amount of the distortion component, the remaining distortion amount detected by the distortion region detection unit 72 is used. Then, the internal parameter calculation unit 76 obtains the distortion component correction amount in this way, and then adds the obtained distortion component correction amount to the internal parameter currently set by the internal parameter setting unit 43. Thus, an actually measured value of the internal parameter as the second distortion correction value is calculated. The actually measured value of the internal parameter becomes a new internal parameter related to a specific video area corresponding to the distortion area 75 in the use video area 16 before the distortion correction processing for setting correction work.

  In addition to the above configuration, in the present embodiment, the internal parameter setting correction unit 70 includes an actual value storage unit 77 as shown in FIG. A calculation unit 76 is connected. The actual measurement value storage unit 77 stores the actual measurement value of the internal parameter calculated by the internal parameter calculation unit 76.

  In addition to the above configuration, in the present embodiment, the internal parameter setting correction unit 70 further includes an internal parameter setting control unit 78 as a distortion correction value setting control unit. The internal parameter setting control unit 78 includes The internal parameter setting unit 43 and the internal parameter calculation unit 76 are connected.

  When the measured value of the internal parameter is calculated by the internal parameter calculation unit 76, the internal parameter setting control unit 78 controls the internal parameter setting unit 43 to store the internal parameter setting unit 43 in the measured value storage unit 77. The measured value of the internal parameter is acquired. Then, the internal parameter setting control unit 78 causes the internal parameter setting unit 43 to send the measured value of the internal parameter acquired from the measured value storage unit 77 to a specific video area corresponding to the distortion area 75 in the use video area 16. It is newly set as an internal parameter used for distortion correction processing for the video inside. At this time, the internal parameter setting control unit 78 continues the currently set internal parameter as an internal parameter used for distortion correction processing for the video area other than the specific video area in the use video area 16. It keeps setting.

  With this configuration, the internal parameter setting correction unit 70 can correct only the internal parameter setting corresponding to the specific video area.

  Then, when the vehicle peripheral image 1 is generated on the work surface 34 after the internal parameter setting correction unit 70 corrects the internal parameter setting, there is no distortion as shown in FIG. The vehicle surrounding image 1 can be generated, and such a vehicle surrounding image 1 can be appropriately used for correcting the setting of the external parameter.

  Then, the vehicle periphery monitoring image 1 generated through the correction of the internal parameter setting by the internal parameter setting correction unit 70 and the correction of the external parameter setting by the external parameter setting correction unit 51 is as shown in FIG. There is no distortion, and the viewpoint converted images 3, 5, 6, and 7 are appropriately connected to each other.

  Therefore, according to the present embodiment, after detecting the distorted area 75 from the vehicle peripheral image 1 including the lattice pattern image 59, the video area other than the specific video area corresponding to the distorted area 75 in the used video area. Assumes that no error has occurred in the currently set internal parameter (first distortion correction value), and calculates a new internal parameter (second distortion correction value) to correct the setting of the internal parameter. The video area to be performed can be limited to a specific video area.

  As a result, even when there is an error in the currently set internal parameter, the vehicle obtained through the distortion correction processing after the internal parameter setting correction work after correcting the internal parameter setting at low cost Based on the peripheral image 1, the external parameter setting can be appropriately corrected.

  Next, an embodiment of the camera mounting position / posture information setting correction method according to the present invention will be described with reference to FIGS.

  In the present embodiment, an external parameter setting correction method applied to the above-described vehicle periphery monitoring device 32 will be described as a camera mounting position / posture information setting correction method.

  In the initial state, it is assumed that the external parameter setting unit 48 sets the design values of the external parameters of the in-vehicle cameras 34, 35, 36, and 37 in the viewpoint conversion unit 46. Further, it is assumed that only the design values of the internal parameters of the in-vehicle cameras 34, 35, 36, and 37 are set in the distortion correction unit 42 by the internal parameter setting unit 43. Examples of such cases include cases where external parameter and internal parameter settings have never been modified, or settings have been modified in the past, but there are errors in the design values of the external and internal parameters. The case where the state which has not been continued is mentioned.

  From the initial state, in the present embodiment, first, as shown in step 1 (ST1) of FIG.

  Here, the specific contents of Step 1 (ST1) are as shown in FIG.

  That is, as shown in FIG. 13, in the internal parameter setting correction work, first, as shown in step 1A (ST1A) of FIG. 13, a lattice pattern 53 is formed around the host vehicle 54 as shown in FIG. The installed work environment is set, and the process proceeds to Step 1B (ST1B). In FIG. 2, the vertical line pattern 57 is parallel to the entire length direction of the host vehicle 54, and the positional relationship between the lattice pattern 53 and the host vehicle is a predetermined predetermined positional relationship. Yes.

  Next, in step 1B (ST1B), the surroundings of the vehicle are photographed by the four in-vehicle cameras 34, 35, 36, and 37 in the work environment installed in step 1A (ST1A), and then the video is cut out. The unit 40 cuts out lattice pattern images corresponding to the in-vehicle cameras 34, 35, 36, and 37 from the captured images of the in-vehicle cameras 34, 35, 36, and 37, respectively.

  Next, in step 1C (ST1C), under the control of the distortion correction control unit 71, each lattice pattern image is converted to each lattice pattern image extracted in step 1B (ST1B) by the distortion correction unit 42. The distortion correction process for the internal parameter setting correction work using the design values of the internal parameters of the corresponding on-vehicle cameras 34, 35, 36, and 37 is performed (see FIG. 8).

  Next, in step 1D (ST1D), the viewpoint conversion unit 46 converts each video obtained by the distortion correction processing for setting correction work in step 1C (ST1C) to each corresponding viewpoint conversion image 3, 5, 6. , 7, and the viewpoint converted images 3, 5, 6, 7 and the illustration image 2 are combined with each other to generate the vehicle peripheral image 1 including the lattice pattern image 59 (see FIG. 9). ).

  Next, in step 1E (ST1E), the distortion region detector 72 calculates the shape of the grid pattern image 59 in the vehicle peripheral image 1 (viewpoint conversion images 3, 5, 6, 7) generated in step 1D (ST1D). By comparing with the regular shape of the lattice pattern 53, the strain region 75 and the strain remaining amount in the strain region 75 are detected.

  Next, in step 1F (ST1F), the internal parameter calculation unit 76 determines whether or not the distortion region 75 is detected in step 1E (ST1E). If detected, the process proceeds to step 1G (ST1G). If not detected, the process proceeds to step 1I (ST1I).

  In step 1G (ST1G), the internal parameter calculation unit 76 causes the internal parameter calculation unit 76 to measure the internal parameter actual value (in relation to a specific video area corresponding to the distortion area 75 before the distortion correction processing for setting correction work). A new internal parameter) is calculated, and the process proceeds to Step 1H (ST1H).

  On the other hand, in step 1I (ST1I), the process proceeds to step 1J (ST1J) without calculating the measured value of the internal parameter.

  Next, in step 1H (ST1H), under the control of the internal parameter setting control unit 78, the internal parameter setting unit 43 calculates the actual value of the internal parameter calculated in step 1G (ST1G) in the use video area 16. A new internal parameter used for distortion correction processing for a video in a specific video area corresponding to the distortion area 75 is newly set, and the process ends. At this time, for the video areas other than the specific video area in the used video area 16, the design value of the currently set internal parameter is continuously set as the internal parameter used for the distortion correction processing.

  On the other hand, in step 1J (ST1J), the internal parameter setting correction is not performed, and the setting of the internal parameter design values for all the used video areas 16 is continued and the process is terminated.

  Next, returning to FIG. 12, in step 2 (ST2), the surroundings of the host vehicle are again detected by the four in-vehicle cameras 34, 35, 36, and 37 under the work environment installed in step 1A (ST1A). After shooting, the image cutout unit 40 cuts out a lattice pattern image corresponding to each of the in-vehicle cameras 34, 35, 36, and 37 from each of the images of the in-vehicle cameras 34, 35, 36, and 37.

  Next, in step 3 (ST3), the currently set internal parameters corresponding to the respective lattice pattern images are used for each lattice pattern image cut out in step 2 (ST2) by the distortion correction unit 42. Each distortion correction process is performed. Since this distortion correction processing is performed using the internal parameters after the setting correction work in Step 1 (ST1), the distortion of the video can be corrected appropriately.

  Next, in step 4 (ST4), each video obtained by the distortion correction processing in step 3 (ST3) by the viewpoint conversion unit 46 is converted into the corresponding viewpoint-converted images 3, 5, 6, and 7, respectively. In addition to the conversion, the viewpoint conversion images 3, 5, 6, 7 and the illustration image 2 are combined with each other to generate the vehicle peripheral image 1 including the lattice pattern image 59 (see FIG. 4). In this vehicle peripheral image 1, distortion is appropriately removed, but there may be a shift in the connection between the viewpoint conversion images 3, 5, 6, and 7 due to the influence of the error of the external parameter.

  Next, in step 5 (ST5), the feature point extraction unit 52 extracts lattice points 60 from the vehicle peripheral image 1 including the lattice pattern image 59 generated in step 4 (ST4) (see FIG. 5).

  Next, in step 6 (ST6), the error detection unit 62 performs joint portions 3a and 3b based on the extraction result of the grid point 60 in step 5 (ST5) and the data stored in the grid pattern database 63. , 5a, 5b, 6a, 6b, 7a, 7b, and the regular shape of the lattice pattern image 59 in the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b. Compare. Based on the result of this comparison, an error in the position of the lattice point 60 in the joint portions 3a, 3b, 5a, 5b, 6a, 6b, 7a, 7b is detected.

  Next, in step 7 (ST7), the first correction amount calculation unit 64 determines whether an error is detected in step 6 (ST6). If detected, the process proceeds to step 8 (ST8). If not detected, the process is terminated. When the process ends here, there is no error in the currently set external parameter.

  Next, in step 8 (ST8), the first correction amount calculation unit 64 calculates the first correction amount and proceeds to step 9 (ST9).

  Next, in step 9 (ST9), the second correction amount calculation unit 65 calculates a second correction amount and proceeds to step 10 (ST10).

  Next, in step 10 (ST10), the connection in the external parameter currently set by the external parameter setting unit 48 is performed by the first correction unit 67 based on the first correction amount calculated in step 8 (ST8). The external parameters (for example, design values) corresponding to the eye portions 8, 10, 11, and 12 are corrected, and the process proceeds to step 11 (ST11).

  Next, in step 11 (ST11), based on the second correction amount calculated in step 9 (ST9), the non-joint region (5c, etc.) in the external parameter currently set by the external parameter setting unit 48 The external parameter (for example, design value) corresponding to is corrected, and the process is completed.

  Then, when the vehicle peripheral image 1 is displayed using the external parameter after the correction of the setting after the correction of the setting of the external parameter is completed, as shown in FIG. A good image in which 6 and 7 are appropriately connected can be visually recognized.

  As described above, according to the present embodiment, the external parameter setting correction unit 51 corrects the external parameter setting (for example, correction from the design value of the external parameter to the corrected external parameter) by the lattice pattern image. 59, which is based on the vehicle peripheral image 1 after the appropriate distortion correction processing, and the error of the position of the grid point 60 detected by the error detection unit 62 for the joint portion. While an appropriate external parameter is actually measured by the correction to be eliminated, the region other than the joint portion in the viewpoint conversion image can be corrected by linear interpolation.

  As a result, it is possible to display a vehicle peripheral image with excellent visibility with little shift between viewpoint-converted images at a low cost.

  In addition, this invention is not limited to embodiment mentioned above, A various change is possible as needed.

  For example, in the above-described embodiment, when the distortion region 75 is detected, the shape of the lattice pattern on the vehicle peripheral image 1 (viewpoint conversion images 3, 5, 6, and 7) and the normal shape of the lattice pattern are determined. Although the comparison is made, the present invention is not limited to such a configuration. For example, the shape of the lattice pattern on the image obtained by the distortion correction processing for the internal parameter setting correction work Then, after comparing with the regular shape of the lattice pattern, an area where distortion remains on the video may be detected as a distortion area.

  Moreover, it is not necessary to be limited to the lattice pattern, and it is also possible to use a pattern that can capture a known shape as a feature point, for example, a checkered pattern. In the case of the checkered pattern, each vertex portion of the checkered pattern of black and white birch (including the case where the black birch and white birch vertices coincide) is used as a feature point.

  In the above-described embodiment, the distortion correction process and the viewpoint conversion for the video in the use video area 16 are performed in separate processes. However, the distortion correction process and the viewpoint conversion are performed using, for example, one mapping table. And may be performed in one step (simultaneously). In this case, it can be considered that the video obtained by the distortion correction processing is subjected to viewpoint conversion at the same time as being obtained (obtained as a state of the image subjected to viewpoint conversion). In this case, the shape of the second object to be photographed on the image obtained by the viewpoint conversion is used for detecting the distortion region.

  In addition to the work environment conditions in the above-described embodiment, the conditions of the work environment in which the internal parameter and external parameter setting correction work is performed include the fact that the host vehicle is stopped, and the setting correction work Other conditions such as a user operation for performing the above may be added.

The block diagram which shows embodiment of the vehicle periphery monitoring apparatus which concerns on this invention The figure which shows the setting state of the work environment for performing the setting correction work of an external parameter in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention The figure which shows the picked-up image of the vehicle-mounted camera at the time of the external parameter setting correction work in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows the display state of a vehicle periphery image in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention when there exists an error in the set external parameter. Explanatory drawing for demonstrating a joint part and a lattice point in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention The figure which shows the detection state of the position error of the lattice point in a joint part in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. Explanatory drawing for demonstrating the linear interpolation for calculating 2nd correction amount in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention The figure which shows the distortion correction process for setting correction work of an internal parameter in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows the vehicle periphery monitoring image produced | generated through the distortion correction process for internal parameter setting correction work and viewpoint conversion in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention The figure which shows typically the lattice point extracted from the vehicle periphery image of FIG. 9 in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows the favorable vehicle periphery image after the setting correction work of the external parameter with the internal parameter setting correction work in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention is performed. 6 is a flowchart showing an embodiment of a camera attachment position / posture information setting correction method according to the present invention. The flowchart which shows the setting correction work of an internal parameter in embodiment of the setting correction method of the camera attachment position and attitude | position information based on this invention The figure which shows an example of the vehicle periphery image conventionally employ | adopted The figure which shows the photographing picture of the vehicle camera Explanatory diagram for explaining the internal parameters of the in-vehicle camera The figure which shows an example of the target board conventionally used for the measurement of the external parameter The figure which shows the installation state of the target board around the own vehicle The figure which shows the photographing picture of the target board

Explanation of symbols

1 Vehicle peripheral image 3 Front viewpoint converted image 5 Left side viewpoint converted image 6 Right side viewpoint converted image 7 Rear viewpoint converted image 16 Used video area 32 Vehicle periphery monitoring device 34 Front camera 35 Left side camera 36 Right side camera 37 Back camera 40 Video cutout unit 42 Distortion correction unit 43 Internal parameter setting unit 46 View point conversion unit 47 Display 48 External parameter setting unit 51 External parameter setting correction unit 52 Feature point extraction unit 53 Grid pattern 62 Error detection unit 64 First correction amount calculation unit 65 2 correction amount calculation unit 67 first correction unit 68 second correction unit

Claims (16)

A plurality of in-vehicle cameras having a wide-angle field of view attached to a predetermined position in the host vehicle, and based on the captured video of the periphery of the host vehicle by each in-vehicle camera, the periphery of the host vehicle is positioned above the host vehicle. A vehicle periphery monitoring device that generates a vehicle periphery image that is an image looked down from and displays it on a display unit,
The vehicle periphery monitoring device body
Video cutout means for cutting out a video in a predetermined use video area for each vehicle-mounted camera used for generating the vehicle peripheral image from each captured video of the plurality of vehicle-mounted cameras,
Distortion correction processing for each in-vehicle camera using a predetermined distortion correction value for each in-vehicle camera that has been set for the video in the use video area for each in-vehicle camera cut out by the image cutout means. Distortion correction means for each,
Distortion correction value setting means for setting the distortion correction value for each vehicle-mounted camera used in the distortion correction processing by the distortion correction means;
The image obtained by the distortion correction processing for each in-vehicle camera by the distortion correction means is set to a predetermined area above the vehicle by using the set mounting position / posture information of each in-vehicle camera. Viewpoint conversion means for generating the vehicle peripheral image in which the viewpoint conversion images for each in-vehicle camera are connected to each other by converting the viewpoint to a viewpoint conversion image that is an image as seen from the viewpoint;
Mounting position / posture information setting means for setting the mounting position / posture information of each on-vehicle camera used for the viewpoint conversion by the viewpoint conversion means;
Mounting position / posture information setting correcting means capable of correcting the setting of the mounting position / posture information by the mounting position / posture information setting means, and
The attachment position / posture information setting correction means includes:
Under a work environment in which a first photographing object having a known shape capable of extracting a plurality of feature points arranged in a straight line is installed at a predetermined position in the vicinity of the host vehicle, A vehicle peripheral image in which the viewpoint conversion images of the respective in-vehicle cameras each including the image of the first imaging object are connected based on the captured image of each in-vehicle camera in which the one imaging object is reflected Is generated, the feature point extracting means for extracting the feature point from the image of the first object to be photographed;
Based on the extraction result of the feature point extraction means, the arrangement state of the feature points at a joint portion with any other viewpoint conversion image for any one viewpoint conversion image, and the first portion at the joint portion. An error detection means for detecting an error in the position of the feature point in the joint portion by comparing with a regular shape of the photographing object;
First correction amount calculating means for calculating a correction amount of first attachment position / posture information corresponding to the joint portion capable of eliminating the error detected by the error detecting means;
Linear interpolation using the calculation result of the first correction amount calculation means for each of the pair of joint portions formed so as to provide an interval in the normal arrangement direction of the feature points in the viewpoint conversion image Corresponding to a region other than the joint portion that can eliminate an error in the position of the feature point in a region other than the joint portion sandwiched between the pair of joint portions in the viewpoint conversion image. Second correction amount calculation means for calculating a correction amount of the second attachment position / posture information;
Based on the correction amount of the first attachment position / posture information, first information for correcting the information corresponding to the joint portion in the attachment position / posture information currently set by the attachment position / posture information setting means is corrected. Correction means,
Based on the correction amount of the second attachment position / posture information, information corresponding to an area other than the joint portion in the attachment position / posture information currently set by the attachment position / posture information setting unit is corrected. A vehicle periphery monitoring device, comprising: a second correction means. The first object to be photographed is a lattice pattern disposed around the own vehicle on a flat work surface on which the own vehicle is placed so that a pattern surface thereof is parallel to the work surface. The vehicle periphery monitoring device according to claim 1, wherein the feature points are lattice points in the lattice pattern. The vehicle according to claim 1 or 2, wherein the mounting position / posture information currently set by the mounting position / posture information setting means is a design value of the mounting position / posture information. Perimeter monitoring device. The vehicle periphery monitoring device according to any one of claims 1 to 3, wherein the attachment position / posture information is an external parameter of the in-vehicle camera. Distortion correction value setting correction means capable of correcting the distortion correction value setting by the distortion correction value setting means,
The distortion correction value setting correction means includes
The usage image in which the second imaging object is reflected from the imaging image of the in-vehicle camera in a work environment in which a second imaging object having a known shape is installed around the vehicle. After the image in the region is cut out, by controlling the distortion correction unit, the distortion correction value is currently set by the distortion correction value setting unit as a distortion correction process for setting and correcting the distortion correction value. Distortion correction control means for performing distortion correction processing on the video in the use video area using the first distortion correction value,
The shape of the second object to be photographed on the video obtained by the distortion correction processing for the setting correction work or on the image obtained by the viewpoint conversion by the viewpoint conversion means for the video, and the second By comparing with the normal shape of the object to be photographed, the distortion correction processing can be accurately performed from the image obtained by the distortion correction processing for the setting correction work or the image obtained by the viewpoint conversion for the video. A distortion area detecting means for detecting a distortion area that is an area in the video or the image in which distortion remains without being performed, together with a residual amount of distortion in the distortion area;
An additional correction amount capable of correcting the distortion in the distortion region detected by the distortion region detection means is obtained based on the remaining amount of distortion, and the obtained additional correction amount is the first correction amount. A distortion correction value calculating means for calculating a second distortion correction value by adding to the distortion correction value of 1;
By controlling the distortion correction value setting means, the distortion correction value setting means causes the second distortion correction value calculated by the distortion correction value calculation means to be stored in the distortion area of the used video area. The distortion correction value used for the distortion correction processing for the video in the corresponding specific video area is newly set, and the video area other than the specific video area in the used video area is the first video area. And a setting control means for continuing to set the distortion correction value as the distortion correction value used for the distortion correction processing, thereby correcting only the setting of the distortion correction value corresponding to the specific video area. Formed as
The attachment position / posture information setting / correcting means is attached position / posture information based on the vehicle peripheral image generated through the distortion correction processing after the distortion correction value setting / correcting work by the distortion correction value setting / correcting means. The vehicle periphery monitoring device according to any one of claims 1 to 4, wherein the vehicle periphery monitoring device is configured to correct the setting. The vehicle periphery monitoring device according to claim 5, wherein the second imaging object is the first imaging object. The vehicle periphery monitoring device according to any one of claims 1 to 6, wherein the first distortion correction value is a design value of the distortion correction value. The vehicle periphery monitoring apparatus according to any one of claims 1 to 7, wherein the distortion correction value is an internal parameter of the in-vehicle camera. From each of the images taken around the vehicle by a plurality of in-vehicle cameras having a wide-angle field of view attached at a predetermined position in the own vehicle, each image in a predetermined use image area for each in-vehicle camera is cut out, Each in-vehicle camera performs distortion correction processing for each in-vehicle camera using a predetermined distortion correction value for each in-vehicle camera that has been set, with respect to the image in the use image area for each extracted in-vehicle camera. An image obtained by viewing the image obtained by the distortion correction processing for each camera from the predetermined viewpoint above the vehicle using the set mounting position / posture information of each in-vehicle camera. By converting the viewpoints into viewpoint conversion images, the viewpoint conversion images of the respective in-vehicle cameras are connected to each other. In the vehicle periphery monitoring device that generates a vehicle periphery image that is an image of a side looking down from above the host vehicle, and displays the generated vehicle periphery image on a display unit, the attachment position used for the viewpoint conversion A camera mounting position / posture information setting correction method for correcting posture information settings,
A first shooting object having a known shape capable of extracting a plurality of feature points arranged in a straight line sets a work environment installed at a predetermined position around the vehicle,
Under this set work environment, for each in-vehicle camera each containing an image of the first object to be photographed, based on a photographed image of each in-vehicle camera in which the first object to be photographed is reflected After generating the vehicle periphery image in which the viewpoint conversion images of the two are connected, the feature points are extracted from the image of the first photographing object,
Based on the extraction result of the feature points, the arrangement state of the feature points at a joint portion with any other viewpoint conversion image for any one viewpoint conversion image, and the first imaging target at the joint portion By comparing the regular shape of the object, an error in the position of the feature point in the joint portion is detected,
Calculating a correction amount of the first attachment position / posture information corresponding to the joint portion capable of eliminating the detected error;
Linear interpolation using the calculation result of the first attachment position / posture information for each of the pair of joint portions formed so as to provide an interval in the normal arrangement direction of the feature points in the viewpoint conversion image Corresponding to a region other than the joint portion that can eliminate an error in the position of the feature point in a region other than the joint portion sandwiched between the pair of joint portions in the viewpoint conversion image. Calculate the correction amount of the second mounting position / posture information,
Based on the correction amount of the first attachment position / posture information, the information corresponding to the joint portion in the currently set attachment position / posture information is corrected,
Based on the correction amount of the second attachment position / posture information, information corresponding to a region other than the joint portion in the currently set attachment position / posture information is corrected.・ Posture information setting correction method. As the first object to be photographed, a lattice pattern installed around the vehicle on a flat work surface on which the vehicle is placed so that its pattern surface is parallel to the work surface is used. The camera attachment position / posture information setting correction method according to claim 9, wherein a lattice point in the lattice pattern is used as the feature point. 11. The camera mounting position / posture information setting correction method according to claim 9, wherein a design value of the mounting position / posture information is used as the currently set mounting position / posture information. The camera mounting position / posture information setting correction method according to claim 9, wherein external parameters of the in-vehicle camera are used as the mounting position / posture information. The usage image in which the second imaging object is reflected from the imaging image of the in-vehicle camera in a work environment in which a second imaging object having a known shape is installed around the vehicle. After the image in the region is cut out, the distortion correction processing for the image in the video region in use using the first distortion correction value that is currently set is used as the distortion correction processing for setting and correcting the distortion correction value. Done
The shape of the second object to be photographed on the image obtained by the distortion correction processing for the setting correction work or the image obtained by the viewpoint conversion on the image, and the normality of the second object to be photographed By comparing with the shape of the image, the distortion correction processing is not accurately performed from the video obtained by the distortion correction processing for the setting correction work or the image obtained by the viewpoint conversion on the video. Detect the distortion area that is the area in the video or the image that remains, along with the remaining amount of distortion in the distortion area,
An additional correction amount capable of correcting the distortion in the detected distortion region is obtained based on the remaining amount of distortion, and the obtained additional correction amount is used as the first distortion correction value. By calculating the second distortion correction value by adding,
The calculated second distortion correction value is newly set as the distortion correction value used in the distortion correction processing for the video in a specific video area corresponding to the distortion area of the used video area. For the video area other than the specific video area in the use video area, the first distortion correction value is continuously set as the distortion correction value used for the distortion correction processing, thereby the specific video area. Only correct the distortion correction value setting corresponding to the video area,
The setting of the attachment position / posture information is corrected based on the vehicle peripheral image generated through the distortion correction processing after the correction is performed. The camera mounting position / posture information setting correction method according to the item. The camera mounting position / posture information setting correction method according to claim 13, wherein the first imaging object is used as the second imaging object. The camera attachment position / posture information setting correction method according to any one of claims 9 to 14, wherein a design value of a distortion correction value is used as the first distortion correction value. The camera mounting position / posture information setting correction method according to any one of claims 9 to 15, wherein an internal parameter of the in-vehicle camera is used as the distortion correction value.

Images