JP2010136082A - Apparatus for monitoring vehicle surroundings, and method of determining position and attitude of camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus of monitoring vehicle surroundings and a method of determining the position and attitude of a camera capable of determining whether an attaching position and an attaching attitude of the camera are correct easily and exactly. <P>SOLUTION: An image display processing means 26 can display an overhead image 28 of vehicle surroundings so that view point conversion images 30, 31, 32, and 33 corresponding to optional cameras 21, 22, 23, and 24 are adjacent to view point conversion images 30, 31, 32, and 33 corresponding to the other cameras 21, 22, 23, and 24 across a first border line parallel to a vehicle entire length direction on the image 28 and a second border line parallel to a vehicle width direction on the image 28. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a vehicle periphery monitoring device and a camera position / posture determination method, and more particularly to a vehicle periphery monitoring device and a camera position / posture determination method suitable for determining whether a camera mounting position and mounting posture are appropriate.

  Conventionally, as a system that supports safe and smooth driving operations of a host vehicle in a parking lot or the like, a plurality of in-vehicle cameras attached to the host vehicle are used to shoot a shooting area such as a road surface around the host vehicle, and a captured image Based on the above, as shown in FIG. 5, a vehicle surroundings overhead image 1 that is an image of the surroundings of the own vehicle viewed from a virtual viewpoint P (see FIG. 6) above the own vehicle is generated and displayed on the display inside the vehicle. A vehicle periphery monitoring device that can monitor the situation around the host vehicle by displaying the information has been adopted.

  Here, as shown in FIG. 5, the vehicle periphery overhead image 1 has an illustration image 2 imitating the own vehicle as if the own vehicle is seen from the virtual viewpoint P at the center thereof.

  Further, as shown in FIG. 5, the vehicle periphery overhead image 1 is such that an image within a predetermined use range (pixel range) in a captured image obtained by capturing an imaging region in front of the host vehicle is viewed from the virtual viewpoint P. It has a front viewpoint conversion image 3 which is an image subjected to viewpoint conversion. The front viewpoint conversion image 3 has a rear end portion (lower end portion in FIG. 5) connected to a front end portion (upper end portion in FIG. 5) of the illustration image 2 and a vehicle on the image 1 as it goes forward. It is formed in the shape of an isosceles trapezoid so that the dimension in the width direction (lateral direction in FIG. 5) increases. Such a front-side viewpoint conversion image 3 is a video image taken by a front camera as an in-vehicle camera attached to the front portion (for example, an emblem portion) of the host vehicle in a posture that faces (captures) the front of the host vehicle. It is generated based on.

  Further, as shown in FIG. 5, the vehicle periphery overhead image 1 is obtained by performing viewpoint conversion so that an image within a predetermined use range in a captured image obtained by capturing an imaging region on the left side of the host vehicle is viewed from the virtual viewpoint P. It has a left side viewpoint conversion image 4 which is an image. The left side viewpoint-converted image 4 has a right end connected to a left end of the illustration image 2 and has a size in the vehicle full length direction (vertical direction in FIG. 5) on the image 1 toward the left. It is formed in an isosceles trapezoidal shape that increases. Further, the left side viewpoint converted image 4 and the above-described front side viewpoint converted image 3 include a front end portion in the left side viewpoint conversion image 4 and a left end portion in the front viewpoint conversion image 3 as shown in FIG. Are adjacent to each other with a boundary line 5 separating them. Such a left side viewpoint conversion image 4 is a left side camera as an in-vehicle camera attached to the left side of the host vehicle (for example, the left door mirror unit) in such a posture that the left side of the host vehicle is looked down obliquely from above. Are generated based on the captured video.

  Furthermore, as shown in FIG. 5, the vehicle periphery overhead image 1 is a viewpoint conversion so that an image within a predetermined use range in a captured image obtained by capturing an imaging region on the right side of the host vehicle is viewed from the virtual viewpoint P. A right-side viewpoint conversion image 6 which is the obtained image. The right-side viewpoint conversion image 6 has an isosceles trapezoidal shape in which the left end portion is connected to the right end portion of the illustration image 2 and the size in the vehicle full length direction on the image increases toward the right. Is formed. Further, the right side viewpoint converted image 6 and the above-described front side viewpoint converted image 3 include a front end part in the right side viewpoint converted image 6 and a right end part in the front side viewpoint converted image 3 as shown in FIG. Are adjacent to each other with a boundary line 7 separating them. Such a right side viewpoint conversion image 6 is a right side camera as an in-vehicle camera attached to the right side portion (for example, the right door mirror portion) of the host vehicle in such a posture that the right side of the host vehicle is looked down obliquely from above. Are generated based on the captured video.

  Further, as shown in FIG. 5, the vehicle periphery overhead image 1 is an image obtained by performing viewpoint conversion so that an image within a predetermined use range in a captured image obtained by capturing an imaging area behind the host vehicle is viewed from the virtual viewpoint P. The rear side viewpoint converted image 8 is provided. The rear-side viewpoint converted image 8 has an isosceles trapezoidal shape in which the front end portion is connected to the rear end portion of the illustration image 2 and the size in the vehicle width direction on the image increases toward the rear. Is formed. Further, as shown in FIG. 5, the rear side viewpoint converted image 8 and the left side viewpoint converted image 4 described above include a left end part in the rear side viewpoint converted image 8 and a rear end part in the left side viewpoint converted image 4. Are adjacent to each other with a boundary line 10 separating them. Further, the rear viewpoint conversion image 8 and the right side viewpoint conversion image 6 described above include a right end portion in the rear side viewpoint conversion image 8 and a rear end portion in the right side viewpoint conversion image 6, as shown in FIG. Are adjacent to each other with a boundary line 11 separating them. Such a rear-side viewpoint conversion image 8 is a captured image of a back camera as an in-vehicle camera attached to the rear part (for example, rear license garnish part) of the host vehicle in a posture so as to look down at the rear side of the host vehicle from obliquely above. It is generated based on.

  In addition, as each vehicle-mounted camera corresponding to each viewpoint conversion image 3, 4, 6, 8, a camera having a super-wide-angle lens such as a fisheye lens is used from the viewpoint of enabling monitoring of the direction of 360 ° around the host vehicle. There were many things.

  In such a vehicle periphery monitoring device, for example, as shown in FIG. 7, a captured image 14 of the in-vehicle camera imaged on an imaging surface (for example, a sensor surface of an image sensor such as a CCD or CMOS) 12 of the in-vehicle camera. The vehicle is obtained by performing coordinate conversion (that is, viewpoint conversion) on a video in a predetermined use range (pixel area) 15 of the vehicle using a so-called mapping table in which a correspondence relationship between the video and the vehicle periphery overhead image 1 is described. In general, the surrounding bird's-eye view image 1 is generated.

  Here, the mapping table is created using external parameters of the in-vehicle camera. As the external parameters, the rotation that defines the direction of the in-vehicle camera (the direction of the optical axis) in the three-dimensional world coordinate system. A translation vector that defines the position of a vehicle-mounted camera in a matrix and a three-dimensional world coordinate system is known.

  If the viewpoint conversion by the mapping table does not involve distortion correction, the viewpoint conversion image obtained by viewpoint conversion is further subjected to distortion correction, and the viewpoint conversion images after distortion correction are joined together. Thus, the vehicle periphery overhead view image 1 is generated.

  Such a mapping table functions correctly if the mounting position and mounting posture of the in-vehicle camera in the own vehicle (in other words, the direction of the optical axis of the in-vehicle camera) is appropriate as designed, and the captured video 14 of each in-vehicle camera Based on the above, it has been possible to generate a good vehicle periphery overhead image 1 in which the viewpoint conversion images 3, 4, 6, and 8 are appropriately connected at the boundary lines 5, 7, 10, and 11.

  However, the in-vehicle camera may not necessarily be attached to the host vehicle with high accuracy as designed. In such a case, each viewpoint conversion image 3, 4 is located at the position of the boundary line 5, 7, 10, 11. There is a possibility that a poor vehicle periphery overhead image 1 in which a deviation between 6, 8 is confirmed is generated.

  Therefore, conventionally, for the vehicle periphery monitoring device, for example, by using a photographed image obtained by photographing a plurality of circular patterns drawn on the target board for calibration at the time of manufacturing or camera replacement. The mounting position and mounting posture of the in-vehicle camera have been calculated as external parameters of the in-vehicle camera. When the calculated external parameter deviates from the design value, the external parameter is corrected and the mapping table is corrected accordingly so that the deviation is eliminated.

  Further, conventionally, after such correction of the mapping table, in order to finally determine whether the mounting position and mounting posture of the in-vehicle camera are appropriate before actual use, the determination is made using the corrected mapping table. The vehicle surroundings bird's-eye view image 1 is displayed.

  As the vehicle periphery bird's-eye view image 1 for this determination, for example, as shown in FIG. 8, an image 17 of a first pattern (for example, a white line) that is long in the vehicle width direction on the image 1 straddles the boundary line 5. Thus, the image 1 as displayed in the two viewpoint conversion images 3 and 4 adjacent to each other, and a second pattern (for example, as shown in FIG. There is an image 1 in which an image 18 of a white line) is displayed in the two viewpoint conversion images 3 and 4 so as to straddle the boundary line 5.

  Note that the image 17 of the first pattern in FIG. 8 and the image 18 of the second pattern in FIG. 9 may be displayed simultaneously.

  The image 17 of the first pattern is obtained by photographing the first pattern parallel to the vehicle width direction of the own vehicle actually arranged on the placement surface (for example, road surface) of the own vehicle. This is obtained by converting the viewpoint of the captured video using the mapping table.

  Similarly, the image 18 of the second pattern is obtained by photographing the second pattern parallel to the vehicle full length direction of the own vehicle actually arranged on the placement surface of the own vehicle, and mapping the taken image to the mapping table. It is obtained by changing the viewpoint using.

  Conventionally, by visually checking such a vehicle periphery overhead image 1 for determination, whether or not the mounting position and mounting posture of each in-vehicle camera are appropriate is determined on the vehicle periphery overhead image 1 of the images 17 and 18 of each pattern. Is determined by confirming the straightness in the vehicle width direction and the full length direction of the vehicle (the degree of parallelism) and the presence or absence of displacement of the images 17 and 18 of each pattern at the position of the boundary 5.

  Here, the description of the straightness of the images 17 and 18 of each pattern will be omitted, and the presence or absence of the shift of the patterns 17 and 18 will be further described. In the vehicle periphery overhead image 1 for determination in FIG. The presence or absence of a shift in the vehicle full length direction between the images 17 of the first pattern in the two viewpoint conversion images 3 and 4 at the position 5 is confirmed.

  Further, in the vehicle periphery bird's-eye view image 1 for determination in FIG. 9, it is confirmed whether there is a shift in the vehicle width direction between the pattern images 18 in the two viewpoint conversion images 3 and 4 at the position of the boundary line 5. It was.

  Then, the shift of the first pattern images 17 in the two viewpoint conversion images 3 and 4 in the position of the boundary line 5 in the vehicle full length direction and the first viewpoint images in the two viewpoint conversion images 3 and 4 in the position of the boundary line 5 are performed. When there is no shift in the vehicle width direction between the two pattern images 18, the first pattern image 17 is parallel to the vehicle width direction and the second pattern image 18 is in the vehicle total length direction. It was determined that the mounting position and the mounting posture of the in-vehicle camera are appropriate on the assumption that they are parallel (that is, have straight travel).

  Conversely, the shift of the first pattern images 17 in the two viewpoint conversion images 3 and 4 at the position of the boundary line 5 in the vehicle full length direction, or the two viewpoint conversion images 3 and 4 at the position of the boundary line 5. When the image 18 of the second pattern in the middle is displaced in the vehicle width direction, it is determined that the mounting position and mounting posture of the in-vehicle camera are inappropriate.

  And as a result of such determination, when the mounting position and mounting posture of the in-vehicle camera are inappropriate, the images 17 of the first pattern in the two viewpoint conversion images 3 and 4 at the position of the boundary line 5 The mapping table is further re-established so that any shift in the vehicle full-length direction and the shift in the vehicle width direction between the images 18 of the second pattern in the two viewpoint conversion images 3 and 4 at the position of the boundary line 5 do not occur. It was supposed to be corrected.

Japanese Patent No. 3300334

  However, conventionally, as shown in FIG. 8 and FIG. 9, at the position of the boundary line 5 that has an inclination with respect to the vehicle overall direction and the vehicle width direction on the vehicle periphery overhead image 1, Since the displacement of the images 17 and 18 of the long pattern in the full length direction was visually confirmed, even when there is no actual displacement due to the illusion of human eyes by sandwiching the inclined boundary line 5 between them, There was a case where it was mistaken that there was a shift.

  As a result, there has conventionally been a problem that it is difficult to accurately determine whether the mounting position and mounting posture of the in-vehicle camera are appropriate.

  Accordingly, the present invention has been made in view of such problems, and a vehicle periphery monitoring device and a camera position / posture determination method that can easily and accurately determine whether a camera mounting position and mounting posture are appropriate. It is intended to provide.

  In order to achieve the above-described object, a vehicle periphery monitoring device according to the present invention is attached to different attachment positions in a vehicle in an attachment posture so as to face a predetermined photographing direction corresponding to the attachment position. A plurality of cameras each capable of shooting a shooting area corresponding to the shooting direction in the camera, and images within the use range corresponding to the cameras in the shot images of the plurality of cameras, Is converted into a viewpoint conversion image that is an image whose viewpoint is converted so as to be seen from a virtual viewpoint above the vehicle, and the viewpoint conversion images obtained by the viewpoint conversion are adjacent to each other across a boundary line. The vehicle periphery overhead image, which is an image as if the surroundings of the vehicle are viewed from the virtual viewpoint, is generated by connecting A vehicle periphery monitoring apparatus comprising: an image display processing unit that displays the generated vehicle periphery overhead image on a display unit, wherein the image display processing unit corresponds to an arbitrary camera as the vehicle periphery overhead image The viewpoint conversion image to be used corresponds to other cameras across the first boundary line parallel to the vehicle full length direction on the vehicle periphery overhead image and the second boundary line parallel to the vehicle width direction on the vehicle periphery overhead image It is possible to display a vehicle surroundings overhead image that is adjacent to the viewpoint conversion image to be displayed.

  According to such a configuration, the viewpoint conversion image corresponding to an arbitrary camera is displayed on the first boundary line parallel to the vehicle full length direction on the vehicle periphery overhead image and the vehicle periphery overhead image by the image display processing unit. By displaying a vehicle surroundings overhead image that is adjacent to a viewpoint conversion image corresponding to another camera across a second boundary line parallel to the vehicle width direction at It is possible to easily and appropriately confirm whether or not there is a shift in the vehicle full-length direction between the viewpoint conversion image corresponding to the camera and the viewpoint conversion image corresponding to another camera, and the second boundary line can be used for any camera. Since it is possible to easily and appropriately confirm whether there is a shift in the vehicle width direction between the corresponding viewpoint conversion image and the viewpoint conversion image corresponding to another camera, it is easy to determine whether the camera mounting position and mounting posture are appropriate. One can be determined accurately.

  Further, the image display processing means selects a display mode at the time of actual use and a display mode at the time of determination for determining whether or not the mounting position and mounting posture of the camera are appropriate by visually confirming a vehicle overhead view image. In the display mode setting state at the time of the determination, the viewpoint conversion image corresponding to the arbitrary camera has the first boundary line and the second boundary line sandwiched between the first boundary line and the second boundary line. A vehicle periphery overhead image that is adjacent to a viewpoint conversion image corresponding to another camera is displayed, and in the display mode setting state during actual use, the viewpoint conversion image corresponding to the arbitrary camera is A vehicle periphery overhead view image adjacent to the viewpoint conversion image corresponding to the other camera across only the third boundary line inclined with respect to the direction and the vehicle width direction may be displayed.

  According to such a configuration, in actual use, it is possible to display a vehicle surroundings overhead view image that emphasizes visibility based on the resolution of each camera.

  Furthermore, the camera position / posture determination method according to the present invention is mounted at different mounting positions on the vehicle in a mounting posture so as to face a predetermined shooting direction corresponding to the mounting position, and the shooting direction around the vehicle When a plurality of cameras each capable of photographing a photographing area corresponding to each of the plurality of cameras can be photographed, the range of use corresponding to each camera in the photographed video of the plurality of cameras when determining the suitability of the respective mounting positions and postures. Are converted into viewpoint-converted images, which are viewpoint-converted images so that images within the range of use are seen from a virtual viewpoint above the vehicle, and each obtained by the viewpoint conversion By connecting the viewpoint conversion images so as to be adjacent to each other across a boundary line, the virtual view of the periphery of the vehicle is obtained. The vehicle surroundings overhead view image that is an image seen from above is generated, the generated vehicle surroundings overhead view image is displayed on the display unit, and the displayed vehicle surroundings overhead view image is visually confirmed, thereby mounting the camera A camera position / posture determination method for determining whether a position and a mounting posture are appropriate, wherein a viewpoint conversion image corresponding to an arbitrary camera is parallel to the vehicle peripheral overhead image on the vehicle peripheral overhead image as the vehicle peripheral overhead image. After displaying the vehicle periphery overhead image adjacent to the viewpoint conversion image corresponding to the other camera across the boundary line of 1 and the second boundary line parallel to the vehicle width direction on the vehicle periphery overhead image, Using the first boundary line, it is confirmed whether or not there is a deviation in the vehicle full length direction between the viewpoint conversion image corresponding to the arbitrary camera and the viewpoint conversion image corresponding to the other camera. , By using the second boundary line to confirm whether or not there is a shift in the vehicle width direction between the viewpoint conversion image corresponding to the arbitrary camera and the viewpoint conversion image corresponding to the other camera, It is characterized by determining the suitability of the mounting position and mounting posture.

  And according to such a method, the presence or absence of the shift | offset | difference of the vehicle full length direction of the viewpoint conversion image corresponding to arbitrary cameras and the viewpoint conversion image corresponding to another camera using the 1st boundary line is convenient and appropriate. In addition, the second boundary line can be used to easily and appropriately check whether there is a shift in the vehicle width direction between the viewpoint conversion image corresponding to an arbitrary camera and the viewpoint conversion image corresponding to another camera. Therefore, it is possible to easily and accurately determine whether the camera mounting position and mounting posture are appropriate.

  Furthermore, as the vehicle periphery overhead image, in the viewpoint conversion image corresponding to the arbitrary camera such that the image of the first pattern elongated in the vehicle width direction straddles the first boundary line and the other The viewpoint conversion corresponding to the arbitrary camera is displayed in the viewpoint conversion image corresponding to the other camera, and the second pattern image that is long in the vehicle total length direction straddles the second boundary line. In the viewpoint conversion image corresponding to the arbitrary camera at the position of the first boundary line after displaying the vehicle periphery overhead view image displayed in the image and the viewpoint conversion image corresponding to the other camera The first boundary line is confirmed by confirming whether or not there is a deviation in the vehicle full length direction between the first pattern image of the first pattern image and the first pattern image in the viewpoint conversion image corresponding to the other camera. Check whether there is a shift in the vehicle full length direction between the viewpoint conversion image corresponding to the arbitrary camera used and the viewpoint conversion image corresponding to the other camera, and the arbitrary conversion at the position of the second boundary line Checking whether there is a shift in the vehicle width direction between the image of the second pattern in the viewpoint conversion image corresponding to the camera and the image of the second pattern in the viewpoint conversion image corresponding to the other camera. By using the second boundary line, the presence or absence of a shift in the vehicle width direction between the viewpoint conversion image corresponding to the arbitrary camera and the viewpoint conversion image corresponding to the other camera may be confirmed. Good.

  And according to such a method, the vehicle full length of the viewpoint conversion image corresponding to arbitrary cameras and the viewpoint conversion image corresponding to another camera by using the image of the 1st pattern and the image of the 2nd pattern Since the presence / absence of the deviation in the direction and the vehicle width direction can be easily and appropriately confirmed, whether or not the camera mounting position and mounting posture are appropriate can be determined more simply and appropriately.

  According to the vehicle periphery monitoring device and the camera position / posture determination method according to the present invention, it is possible to easily and accurately determine whether the camera mounting position and mounting posture are appropriate.

  Hereinafter, an embodiment of a vehicle periphery monitoring device according to the present invention will be described with reference to FIGS. 1 and 2.

  Note that portions having the same or similar basic configuration as those in the related art will be described using the same reference numerals.

  As shown in FIG. 1, the vehicle periphery monitoring device 20 in this embodiment includes four in-vehicle cameras 21, 22, 23, and 24 mounted on the host vehicle, such as super wide-angle lenses such as fish-eye lenses and CCD or CMOS. Car-mounted cameras 21, 22, 23, and 24 each incorporating an image sensor are provided.

  One of the four in-vehicle cameras 21, 22, 23, and 24 is a front camera 21 that is attached to the front of the host vehicle in a posture facing the front of the host vehicle. The front camera 21 captures a predetermined range of imaging area over a wide viewing angle in front of the host vehicle. In addition, one of the remaining three in-vehicle cameras 22, 23, 24 is attached to the left side of the host vehicle 22 in such a posture that the left side of the host vehicle is looked down obliquely from above. The left side camera 22 captures a predetermined range of imaging area over a wide viewing angle on the left side of the host vehicle. Furthermore, one of the remaining two in-vehicle cameras 23 and 24 is a right side camera 23 attached to the right side of the own vehicle in such a posture that the right side of the own vehicle is looked down obliquely from above. The right side camera 23 takes a picture of a predetermined range over a wide viewing angle on the right side of the host vehicle. Furthermore, the remaining one in-vehicle camera 24 is a back camera 24 that is attached to the rear of the host vehicle in such a posture that the rear of the host vehicle is looked down obliquely from above. A predetermined range of the imaging region over a wide viewing angle behind the vehicle is imaged.

  Further, as shown in FIG. 1, the vehicle periphery monitoring device 20 in the present embodiment has an image display processing unit 26 as an image display processing unit, and each in-vehicle camera 21 is included in the image display processing unit 26. , 22, 23, and 24 are connected to each other.

  The image display processing unit 26 performs various processes for displaying the vehicle periphery overhead image by executing a program stored in a memory (not shown).

  That is, the image display processing unit 26 acquires captured images of the in-vehicle cameras 21, 22, 23, and 24, and the in-vehicle cameras 21, 22 from the acquired captured images of the in-vehicle cameras 21, 22, 23, and 24. , 23, and 24, images within a predetermined use range (pixel area) set individually are extracted. Further, the image display processing unit 26 displays the extracted video within the usage range for each of the in-vehicle cameras 21, 22, 23, and 24, and the video within the usage range is a virtual viewpoint P above the host vehicle (see FIG. 6). ) Are converted into viewpoint-converted images, which are images that have been viewpoint-converted so as to be seen from above. Note that this viewpoint conversion may be accompanied by distortion correction that removes distortion (distortion) of the image within the use range of each of the in-vehicle cameras 21, 22, 23, and 24. If the viewpoint conversion does not involve distortion correction, the image display processing unit 26 performs distortion correction before and after the viewpoint conversion. Then, the image display processing unit 26 connects the viewpoint conversion images of the respective cameras 21, 22, 23, and 24 from which the distortion obtained by the viewpoint conversion is removed so as to be adjacent to each other across the boundary line. Then, a vehicle periphery overhead image, which is an image obtained by looking down from the virtual viewpoint P (see FIG. 6) above the host vehicle, is generated.

  Furthermore, as shown in FIG. 1, the vehicle periphery monitoring apparatus 20 in this embodiment has a display 27 as a display unit. The above-described image display processing unit 26 is connected to the display 27, and the image display processing unit 26 displays the generated vehicle periphery overhead image on the display 27.

  In this embodiment, the image display processing unit 26 can display a vehicle surroundings overhead image (hereinafter referred to as a vehicle surroundings overhead image) 28 as shown in FIG. By visually confirming the vehicle surroundings overhead view image 28, it is possible to determine whether the mounting positions and mounting postures of the on-vehicle cameras 21, 22, 23, and 24 are appropriate.

  As shown in FIG. 2, the vehicle periphery overhead image 28 in the determination mode has a substantially rectangular illustration image 2 imitating the host vehicle as if the host vehicle is viewed from the virtual viewpoint P at the center thereof. . The illustration image 2 is uniquely created by a program executed by the image display processing unit 26, not based on the captured images of the in-vehicle cameras 21, 22, 23, and 24.

  Further, as shown in FIG. 2, the vehicle periphery overhead image 28 in the determination mode is an overhead view of the video in the use range in the captured image of the front camera 21 that images the imaging area in front of the host vehicle from the virtual viewpoint P. In the determination mode, the front viewpoint conversion image 30 is an image whose viewpoint has been converted.

  In this determination mode, the front-side viewpoint conversion image 30 includes a rectangular central portion 30a, a right triangular portion 30b positioned on the left side of the central portion 30a, and a right angle positioned on the right side of the central portion 30a. It is comprised by the triangular-shaped right side part 30c.

  More specifically, the center portion 30a has a rear end portion (lower end portion in FIG. 2) connected to a front end portion (upper end portion in FIG. 2) of the illustration image 2 and a vehicle width direction on the image 28. The dimension in the horizontal direction in FIG. 2 and the display position in the vehicle width direction match the illustration image 2.

  Further, the left side portion 30b has a rear end portion (corresponding to the base of the right triangle) located on the same straight line as the rear end portion of the central side portion 30a, and its left end portion (corresponding to the height of the right triangle) A hypotenuse that extends perpendicularly from the left end at the end to the front end at the central portion 30a and a position where the position in the vehicle full length direction on the image 28 is the same, and extends from the right end at the rear end to the front end at the left end It is formed in a right triangle shape having In the left side portion 30b having such a shape, the right end at the rear end portion is in contact with the left end at the rear end portion of the central side portion 30a.

  Further, the right side portion 30c has a rear end portion (corresponding to the bottom of the right triangle) located on the same straight line as the rear end portion of the central side portion 30a, and its right end portion (corresponding to the height of the right triangle) A hypotenuse that extends perpendicularly from the right end at the end to the front end in the central portion 30a and a position where the position in the vehicle full length direction on the image 28 is the same, and extends from the left end at the rear end to the front end at the right end. It is formed in a right triangle shape having In the right side portion 30c having such a shape, the left end at the rear end portion thereof is in contact with the right end at the rear end portion of the central side portion 30a.

  Further, as shown in FIG. 2, the vehicle periphery overhead image 28 in the determination mode is an overhead view of the video in the use range in the captured image of the left side camera 22 that captures the imaging region on the left side of the host vehicle from the virtual viewpoint P. In the determination mode, the left-side viewpoint conversion image 31 is an image that has been subjected to viewpoint conversion. In this determination mode, the left-side viewpoint-converted image 31 is positioned on the rear side of the central part 31a, the square-shaped central part 31a, the right-sided triangular front part 31b located on the front side of the central part 31a. And a rear portion 31c having a right triangle shape.

More specifically, the center side portion 31a is connected to the left end portion of the illustration image 2 at the right end portion thereof, the dimension in the vehicle full length direction (vertical direction in FIG. 2) on the image 28, and the vehicle full length direction. The display position at is coincident with the illustration image 2. In the present embodiment, the rear end portion of the left portion 30b in the front-side viewpoint conversion image 30 in the determination mode is the vehicle width direction on the image 28 as the second boundary line at the front end portion of the central portion 31a. _Are adjacent to each other so as to sandwich a front / left side second boundary line _LFL2 parallel _thereto . In addition, the front end part of the center side part 31a is formed in the same size as the rear end part of the left side part 30b in the front side viewpoint conversion image 30 in the determination mode.

In addition, the front side portion 31b has a right end portion (corresponding to the base of a right triangle) located on the same straight line as a right end portion of the central side portion 31a, and a front end portion (corresponding to the height of the right triangle) From the front end to the left, the left end in the central portion 31a and the position in the vehicle width direction on the image 28 extend at right angles from the front end to the left, and extend from the rear end at the right end to the left end at the front end. It is formed in a right triangle shape having a hypotenuse. The front part 31b having such a shape has a rear end at the right end thereof in contact with a front end at the right end of the central part 31a. And in this embodiment, the left end part of the center side part 30a in the front side conversion image 30 at the time of the determination mode is in the vehicle full length direction on the image 28 as the first boundary line at the right end part of the front part 31b. Adjacent to each other with a parallel front / left side first boundary line _LFL1 therebetween. In addition, the right end part of the front side part 31b is formed in the same dimension as the left end part of the center side part 30a in the front side viewpoint conversion image 30 in the determination mode. Further, in the present embodiment, the oblique side of the left side portion 30b in the front-side viewpoint conversion image 30 in the determination mode is inclined forward with respect to the vehicle full length direction and the vehicle width direction on the image 28 on the oblique side of the front side portion 31b. / Adjacent to the left side third boundary line _LFL3 . Note that the hypotenuse of the front part 31b is formed in the same size as the hypotenuse of the left part 30b in the front-side viewpoint conversion image 30 in the determination mode.

  Further, the rear part 31c has its right end (corresponding to the base of a right triangle) positioned on the same straight line as the right end of the central part 31a, and its rear end (corresponding to the height of a right triangle) From the rear end at the right end portion to the left, the left end portion at the central portion 31a and the position in the vehicle width direction on the image 28 extend at a right angle to the left end at the rear end portion from the front end at the right end portion. It is formed in a right triangle shape having an extending hypotenuse. The rear part 31c having such a shape has a front end at the right end thereof in contact with a rear end at the right end of the central part 31a.

  Furthermore, as shown in FIG. 2, the vehicle periphery overhead image 28 in the determination mode is an image within the use range in the captured image of the right side camera 23 that captures the imaging region on the right side of the host vehicle from the virtual viewpoint P. It has a right-side viewpoint converted image 32 in the determination mode, which is an image whose viewpoint has been converted so that it can be seen from above. In this determination mode, the right-side viewpoint conversion image 32 is positioned on the rear side of the square side portion 32a, the right side triangular front side portion 32b located on the front side of the central side portion 32a, and the central side portion 32a. And a rear side portion 32c having a right triangle shape.

More specifically, the center side portion 32a has its left end connected to the right end of the illustration image 2, and the size in the vehicle full length direction on the image 28 and the display position in the vehicle full length direction are illustration images. Is consistent with 2. In the present embodiment, the rear end portion of the right side portion 30c of the front-side viewpoint conversion image 30 in the determination mode is the vehicle end direction on the image 28 as the second boundary line at the front end portion of the center side portion 32a. _Are adjacent so as to sandwich a front / right side second boundary line _LFR2 parallel _thereto . In addition, the front end part of the center side part 32a is formed in the same dimension as the rear end part of the right side part 30c in the front side viewpoint conversion image 30 in the determination mode.

The front side portion 32b has a left end portion (corresponding to the base of a right triangle) located on the same straight line as the left end portion of the central side portion 32a, and a front end portion (corresponding to the height of the right triangle) at its left end. From the front end to the right, the right end of the central portion 32a and the position on the image 28 in the vehicle width direction extend at right angles to the right, and extend from the rear end at the left end to the right end at the front end. It is formed in a right triangle shape having a hypotenuse. In the front part 32b having such a shape, the rear end of the left end part thereof is in contact with the front end of the left end part of the central part 32a. In the present embodiment, at the left end of the front portion 32b, the right end of the central portion 30a in the forward viewpoint conversion image 30 in the determination mode is in the vehicle full length direction on the image 28 as the first boundary line. Adjacent to each other with a parallel front / right side first boundary line _LFR1 therebetween. In addition, the left end part of the front side part 32b is formed in the same size as the right end part of the center side part 30a in the front side viewpoint conversion image 30 in the determination mode. Furthermore, in the present embodiment, the oblique side of the right side part 30c in the front-side viewpoint conversion image 30 in the determination mode is inclined forward with respect to the entire vehicle length direction and the vehicle width direction on the image 28 on the oblique side of the front part 32b. / Adjacent to the third boundary line _LFR3 on the right side. The hypotenuse of the front part 32b is formed to be the same size as the hypotenuse of the right part 30c in the front-side viewpoint conversion image 30 in the determination mode.

  Further, the rear portion 32c has a left end portion (corresponding to the base of a right triangle) located on the same straight line as the left end portion of the central portion 32a, and a rear end portion (corresponding to the height of the right triangle) From the rear end at the left end portion to the right, the right end portion at the central portion 32a and the position in the vehicle width direction on the image 28 extend at right angles to the right end, from the front end at the left end portion to the right end at the rear end portion. It is formed in a right triangle shape having an extending hypotenuse. The rear part 32c having such a shape has a front end at the left end part in contact with a rear end of the left end part at the center part 32a.

  Further, as shown in FIG. 2, the vehicle periphery overhead image 28 in the determination mode is an overhead view of the video in the use range in the captured image of the back camera 24 that captures the imaging area behind the host vehicle from the virtual viewpoint P. In the determination mode, the rear-side viewpoint converted image 33 is an image whose viewpoint has been converted. In this determination mode, the rear-side viewpoint conversion image 33 includes a rectangular central portion 33a, a right triangular portion 33b located on the left side of the central portion 33a, and a right angle located on the right side of the central portion 33a. It is comprised by the triangle-shaped right side part 33c.

More specifically, the center side portion 33a has its front end connected to the rear end of the illustration image 2, and the dimension in the vehicle width direction on the image 28 and the display position in the vehicle width direction are illustrated. It matches image 2. In the present embodiment, the right end portion of the rear portion 31c in the left-side viewpoint converted image 31 in the determination mode is the total length of the vehicle on the image 28 as the first boundary line at the left end portion of the central portion 33a. Adjacent to each other with a rear / left side first boundary line _LBL1 parallel to the direction in between. In addition, the left end part of the center side part 33a is formed in the same dimension as the right end part of the rear part 31c in the left side viewpoint conversion image 31 in the determination mode. Further, in the present embodiment, at the right end of the center side portion 33a, the left end of the rear portion 32c in the right-side viewpoint converted image 32 in the determination mode is the total vehicle length on the image 28 as the first boundary line. It adjoins so that the 1st boundary line _LBR1 of back / right side parallel to a direction may be pinched _| interposed. In addition, the right end part of the center side part 33a is formed in the same dimension as the left end part of the rear side part 32c in the right side viewpoint conversion image 32 in the determination mode.

The left side portion 33b has a front end (corresponding to the base of a right triangle) located on the same straight line as the front end of the center side portion 33a, and its left end (corresponding to the height of a right triangle) is the front end. The rear end of the center part 33a extends rearward from the left end of the center part to a position where the position in the vehicle full length direction on the image 28 is the same as the rear end part, and extends from the right end of the front end part to the rear end of the left end part. It is formed in a right triangle shape having a hypotenuse. In the left side portion 33b having such a shape, the right end of the front end portion thereof is in contact with the front end of the left end portion of the central side portion 33a. In the present embodiment, at the front end of the left side portion 33b, the rear end portion of the center side portion 31a in the left-side viewpoint conversion image 31 in the determination mode is the vehicle width on the image 28 as the second boundary line. Adjacent to each other across a rear / left side second boundary line _LBL2 parallel to the direction. In addition, the front end part of the left side part 33b is formed in the same dimension as the rear end part of the center side part 31a in the left side viewpoint conversion image 31 in the determination mode. Further, in the present embodiment, the hypotenuse of the rear part 31c in the left-side viewpoint conversion image 31 in the determination mode is inclined with respect to the entire vehicle length direction and the vehicle width direction on the image 28 on the hypotenuse of the left part 33b. The rear / left side third boundary line _LBL3 is adjacent to each other. Note that the hypotenuse of the left part 33b is formed in the same size as the hypotenuse of the rear part 31c in the left side viewpoint conversion image 31 in the determination mode.

Further, the right side portion 33c has its front end (corresponding to the base of a right triangle) located on the same straight line as the front end of the central side portion 33a, and its right end (corresponding to the height of the right triangle) is its front end. The rear end of the central portion 33a and the rear end of the vehicle 28 in the full length direction on the image 28 extend at right angles from the right end of the section to the rear, and extend from the left end of the front end to the rear end of the right end. It is formed in a right triangle shape having a hypotenuse. In the right side portion 33c having such a shape, the left end of the front end portion thereof is in contact with the front end of the right end portion of the central side portion 33a. In the present embodiment, at the front end of the right portion 33c, the rear end of the central portion 32a in the right-side viewpoint conversion image 32 in the determination mode is the vehicle width on the image 28 as the second boundary line. Adjacent to each other so as to sandwich a rear / right side second boundary line _LBR2 parallel to the direction. In addition, the front end part of the right side part 33c is formed in the same dimension as the rear end part of the center side part 32a in the right side viewpoint conversion image 32 in the determination mode. Further, in the present embodiment, the hypotenuse of the rear portion 32c in the right-side viewpoint conversion image 32 in the determination mode is inclined with respect to the entire vehicle length direction and the vehicle width direction on the image 28 on the hypotenuse side of the right portion 33c. Further, they are adjacent so as to sandwich the rear / right side third boundary line _LBR3 . The oblique side of the right side portion 33c is formed to have the same size as the oblique side of the rear side portion 32c in the right side viewpoint converted image 32 in the determination mode.

That is, in the determination mode vehicle periphery overhead image 28, the viewpoint conversion images 30, 31, 32, 33 corresponding to any one of the cameras 21, 22, 23, 24 are parallel to the vehicle full length direction on the image 28. The first boundary lines L _FL1 , L _FR1 , L _BL1 , L _BR1 and the second boundary lines L _FL2 , L _FR2 , L _BL2 , L _BR2 parallel to the vehicle width direction on the image 28 are sandwiched between the other boundary lines. The image 28 is adjacent to the viewpoint conversion images 30, 31, 32, 33 corresponding to the two cameras 21, 22, 23, 24.

Here, the first boundary lines L _FL1 , L _FR1 , L _BL1 , and L _BR1 on the image 28 that are parallel to the entire vehicle length direction are the boundary lines 5, 7, 10, and 10 in the vehicle periphery monitoring image 1 shown in FIG. Compared to 11, the visual illusion is less likely to occur when the presence / absence of the shift of the viewpoint-converted images 30, 31, 32, 33 in the vehicle full length direction is confirmed.

Accordingly, in this embodiment, the viewpoint conversion image 30 and 31 corresponding to an arbitrary one-vehicle cameras 21, 22, 23, 24 with the first boundary line _{L FL1, L FR1, L BL1} , L BR1 , 32, 33 and other in-vehicle cameras 21, 22, 23, 24 and the viewpoint-converted images 30, 31, 32, 33 corresponding to the image 28 on the image 28 in a simple and appropriate manner. Can do.

Further, the second boundary lines L _FL2 , L _FR2 , L _BL2 , L _BR2 parallel to the vehicle width direction are compared with the boundary lines 5, 7, 10, 11 in the vehicle periphery monitoring image 1 shown in FIG. It has a shape in which an optical illusion is unlikely to occur when the presence or absence of a shift in the vehicle width direction on the image 28 between the viewpoint conversion images 30, 31, 32 and 33 is confirmed.

Therefore, in the present embodiment, the viewpoint conversion images 30 and 31 corresponding to any one of the in-vehicle cameras 21, 22, 23, and 24 using the second boundary lines L _FL2 , L _FR2 , L _BL2 , and L _BR2. , 32, 33 and other in-vehicle cameras 21, 22, 23, 24, and the viewpoint conversion images 30, 31, 32, 33 corresponding to the vehicle 28 in the vehicle width direction on the image 28 are simply and appropriately confirmed. Can do.

  As shown in FIG. 2, for example, the front part 31a in the left-side viewpoint converted image 31 in the determination mode can be displayed ahead of the left part 30b in the front-side viewpoint converted image 30 in the determination mode. This is because there is an overlapping portion in the shooting area between the left side camera 22 and the front camera 21. The same applies to other cameras.

  Further, in FIG. 2, for the sake of convenience, different hatching is applied to each viewpoint conversion image 30, 31, 32, 33 for easy understanding of the arrangement state of each viewpoint conversion image 30, 31, 32, 33. .

  Returning to FIG. 1, a storage unit 35 is connected to the image display processing unit 26, and the storage unit 35 includes videos within the use range for each of the in-vehicle cameras 21, 22, 23, and 24, and determination. A mapping table (hereinafter referred to as a determination mode mapping table) in which the correspondence relationship of coordinates with the vehicle periphery overhead image 28 in the mode is described is stored.

  Using this determination mode mapping table, the image display processing unit 26 extracts the video within the usage range from the video captured by the in-vehicle cameras 21, 22, 23, and 24, and converts the viewpoint of the video within the extracted usage range. By performing viewpoint conversion to images 30, 31, 32, and 33, the vehicle periphery overhead image 28 in the determination mode is generated.

  In addition to the above-described configuration, in the present embodiment, an input operation unit 36 is connected to the image display processing unit 26. The input operation unit 36 is connected to the image display processing unit 26 during actual use. The display mode and the display mode at the time of determination for determining the suitability of the mounting positions and mounting postures of the in-vehicle cameras 21, 22, 23, and 24 can be selectively set.

  Further, in the present embodiment, in addition to the determination mode mapping table, the storage unit 35 has images within the use range for each of the in-vehicle cameras 21, 22, 23, and 24, and the same as that shown in FIG. A mapping table (hereinafter referred to as an actual use mode mapping table) in which a coordinate relationship with a vehicle periphery overhead image (hereinafter referred to as an actual use mode vehicle periphery overhead image) 1 is stored is stored.

  Then, when the display mode at the time of determination is set by the input operation unit 36, the image display processing unit 26 uses the determination mode mapping table stored in the storage unit 35 to determine as shown in FIG. The vehicle periphery overhead image 28 is displayed in the mode.

  Further, the image display processing unit 26 is stored in the storage unit 35 on condition that the vehicle speed pulse of the host vehicle is equal to or lower than a predetermined speed in a state where the display mode during actual use is set by the input operation unit 36. Further, the vehicle use overhead view image 1 in the actual use mode as shown in FIG. 5 is displayed using the actual use mode mapping table.

  Here, from the viewpoint of the visibility of the vehicle periphery overhead image based on the resolution of each of the in-vehicle cameras 21, 22, 23, and 24, the vehicle periphery overhead image 1 in the actual use mode is more than the vehicle periphery overhead image 28 in the determination mode. Is preferred. For this reason, in this embodiment, preferable vehicle periphery overhead images 1 and 28 are selectively displayed according to the application.

In the present embodiment, the boundary lines 5, 7, 10, and 11 in the vehicle periphery overhead image 1 in the actual use mode are the third boundary lines L _FL3 , L _FR3 , and L _BL3 in the vehicle periphery overhead image 28 in the determination mode. , L _BR3 . In the present embodiment, the vehicle periphery overhead image 28 in the determination mode and the vehicle periphery overhead image 1 in the actual use mode are the central portions of the viewpoint conversion images 30, 31, 32, and 33 in the vehicle periphery overhead image 28 in the determination mode. Images corresponding to 30a, 31a, 32a, 33a, and 34a are common to each other. Therefore, the determination mode mapping table is simply created by changing a part of the actual use mode mapping table.

  In addition, the mapping table in the judgment mode and the mapping table in the actual use mode are different in the video of the usage range used for generating the vehicle periphery overhead image, and the in-vehicle camera used for creating the mapping table It goes without saying that the external parameters 21, 22, 23, and 24 match each other.

  Next, an embodiment of the camera position / posture determination method according to the present invention will be described with reference to FIGS.

  In the present embodiment, as shown in FIG. 3, the vehicle of the host vehicle 37 in the real world is positioned at the front and rear positions of the host vehicle 37 on the placement surface (for example, road surface) 38 of the host vehicle 37 in the real world. A pair of first patterns 41 composed of white lines parallel to the width direction are arranged.

  In the present embodiment, as shown in FIG. 3, a pair of second patterns 42 formed of white lines parallel to the vehicle full length direction of the host vehicle 37 at the left and right positions of the host vehicle 37 on the placement surface 38. Place.

  Next, in this state, the display mode at the time of determination is set for the image display processing unit 26 by the input operation unit 36.

  Next, in the setting state of the display mode at the time of this determination, the image display processing unit 26 transmits the own vehicle 37 in which the first pattern 41 and the second pattern 42 are arranged from each of the in-vehicle cameras 21, 22, 23, 24. A captured image obtained by capturing an imaging region around is acquired.

  Next, the image display processing unit 26 performs viewpoint conversion on the captured images of the in-vehicle cameras 21, 22, 23, and 24 using the determination mode time mapping table stored in the storage unit 35, as illustrated in FIG. In the determination mode, the vehicle periphery overhead image 28 is generated and displayed on the display 27.

  In this determination mode, the first pattern 41 parallel to the vehicle width direction on the image 28 (that is, the vehicle width direction of the illustration image 2) is located at positions before and after the illustration image 2 of the own vehicle in the vehicle surrounding overhead image 28. Each image 41 'is included.

  If the image 41 ′ of each first pattern 41 is not parallel to the vehicle width direction on the image 28 but has an inclination, the mounting posture of the in-vehicle cameras 21, 22, 23, 24 at that time However, in the present embodiment, it is assumed that such an inclination of the image 41 ′ of the first pattern 41 does not occur.

Here, in the present embodiment, the image 41 ′ of the first pattern 41 on the front side with respect to the illustration image 2 crosses the front / left side first boundary line _LFL1, and the front viewpoint conversion image 30 in the determination mode. Are displayed in both images 30 and 31 of the left-side viewpoint converted image 31 in the determination mode. Further, the image 41 ′ of the first pattern 41 on the front side crosses the front / right side first boundary line _LFR1, and the front viewpoint conversion image 30 in the determination mode and the right side viewpoint conversion in the determination mode. It is displayed in both images 30 and 32 with the image 32.

Further, in the present embodiment, the image 41 ′ of the first pattern 41 on the rear side with respect to the illustration image 2 is the same as the rear viewpoint conversion image 33 in the determination mode so as to straddle the rear / left side first boundary line _LBL1. It is displayed in both images 33 and 31 with the left-side viewpoint conversion image 31 in the determination mode. In addition, the image 41 ′ of the first pattern 41 on the rear side crosses the rear / right side first boundary line _LBR1, and the rear viewpoint conversion image 33 in the determination mode and the right side viewpoint conversion in the determination mode. It is displayed in both images 33 and 32 with the image 32.

  In the determination mode, the second pattern 42 parallel to the vehicle full length direction on the image 28 (that is, the vehicle full length direction of the illustration image 2) is located at the left and right positions of the own vehicle illustration image 2 in the vehicle periphery overhead image 28. The images 42 'are respectively included.

  If the images 42 ′ of the respective second patterns 42 are not parallel to the vehicle full length direction on the image 28 but have an inclination, the mounting postures of the in-vehicle cameras 21, 22, 23, 24 are at that time. However, for the sake of convenience, it is assumed that such an inclination of the image 42 ′ of the second pattern 42 does not occur in this embodiment.

Here, in the present embodiment, the image 42 ′ of the second pattern 42 on the left side with respect to the illustration image 2 crosses the front / left side second boundary line _LFL2 so as to straddle the front viewpoint conversion image in the determination mode. 30 and the left side viewpoint conversion image 31 in the determination mode. Further, the image 42 ′ of the second pattern 42 on the left side faces the rear / left side second boundary line _{LBL 2} so as to straddle the determination mode rear viewpoint conversion image 33 and the determination mode left side viewpoint. It is displayed in both images 33 and 31 with the converted image 31.

Further, in the present embodiment, the image 42 ′ of the second pattern 42 on the right side with respect to the illustration image 2 crosses the front / right side second boundary line _LFR2, and the front viewpoint conversion image 30 in the determination mode. Are displayed in both images 30, 32 of the right-side viewpoint converted image 32 in the determination mode. In addition, the image 42 ′ of the second pattern 42 on the right side has a rear viewpoint conversion image 33 in the determination mode and a right side viewpoint in the determination mode so as to straddle the rear / right side second boundary line _LBR2. It is displayed in both images 33 and 32 with the converted image 32.

Then, in the display state of the vehicle periphery overhead image 28 in the determination mode, the user can convert the viewpoint converted images 30, 31, 31 adjacent to each other at the positions of the first boundary lines L _FL1 , L _FR1 , L _BL1 , L _BR1 . The presence / absence of displacement in the vehicle full length direction on the image 28 between 32 and 33 is determined based on the vehicle full length direction of the image 41 ′ of the first pattern 41 at the position of the first boundary lines L _FL1 , L _FR1 , L _BL1 , L _BR1. The presence or absence of deviation can be easily and appropriately visually confirmed.

At the same time, the user shifts in the vehicle width direction on the image 28 between the viewpoint conversion images 30, 31, 32, 33 adjacent to each other at the positions of the second boundary lines L _FL2 , L _FR2 , L _BL2 , L _BR2. The presence / absence of the image can be simply and appropriately visually confirmed as the presence / absence of a shift in the vehicle width direction of the image 42 ′ of the second pattern 42 at the position of the second boundary lines L _FL2 , L _FR2 , L _BL2 , L _BR2. it can.

  As a result, when no deviation is confirmed, it is determined that the mounting positions and mounting postures of the in-vehicle cameras 21, 22, 23, and 24 are appropriate. In this case, the actual use mode mapping table is not corrected.

  On the other hand, when the deviation is confirmed, it is determined that the mounting position and the mounting posture of the in-vehicle cameras 21, 22, 23, and 24 are inappropriate, and the determination mode is set so that the confirmed deviation is eliminated. Correct the external parameters in the hour mapping table.

  Then, the actual use mode mapping table is corrected by applying the corrected external parameter to the actual use mode mapping table stored in the storage unit 35.

As described above, according to the present embodiment, the viewpoint corresponding to any one of the in-vehicle cameras 21, 22, 23, and 24 using the first boundary lines _LFL1 , _LFR1 , _LBL1 , and _LBR1. Whether the converted images 30, 31, 32, 33 and the viewpoint converted images 30, 31, 32, 33 corresponding to the other in-vehicle cameras 21, 22, 23, 24 are misaligned in the vehicle full length direction is simply and appropriately confirmed. be able to. Further, the viewpoint conversion images 30, 31, 32, 33 corresponding to any one of the on-vehicle cameras 21, 22, 23, 24 using the second boundary lines L _FL2 , L _FR2 , L _BL2 , L _BR2 and others The presence or absence of a shift in the vehicle width direction from the viewpoint-converted images 30, 31, 32, 33 corresponding to the in-vehicle cameras 21, 22, 23, 24 can be easily and appropriately confirmed. As a result, it is possible to easily and accurately determine whether or not the mounting positions and mounting postures of the in-vehicle cameras 21, 22, 23, and 24 are appropriate.

  In addition, this invention is not limited to embodiment mentioned above, A various change is possible as needed.

The block diagram which shows embodiment of the vehicle periphery monitoring apparatus which concerns on this invention The figure which shows the display state of the vehicle periphery overhead image in determination mode in embodiment of the vehicle periphery monitoring apparatus which concerns on this invention. The figure which shows the arrangement | positioning state of the 1st pattern and 2nd pattern on a mounting surface in embodiment of the camera position and attitude | position determination method which concerns on this invention The figure which shows the display state of the vehicle periphery bird's-eye view image at the time of the determination mode containing the image of the 1st pattern and the image of the 2nd pattern in embodiment of the camera position / posture determination method according to the present invention The figure which shows an example of the vehicle periphery overhead view image conventionally employ | adopted Explanatory drawing for demonstrating the virtual viewpoint of a vehicle periphery overhead view image The figure which shows the photographing picture of the vehicle camera The figure which shows the display state of the vehicle periphery overhead image in which the image of the 1st pattern was contained in the conventional camera position / attitude | position determination method The figure which shows the display state of the vehicle periphery overhead view image in which the image of the 2nd pattern was contained in the conventional camera position and attitude | position determination method

Explanation of symbols

DESCRIPTION OF SYMBOLS 20 Vehicle periphery monitoring apparatus 21 Front camera 22 Left side camera 23 Right side camera 24 Back camera 26 Image display process part 27 Display 28 Vehicle periphery overhead view image at the time of judgment mode 30 Front side viewpoint conversion image at the time of judgment mode 31 Left side at the time of judgment mode Viewpoint conversion image 32 Viewpoint conversion image on the right side in determination mode 33 Viewpoint conversion image on the rear side in determination mode

Claims (4)

A plurality of mounting positions which are mounted at different mounting positions on the vehicle in a mounting posture so as to face a predetermined shooting direction corresponding to the mounting position, and each of the shooting areas corresponding to the shooting direction around the vehicle can be shot. With the camera
A viewpoint that is a viewpoint-converted image in the use range corresponding to each camera in the shot images of the plurality of cameras, such that the image in the use range is seen from a virtual viewpoint above the vehicle. An image in which the surroundings of the vehicle are looked down on from the virtual viewpoint by connecting the viewpoint-converted images obtained by the viewpoint conversion so as to be adjacent to each other with a boundary line between them. A vehicle periphery monitoring device comprising: an image display processing means for generating a vehicle periphery overhead image that is, and displaying the generated vehicle periphery overhead image on a display unit,
The image display processing means includes a first boundary line parallel to the vehicle full-length direction on the vehicle periphery overhead image and a vehicle on the vehicle periphery overhead image as a viewpoint image of the vehicle periphery. A vehicle periphery monitoring device capable of displaying a vehicle periphery overhead view image adjacent to a viewpoint conversion image corresponding to another camera across a second boundary line parallel to the width direction. The image display processing means selectively selects a display mode at the time of actual use and a display mode at the time of determination for determining whether or not the mounting position and mounting posture of the camera are appropriate by visually confirming a vehicle surroundings overhead view image. In the setting state of the display mode at the time of the determination, the viewpoint conversion image corresponding to the arbitrary camera is displayed on the other side across the first boundary line and the second boundary line. A vehicle surroundings overhead view image that is adjacent to the viewpoint conversion image corresponding to the camera is displayed, and in the setting state of the display mode at the time of actual use, the viewpoint conversion image corresponding to the arbitrary camera is A vehicle overhead view image adjacent to a viewpoint conversion image corresponding to the other camera is displayed with only a third boundary line inclined with respect to the vehicle width direction. The vehicle periphery monitoring device according to claim 1. A plurality of mounting positions which are mounted at different mounting positions on the vehicle in a mounting posture so as to face a predetermined shooting direction corresponding to the mounting position, and each of the shooting areas corresponding to the shooting direction around the vehicle can be shot. When determining the suitability of the respective mounting positions and mounting postures, the images in the use ranges corresponding to the cameras in the captured images of the plurality of cameras are indicated by the images in the use ranges. Are converted into viewpoint-converted images, which are viewpoint-converted images so as to be seen from a virtual viewpoint above, and the viewpoint-converted images obtained by the viewpoint conversion are adjacent to each other across a boundary line By connecting to the vehicle, a vehicle surroundings overhead image that is an image of the surroundings of the vehicle as seen from the virtual viewpoint is obtained. The camera position / posture for determining whether or not the camera mounting position and mounting posture are appropriate by displaying the generated vehicle peripheral overhead image on a display unit and visually confirming the displayed vehicle peripheral overhead image. A determination method comprising:
As the vehicle periphery overhead image, a viewpoint conversion image corresponding to an arbitrary camera includes a first boundary line parallel to the vehicle full length direction on the vehicle periphery overhead image and a second boundary parallel to the vehicle width direction on the vehicle periphery overhead image. After displaying the vehicle overhead view image adjacent to the viewpoint conversion image corresponding to the other camera across the boundary line of
The first boundary line is used to check whether or not there is a shift in the vehicle full-length direction between the viewpoint conversion image corresponding to the arbitrary camera and the viewpoint conversion image corresponding to the other camera, and the second boundary By confirming whether or not there is a shift in the vehicle width direction between the viewpoint conversion image corresponding to the arbitrary camera and the viewpoint conversion image corresponding to the other camera using a line, the mounting position and the mounting posture of the camera are determined. A camera position / posture determination method characterized by determining suitability. As the vehicle periphery overhead image, in the viewpoint conversion image corresponding to the arbitrary camera and in the other camera such that the image of the first pattern elongated in the vehicle width direction straddles the first boundary line In the viewpoint conversion image that is displayed in the corresponding viewpoint conversion image and that corresponds to the arbitrary camera such that the image of the second pattern that is long in the vehicle total length direction straddles the second boundary line and After displaying a vehicle surroundings overhead view image as displayed in the viewpoint conversion image corresponding to the other camera,
The image of the first pattern in the viewpoint conversion image corresponding to the arbitrary camera at the position of the first boundary line and the image of the first pattern in the viewpoint conversion image corresponding to the other camera. The vehicle full length direction of the viewpoint conversion image corresponding to the arbitrary camera using the first boundary line and the viewpoint conversion image corresponding to the other camera by confirming whether or not there is a shift in the vehicle full length direction Check whether there is any deviation,
The image of the second pattern in the viewpoint conversion image corresponding to the arbitrary camera at the position of the second boundary line and the image of the second pattern in the viewpoint conversion image corresponding to the other camera. The vehicle width direction of the viewpoint conversion image corresponding to the arbitrary camera using the second boundary line and the viewpoint conversion image corresponding to the other camera by checking the presence or absence of the shift in the vehicle width direction The camera position / posture determination method according to claim 3, wherein the presence or absence of deviation is confirmed.

Images