JP5317655B2 - Vehicle driving support device and vehicle driving support method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a vehicle-driving support device and vehicle-driving support method, capable of readily and properly verifying the states of left and right side areas, with respect to own vehicle. <P>SOLUTION: The vehicle-driving support device is provided with: an attitude detector 17 for detecting the attitude of a vehicle with respect to an object included in a photographed video image 3 of an on-board camera 11; an image generator 18 for generating such first image 1 and second image 2, which include areas needed to confirm the states, irrespective of the attitude of the vehicle so that the angles of the object are mutually left-right symmetrical in displayed states, on the basis of the detection result of the attitude detector 17; and an image compositing device 20 for compositing the images 1 and 2 generated by the image generator 18 and displaying the image composited in a displaying part 14. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a vehicle driving support device and a vehicle driving support method, and in particular, supports driving of a vehicle by displaying images for confirming the situation of left and right side regions with respect to the front end of the vehicle. The present invention relates to a vehicle driving support device and a vehicle driving support method suitable for the above.

  In recent years, various images for confirming the situation of the surroundings of the host vehicle are generated using images shot by an in-vehicle camera equipped with a super-wide-angle lens such as a fisheye lens, and the generated images are displayed on a display inside the vehicle. As a result, a system that supports the safe and smooth driving of the host vehicle by reducing the blind spots around the host vehicle has been adopted.

  As one mode of image display by such a system, as shown in FIG. 15, a first image 1 for confirming the state of the left side region (region in the real world) with respect to the front end of the host vehicle. And the second image 2 for confirming the situation of the right side area (real world area) with respect to the front end of the host vehicle are simultaneously displayed in a state where the display area 4 of the display is juxtaposed side by side. Such images 1 and 2 were used particularly for safety confirmation when approaching an intersection.

  Here, the two images 1 and 2 shown in FIG. 15 are obtained by an in-vehicle camera (not shown) mounted on the own vehicle when the own vehicle enters a T-junction intersection as shown in FIG. A shooting region R having a wide viewing angle from the front of the host vehicle to the left and right sides of the front end of the host vehicle as shown in FIG. 16 is shot and generated using the shot image 3 shown in FIG. 17 obtained by the shooting. It has been done.

  That is, the first image 1 shown in FIG. 15 is generated by cutting out the video 5 within a predetermined range from the photographed video 3 of FIG.

  Further, the second image 2 shown in FIG. 15 is also generated by cutting out a video 6 within a predetermined range from the photographed video 3 shown in FIG.

The first image 1 generated in this way includes an approach destination roadway when the host vehicle turns left at the intersection of FIG. 16, a sidewalk along the roadway, and a curb. Further, the second image 2 includes a roadway behind the approaching roadway when the host vehicle turns right at the intersection of FIG. 16, a sidewalk and a curb along this roadway.

  Therefore, the user can confirm the traffic of the person and the vehicle from the left and right of the own vehicle at the intersection by looking at these images 1 and 2.

JP 2007-299047 A

  However, conventionally, both the video 5 used for generating the first image 1 in the captured image 3 of the in-vehicle camera and the video 6 used for generating the second image 2 in the captured image 3 of the in-vehicle camera are both in advance. Since the image was fixed within a set range, depending on the position of the host vehicle with respect to objects around the host vehicle, the situation of the area that the user wanted to check could not be checked easily and appropriately. .

  For example, as shown in FIG. 18, when the own vehicle enters an intersection similar to that shown in FIG. 16 with an inclination in the direction of the road to be entered (left direction in FIG. 18), As shown in FIG. 18, the imaging region R of the in-vehicle camera is also tilted in the direction of the destination road from that shown in FIG. 16. As a result, as shown in FIG. 19, a large image of wrinkles present on the left side of the host vehicle that was not included in the case of FIG.

  Conventionally, even in such a case, the video 5 used to generate the first image 1 is fixed to the video 5 in the same coordinate range (pixel area) as shown in FIG. In addition, the video 6 used to generate the second image 2 is also fixed to the video 6 within the same coordinate range as that shown in FIG.

  Accordingly, the first image 1 generated using the captured video 3 shown in FIG. 19 has a high ratio of unnecessary wrinkles existing on the left side of the host vehicle as shown in FIG. Since the range included in the image 1 is inadequate for a roadway or a sidewalk of an approach destination, it is difficult to appropriately confirm the situation around the entrance road.

  Further, in the first image 1 and the second image 2 shown in FIG. 20, the angles of the objects on the images 1 and 2 are asymmetric with each other in the display state on the display. Specifically, the first image 1 and the second image 2 shown in FIG. 20 both include white lines of sidewalks that are located on a straight line in the real world, but in FIG. The angle formed by the longitudinal direction of the white line of the sidewalk in the first image 1 with respect to the image width direction (provided that the clockwise direction in FIG. 20 is positive) is the longitudinal direction of the white line of the sidewalk in the second image 2. The angle is larger than the angle formed with respect to the horizontal direction of the image (however, the angle in which the counterclockwise direction in FIG. 20 is positive). That is, in FIG. 20, the first image 1 has a larger object angle on the image than the second image 2.

  Therefore, it is difficult for the user to match the direction of traffic at the intersection in the real world and the direction of traffic on the images 1 and 2, and the right and left traffic status at the intersection can be easily and appropriately confirmed. Has become difficult.

  Such a situation is not limited to the case where the host vehicle enters with the T-junction or the intersection of the crossroads tilted. For example, as shown in FIG. This also occurred when the host vehicle entered a straight road at the intersection of a road whose shape was asymmetric with respect to the approaching road.

  Therefore, the present invention has been made in view of such a point, and a vehicle driving support device and a vehicle driving support capable of simply and appropriately confirming the situation of the left and right side regions with respect to the front end portion of the host vehicle. It is intended to provide a method.

In order to achieve the above-described object, a vehicle driving support device according to the present invention is mounted on a vehicle and can shoot a wide viewing angle shooting region extending from the front of the vehicle to the left and right sides of the front end of the vehicle. The first image for confirming the situation of the left side area with respect to the front end portion of the vehicle and the situation of the right side area with respect to the front end portion of the vehicle on the basis of the in-vehicle camera and the captured image of the in-vehicle camera An image display processing device that generates a second image for confirmation and simultaneously displays the generated first image and the second image in a state of being arranged side by side on the display unit, a vehicle driving support device that is possible to assist the driving of the vehicle by displaying an image and the second image, the image display processing apparatus is captured by the onboard camera That the shape that is preset as indicating the posture of the vehicle included in the video, by approximating the shape shown in the captured image in the actual captured the captured regions by the onboard camera, the shooting Regardless of the posture of the vehicle, the posture detection device detects the posture of the vehicle with respect to the object included in the image, and the first image and the second image based on the detection result of the posture detection device. An image generation device that always includes an area that is considered to require a situation check, and that generates the first image and the second image, respectively, such that the angle of the object on the image is bilaterally symmetrical in the display state; An image synthesizing device for synthesizing the first image and the second image generated by the image generating device and displaying the synthesized image on the display unit; It is characterized in that was example.

  According to such a configuration, the image display processing device always includes an area that is considered necessary for the situation check regardless of the vehicle posture based on the detection result of the posture detection device, and the object on the image. The first image and the second image can be displayed so that their angles are symmetrical with each other in the display state, so that the situation of the left and right side regions with respect to the front end portion of the host vehicle can be confirmed easily and appropriately. can do.

  In addition, the image generation device includes a first original image that is a source of the first image that belongs to a range corresponding to a detection result of the posture detection device, from the captured video of the vehicle-mounted camera, and the posture detection. The second original image that is the original of the second image that belongs to the range according to the detection result of the device is acquired respectively, and the preset first image is acquired with respect to the acquired first original image The vertical and horizontal dimensions are changed so as to match the first display range in which the first image is displayed on the display unit, and the preset display is performed on the acquired second original image. By changing the vertical and horizontal dimensions to match the second display range in which the second image is displayed in the unit, the angles of the objects on the image are symmetrical with each other in the display state. 1 image and the first Images of may be formed so as to respectively generate.

  According to such a configuration, the image generation apparatus changes the vertical and horizontal dimensions of the first original image and the second original image, so that the angles of the objects on the image are mutually changed in the display state. Since the first image and the second image that are symmetric can be generated, the processing required for generating the first image and the second image can be simplified.

Further, the vehicle driving support apparatus main body stores a first display range that is a first display range that is set in advance and that is located on the left side in the display area of the display unit. A display range storage device, and a second display that is a second display range that is set in advance and has the same vertical and horizontal dimension ratio as the first display range that is positioned on the right side of the display area of the display unit A second display range storage device in which range information is stored, and the vehicle-mounted camera serving as a reference for displaying the first image virtually located in the first display range set in advance A first display reference line segment having an elliptical arc shape indicating a shape set in advance as indicating the posture of the vehicle included in a video that can be shot , the left end portion of which is the left end side of the first display range Is divided at a predetermined ratio and A first display reference line segment storage device that stores information on a first display reference line segment whose end reaches the lower right end of the first display range, and the preset second display An elliptical arc shape indicating a shape set in advance as indicating the posture of the vehicle included in an image that can be taken by the in-vehicle camera that is a reference when displaying the second image that is virtually located in the range The second display reference line segment, the right end portion of which splits the right end side of the second display range at the predetermined ratio, and the left end portion reaches the lower left end portion of the second display range. A second display reference line segment storage device in which information of the second display reference line segment is stored, and a first for acquiring the first original image in a preset imaging range of the in-vehicle camera. The in-vehicle camera in the imaging range A first angle range storage device storing information on a first angle range centered on a point corresponding to the optical axis, and the second original image in a preset imaging range of the in-vehicle camera are acquired. And a second angle range storage device in which information on the second angle range centered on a point corresponding to the optical axis of the in-vehicle camera in the imaging range is stored. The posture detection device extracts a plurality of feature lines that are regarded as a break of an object in the captured video from the shape shown in the captured video actually captured by the in- vehicle camera; Corresponding to the optical axis of the in-vehicle camera taken on the image of the feature lines from the left to the center side in the image of the feature lines extracted by the feature line extraction device Points to do In the first angle range centered on the same as the first elliptical arc-shaped first display reference line segment, which is set in advance as indicating the posture of the vehicle included in an image that can be photographed by the in-vehicle camera a first search unit that sequentially searches the first line segment which can be approximated to a first elliptical arcuate segment showing the shape that is, from the characteristic lines extracted by the characteristic line extraction unit In a second angle range centered on a point corresponding to the optical axis of the in-vehicle camera taken on the image made of the feature lines of the feature lines from the right side to the center side in the image to be the second ellipse arc showing the shape that is preset as indicating the posture of the vehicle included in the second image may be captured by the elliptical arc-shaped second display reference line segment as well as the in-vehicle camera Approximate line A second search device that sequentially searches for a second line segment that can be detected, and detects the attitude of the vehicle as a search result of the first search device and the second search device. is formed, the image generation apparatus, on which the first line segment retrieved by the first retrieval apparatus is approximated to the first elliptical arcuate segment, the first segment after the approximation And the first angle range stored in the first angle range storage device, a rectangular first acquisition range for acquiring the first original image from the image composed of the feature lines As a first elliptic arc included in the first ellipse at the lower right end thereof, the first elliptic arc included in the first ellipse and corresponding to the first angle range and including the approximated first line segment The right end portion and the lower end portion are positioned, and the left end side of the first line segment after the approximation is First acquisition range extraction unit for extracting a first acquisition range as to divide the serial at a predetermined ratio, the second line segment retrieved by the second search unit of the second ellipse arc Based on the second line segment after the approximation and the second angle range stored in the second angle range storage device after being approximated to the line segment, from the image consisting of the characteristic line, A rectangular second acquisition range for acquiring the second original image is a second elliptical arc included in the second ellipse at the lower left end thereof and corresponds to the second angular range. The left and lower ends of the second elliptical arc including the approximated second line segment are located, and the approximated second line segment is divided at the predetermined ratio by dividing the right end side thereof. A second acquisition range extracting device for extracting a second acquisition range, and a captured video of the in-vehicle camera A video corresponding to the first acquisition range extracted by the first acquisition range extraction device is acquired as the first original image, and the first original image is acquired with respect to the acquired first original image. By the first original image size changing device that generates the first image by changing the vertical and horizontal dimensions so as to fit the display range, and the second acquisition range extracting device in the captured video of the in-vehicle camera Vertical and horizontal dimensions so that the extracted video corresponding to the second acquisition range is acquired as the second original image, and the acquired second original image fits the second display range. And a second original image size changing device that generates the second image by making the above change.

  According to such a configuration, the attitude detection device acquires the attitude of the vehicle as a search result of the first line segment and the second line segment using the feature line extracted from the captured video of the in-vehicle camera. Then, the image generation device acquires the first original image using the first acquisition range extracted based on the first line segment and the preset first angle range, and the second The original image is acquired using the second acquisition range extracted based on the second line segment and the preset second angle range, and the vertical and horizontal dimensions are changed with respect to the first original image. The first image can be generated by performing the above, and the second image can be generated by changing the vertical and horizontal dimensions with respect to the second original image. Therefore, the first image and the second image can be generated. Can be easily and appropriately generated by image processing.

  Furthermore, the feature line extraction device may be formed so as to extract, as the feature line, a portion where a color difference with an adjacent pixel in a captured image of the in-vehicle camera is equal to or greater than a threshold value.

  And according to such a structure, the detection of the attitude | position of the vehicle by an attitude | position detection apparatus is performed by image processing by extracting as a feature line the location where the color difference with the adjacent pixel in the picked-up image of a vehicle-mounted camera is more than a threshold value. It can be carried out simply and appropriately.

  In addition, the feature line extraction device thickens the extracted feature line once, interpolates the cut of the feature line, and then thins the thick feature line to return to the original thickness. Thus, the feature line in a state where there is no break having a predetermined number of pixels or less may be extracted.

  According to such a configuration, it is possible to extract the feature line in which the discontinuity is interpolated, so that it is possible to more reliably detect the vehicle posture by the posture detection device.

  Further, the first search device is configured to search the first line segment in a search range corresponding to the first angle range, and the second search device is configured to search for the second line. The second line segment may be searched in a search range corresponding to the angle range.

  According to such a configuration, the search range of the first line segment by the first search device and the search range of the second line segment by the second search device are limited to the necessary minimum ranges, respectively. Therefore, the first image and the second image can be generated quickly.

  Furthermore, when the first acquisition range extraction device extracts the first acquisition range, the first acquisition range extraction device extracts a range having the same width as the first elliptic arc including the entire first elliptic arc. In addition, when the first line segment after the approximation reaches the left end side in the range where the horizontal width is the same, the first line segment after the approximation with respect to the range where the horizontal width is the same. The first acquisition range is extracted only by limiting the vertical width in consideration of the division position of the left end side of the first acquisition range, and the left end side in the extracted range having the same horizontal width is the post-approximation When the position is located on the left side of the left end portion of the first line segment, the range on the left side of the approximated left end portion of the first line segment is removed from the range having the same horizontal width. The width is limited by the new width obtained by the width limitation. The first acquisition range is extracted by limiting the vertical width of the first range after considering the division position of the left end side of the first acquisition range by the first line segment after the approximation. When the second acquisition range extraction device extracts the second acquisition range, the second acquisition range extraction device extracts and extracts a range having the same width as the second elliptic arc including the entire second elliptic arc. When the second line segment after the approximation reaches the right end side in the range where the horizontal width is the same, the second line segment after the approximation with respect to the range where the horizontal width is the same. The second acquisition range is extracted only by limiting the vertical width in consideration of the division position of the right end side of the second acquisition range, and the right end side in the extracted range having the same horizontal width is the post-approximation If it is located on the right side of the right end of the second line segment, For a range having the same width, the lateral width is limited by removing a range on the right side of the right end portion of the second line segment after the approximation, and a new range obtained by the lateral width limitation. On the other hand, the second acquisition range is extracted by limiting the vertical width in consideration of the division position of the right end side of the second acquisition range by the approximated second line segment. It may be.

  And according to such composition, when extracting the 1st acquisition range with the 1st acquisition range extraction device, the range located in the left of the left end part of the approximated first line segment is When approximation is performed when the second acquisition range is extracted by the second acquisition range extraction device by excluding it from the first acquisition range as a range unnecessary for confirmation of the situation of the left side region with respect to the front end of the host vehicle Confirming the situation by excluding the range located to the right of the right end of the second line segment from the second acquisition range as a range unnecessary for confirming the situation of the right side of the front end of the host vehicle Thus, it is possible to obtain the first image and the second image in which unnecessary areas are sufficiently reduced.

Further, the vehicle driving support method according to the present invention captures a wide viewing angle imaging region from the front of the vehicle to the left and right sides of the front end of the vehicle using an in-vehicle camera mounted on the vehicle, A first image for confirming the state of the left side region with respect to the front end portion of the vehicle and a second for confirming the state of the right side region with respect to the front end portion of the vehicle based on the captured image of the camera A vehicle driving support method for supporting driving of the vehicle by generating an image and simultaneously displaying the generated first image and second image on the display unit as being arranged side by side on the left and right sides. In the photographing region actually photographed by the in-vehicle camera with respect to a shape set in advance as indicating the posture of the vehicle included in an image that can be photographed by the camera By approximating the shape shown in shadow image, and detects the posture of the vehicle relative to the object included in the captured image, based on the attitude of said detected vehicle, the first image and the second image As described above, the first image and the second image that always include a region that is regarded as necessary for checking the situation regardless of the posture of the vehicle and that the angles of the objects on the image are symmetrical with each other in the display state. Each image is generated, and the generated first image and second image are combined and displayed on the display unit.

  And according to such a method, based on the detection result of the posture of the vehicle, it always includes a region regarded as necessary for the situation check regardless of the posture of the vehicle, and the angle of the object on the image is in the display state. Since the first image and the second image that are bilaterally symmetric can be displayed, the situation of the left and right side regions with respect to the front end portion of the host vehicle can be easily and appropriately confirmed.

  Further, from the captured video of the in-vehicle camera, the first original image that is the source of the first image that belongs to the range corresponding to the detected posture of the vehicle, and the detected posture of the vehicle A second original image that is a source of the second image that belongs to the selected range, and the first original image in the display unit that is set in advance for the acquired first original image. The vertical and horizontal dimensions are changed so as to fit the first display range in which the image of the second image is displayed, and the second original image in the display unit set in advance is applied to the acquired second original image. By changing the vertical and horizontal dimensions so as to match the second display range in which the image is displayed, the first image and the first image in which the angles of the objects on the image are symmetrical to each other in the display state Generate two images respectively It may be.

  According to such a method, by changing the vertical and horizontal dimensions of the first original image and the second original image, the angles of the objects on the image are symmetrical with each other in the display state. Since the first image and the second image can be respectively generated, the processing required for generating the first image and the second image can be simplified.

Furthermore, the first display range is a rectangular first display range located on the left side of the display area of the display unit, and the second display range is on the right side of the display area of the display unit. A rectangular second display range having the same vertical / horizontal size ratio as the first display range, and a reference for displaying the first image virtually located in the first display range; The first display reference line segment having an elliptical arc shape indicating a shape set in advance as an indication of the posture of the vehicle included in the video that can be taken by the in-vehicle camera , the left end portion of which is the first display reference line A first display reference line segment in which the left end side of the display range is divided at a predetermined ratio and the right end portion reaches the lower right end portion of the first display range, and the second display range is virtually included in the second display range. as a reference for displaying the second image located Serial and a second display reference line elliptic arc shaped showing the shape that is preset as indicating the posture of the vehicle included in the video that may be captured by the onboard camera, display the right end of the second A second display reference line segment in which a right end side of the range is divided at the predetermined ratio and a left end portion thereof reaches a lower left end portion of the second display range; and the first in the imaging range of the in-vehicle camera. A first angle range for acquiring an original image of the first vehicle, the first angle range centering on a point corresponding to the optical axis of the vehicle-mounted camera in the imaging range, and the imaging range of the vehicle-mounted camera A second angle range for acquiring the second original image, and a second angle range centered on a point corresponding to the optical axis of the in-vehicle camera in the imaging range; In fact by the onboard camera From among the shape shown in captured photographed image, after extracting a plurality of characteristic lines which are considered the object breaks in the captured image, from the left toward the center of the extracted the consisting feature-ray image On the other hand, the first elliptical arc is within a first angle range centered on a point corresponding to the optical axis of the in-vehicle camera taken on the image composed of the feature lines among the extracted feature lines. approximating the first ellipse arcuate segment showing the shape that is preset as indicating the posture of the vehicle included in a video that can be taken by Jo of the first display reference line segment as well as the in-vehicle camera The first line segment that can be searched is sequentially searched, and from the right side to the center side of the extracted image of the feature lines, the extracted feature lines are displayed on the image of the feature lines. Taken In the second angle range centered on a point corresponding to the optical axis of the in- vehicle camera, the image can be captured by the in-vehicle camera in the same manner as the second elliptical arc-shaped second display reference line segment. by sequentially searching for a second line segment which can be approximated to the second ellipse arcuate segment showing the shape that is preset as indicating the orientation of the vehicle to be the attitude of the vehicle, in terms of the first line segment and is detected as the search result of the second line segment, and the retrieved said first line segment to approximate to the first elliptical arcuate segments, first after approximation Based on one line segment and the first angle range set in advance, a rectangular first acquisition range for acquiring the first original image from the image including the feature lines is At the lower end, the first elliptic arc included in the first ellipse, A first elliptical arc corresponding to an angle range of 1 and including the first line segment after approximation is positioned at a right end and a lower end thereof, and the first line segment after approximation is positioned on the left end side thereof. extracting first acquisition range as to divide by the predetermined ratio, in terms of the retrieved second line segment is approximated to the second ellipse arcuate segment, a second after approximation Based on the line segment and the preset second angle range, the lower left end of the rectangular second acquisition range for acquiring the second original image from the image composed of the feature lines And a second elliptical arc included in the second ellipse, corresponding to the second angular range and including the second line segment after approximation, the left end and the lower end of the second elliptical arc Is located, and the right end side of the second line segment after the approximation is divided by the predetermined ratio An acquisition range is extracted, and a video corresponding to the first acquisition range extracted in the captured video of the in-vehicle camera is acquired as the first original image, and the acquired first original image is compared with the acquired first original image. The first image is generated by changing the vertical and horizontal dimensions so as to be adapted to the first display range, and the video corresponding to the second acquisition range extracted in the video captured by the in-vehicle camera is displayed. Obtaining the second original image, and generating the second image by changing the vertical and horizontal dimensions so as to fit the second display range with respect to the acquired second original image. Also good.

  According to such a method, the posture of the vehicle is acquired as a search result of the first line segment and the second line segment using the feature line extracted from the captured image of the in-vehicle camera, and the first An original image is acquired using a first acquisition range extracted based on a first line segment and a preset first angle range, and the second original image is defined as a second line segment. The first image is generated by using the second acquisition range extracted based on the preset second angle range and changing the vertical and horizontal dimensions with respect to the first original image. Since the second image can be generated by changing the vertical and horizontal dimensions of the second original image, the first image and the second image are generated easily and appropriately by image processing. be able to.

  Moreover, you may extract the location where the color difference with the adjacent pixel in the picked-up image of the said vehicle-mounted camera becomes more than a threshold value as the said feature line.

  And according to such a method, the detection of the posture of the vehicle is simply and appropriately performed by image processing by extracting a part where the color difference with the adjacent pixel in the captured image of the in-vehicle camera is equal to or larger than the threshold value as a feature line. It can be carried out.

  Further, after extracting the feature line once thick and interpolating the discontinuity of the feature line, by thinning the thick feature line and returning it to the original thickness, a predetermined number of pixels or less is obtained. You may extract the said feature line of a state which does not have a break.

  And according to such a method, since the characteristic line in which the interruption was interpolated can be extracted, the posture of the vehicle can be detected more reliably.

  Furthermore, the first line segment is searched in a search range corresponding to the first angle range, and the second line segment is searched in a search range corresponding to the second angle range. May be.

  According to such a method, the search range of the first line segment and the search range of the second line segment can be limited to the necessary minimum ranges, respectively. Images can be generated quickly.

  Further, when extracting the first acquisition range, a range having the same horizontal width as that of the first elliptic arc including the entire first elliptic arc is extracted, and on the left end side in the range where the extracted horizontal width is the same If the first line segment after the approximation has reached, the division position of the left end side of the first acquisition range by the first line segment after the approximation with respect to the range having the same horizontal width The first acquisition range is extracted only by limiting the vertical width in consideration of the width, and the left end side in the extracted range having the same horizontal width is more than the left end portion of the first line segment after the approximation In the case of being located on the left side, the width is limited by removing the range on the left side from the left end portion of the first line segment after the approximation for the range having the same width. To the new range obtained by the limitation of the first When extracting the first acquisition range by extracting the first acquisition range by limiting the vertical width in consideration of the division position of the left end side of the first acquisition range by the minute, when extracting the second acquisition range, the second When the second elliptical arc including the entire elliptical arc is extracted in a range having the same width as the second elliptical arc, and the second line segment after the approximation has reached the right end side in the extracted horizontal width in the same range The second acquisition is performed by limiting only the vertical width in consideration of the division position of the right end side of the second acquisition range by the second line segment after the approximation with respect to the range having the same horizontal width. When the range is extracted and the right end side in the extracted range having the same width is located on the right side of the right end of the second line segment after the approximation, , Remove the range on the right side of the right end of the approximated second line segment The width of the second acquisition range is further considered with respect to the new range obtained by the limitation of the width, and the division position of the right end side of the second acquisition range by the second line segment after the approximation is considered. The second acquisition range may be extracted by limiting the vertical width.

  And according to such a method, when extracting the 1st acquisition range, the range located in the left side of the left end part of the 1st line segment is the situation of the area on the left side with respect to the front end part of the own vehicle. When the second acquisition range is extracted as a range unnecessary for the confirmation of the first acquisition range, the range located to the right of the right end portion of the second line segment is set to the front end portion of the host vehicle. By excluding from the second acquisition range as a range unnecessary for the confirmation of the situation of the right side area, it is possible to obtain the first image and the second image in which the area unnecessary for the situation confirmation is sufficiently reduced. .

  According to the vehicle driving support device and the vehicle driving support method according to the present invention, it is possible to easily and appropriately confirm the situation of the left and right side regions with respect to the front end portion of the host vehicle.

  Hereinafter, an embodiment of a vehicle driving support apparatus according to the present invention will be described with reference to FIGS.

  Note that portions having the same or similar basic configuration as those in the related art will be described using the same reference numerals.

As shown in FIG. 1, the vehicle driving support device 10 according to the present embodiment includes a single vehicle mounted on the host vehicle as being mounted at a predetermined mounting position of the host vehicle (for example, the center at the front end of the host vehicle). An in-vehicle camera 11 is provided. This in-vehicle camera 11 includes a super-wide-angle lens such as a fish-eye lens and an image sensor such as a CCD or a CMOS. Similarly to the one shown in FIG. An imaging region R (see FIG. 16) having a wide viewing angle extending on both sides is imaged. In this super wide-angle lens, a line segment orthogonal to the optical axis of the lens, that is, a line segment orthogonal to the front-rear direction of the vehicle is photographed in an elliptical shape. When a plurality of perpendicular line segments exist in parallel, each is photographed as a group of concentric elliptical arcs. The orthogonal line segment near the vehicle becomes a large elliptical arc, and the orthogonal line segment far from the vehicle becomes a small elliptical arc.

  As shown in FIG. 1, an in-vehicle camera 11 is connected to an image display processing unit 12 as an image display processing device, and the image display processing unit 12 captures a video image taken by the in-vehicle camera 11. To get from.

  And the image display process part 12 is the 1st for confirming the condition of the area | region of the left side with respect to the front-end part of the own vehicle based on the picked-up image acquired from the vehicle-mounted camera 11, similarly to what was shown in FIG. Image 1 and a second image 2 for confirming the situation of the region on the right side with respect to the front end portion of the host vehicle.

  Further, as shown in FIG. 1, a display 14 as a display unit is connected to the image display processing unit 12, and the image display processing unit 12 generates the generated first image 1 and second image 2. Are simultaneously displayed as a state of being arranged in the left and right directions in the display area 4 (see FIG. 15) of the display 14, thereby supporting the driving of the host vehicle. Specifically, the image display processing unit 12 displays the first image 1 on the left side in the display area 4 and the second image 2 on the right side in the display area 4 as shown in FIG. It has become.

  The image display processing unit 12 is a functional block of a CPU that functions to display the first image 1 and the second image 2 by executing a program stored in a memory such as a ROM (not shown). There may be.

  Furthermore, as shown in FIG. 1, the image display processing unit 12 is connected to a control unit 15 including a CPU functional block, and the control unit 15 performs image display processing by the image display processing unit 12. Is to control. That is, the vehicle speed pulse of the host vehicle is input to the control unit 15, and the control unit 15 displays the image display processing unit when the vehicle speed of the host vehicle indicated by the vehicle speed pulse is equal to or lower than a predetermined speed. 12, a first image and a second image are generated and displayed.

  As shown in FIG. 1, a storage unit 16 is connected to the image display processing unit 12, and the image display processing unit 12 has a first image 1 and a second image 2 connected to the storage unit 16. Various information used to generate the is stored.

  The image display processing unit 12 will be described in further detail. As shown in FIG. 2, the image display processing unit 12 includes a posture detection unit 17 as a posture detection device. The posture detection unit 17 includes an in-vehicle camera. 11 is input.

Then, the attitude detection unit 17 detects the attitude of the host vehicle with respect to the object included in the captured video based on the input captured video of the in-vehicle camera 11, and outputs the detection result to the subsequent stage. Specifically, the attitude detection unit 17 actually uses the in-vehicle camera 11 for a shape (elliptical shape) preset as indicating the attitude of the host vehicle included in the video that can be captured by the in-vehicle camera 11. By approximating the shape shown in the captured image in the captured image area, the posture of the host vehicle with respect to the object included in the captured image is detected.

  As shown in FIG. 2, the image display processing unit 12 includes an image generation unit 18 as an image generation device connected to the subsequent stage of the posture detection unit 17, and the image generation unit 18 includes a posture. The detection result of the attitude of the host vehicle output from the detection unit 17 is input.

  Then, based on the detection result of the input posture detection unit 17, the image generation unit 18 sets, as the first image and the second image, regions that are considered necessary for the situation check regardless of the posture of the host vehicle. A first image and a second image are generated so that they are always included and the angle of the object on the image is bilaterally symmetric (line symmetric) in the display state.

  Specifically, for example, as shown in FIG. 16, the image generation unit 18, as shown in FIG. 16, when the host vehicle enters a straight intersection at a T-shaped road or a crossroad, As shown in FIG. 1, the first image 1 in which only the area considered necessary for the situation confirmation on the left side with respect to the front end portion of the host vehicle is displayed in a sufficiently large size, and the situation on the right side with respect to the front end portion of the host vehicle. A second image 2 in which only a region deemed necessary for confirmation is displayed in a sufficiently large size is generated. Note that the first image 1 and the second image 2 are substantially symmetrical with respect to each other in the display state on the display 14 in terms of the angle of the object on the image (in other words, the inclination). For example, as shown in FIG. 15, the first image 1 is an image 1 in which the white line (object) of the sidewalk on the image 1 is inclined by a predetermined angle clockwise with respect to the image width direction. The second image 2 is an image 2 in which the white line (object) of the sidewalk on the image 2 is inclined at the same angle as the predetermined angle counterclockwise with respect to the image width direction.

  Further, for example, as illustrated in FIG. 18, the image generation unit 18 may enter when the host vehicle enters the intersection of the T-shaped road or the cross road while leaning leftward (steering the steering wheel in the left turn direction). As shown in FIG. 21, when the host vehicle enters the intersection of the Y-shaped road in a straight line state, unlike the conventional images 1 and 2 as shown in FIG. 20, as shown in FIG. A first image 1 and a second image 2 suitable for confirming the situation of the left and right regions with respect to the front end of the host vehicle are generated.

  Here, in the first image 1 shown in FIG. 3, an unnecessary object (for example, an obstacle such as a bag) is sufficiently reduced in an area that is considered necessary for checking the situation on the left side of the front end of the host vehicle. The image 1 is displayed in a sufficiently large size. In addition, the second image 2 shown in FIG. 3 is also displayed in a sufficiently large area in which an unnecessary object is sufficiently reduced in an area considered necessary for checking the situation on the right side with respect to the front end of the host vehicle. The image 2 is displayed. Further, the first image 1 and the second image 2 shown in FIG. 3 are substantially symmetrical with respect to each other in the display state on the display 14 in the same manner as that shown in FIG. ing.

  Then, the image generation unit 18 outputs the generated first image 1 and second image 2 to the subsequent stage.

  Further, referring back to FIG. 2, the image display processing unit 12 includes an image composition unit 20 as an image composition device connected to a subsequent stage of the image generation unit 18. The image composition unit 20 includes an image generation unit 20. The first image 1 and the second image 2 output from the unit 18 are input.

  Then, the image synthesizing unit 20 synthesizes the input first image 1 and second image 2 and outputs them to the display 14, whereby the first image 1 as shown in FIG. 15 or FIG. A composite image in which the second image 3 is juxtaposed side by side is displayed on the display 14.

  Therefore, according to the present embodiment, the image display processing unit 12 always includes a region that is considered necessary for the situation check regardless of the vehicle posture based on the detection result of the posture detection unit 17, and on the image. Since the first image 1 and the second image 2 in which the angle of the object is substantially bilaterally symmetric can be displayed, the user can easily determine the situation of the left and right side areas with respect to the front end of the host vehicle. And it can be confirmed appropriately.

In addition to the above configuration, in the present embodiment, the image generation unit 18 further responds to the detection result of the posture detection unit 17 from the captured video 3 of the vehicle-mounted camera 11 similar to that shown in FIGS. 17 and 19. A first original image that is a source of the first image 1 that belongs to a range (a first acquisition range described later) is acquired, and a display 14 that is set in advance for the acquired first original image. By changing the vertical and horizontal dimensions so as to fit the first display range 21 (see FIG. 4) in which the first image 1 is displayed in the display area 4, the angle of the object on the image is adjusted. The first image 1 is generated.

Further, the image generation unit 18 is a source of the second image 2 belonging to a range (second acquisition range described later) according to the detection result of the posture detection unit 17 from the captured video 3 of the in-vehicle camera 11. A second original image is acquired, and a second display range 22 (FIG. 4) in which the second image 2 in the display area 4 of the display 14 set in advance is displayed with respect to the acquired second original image. The second image 2 with the adjustment of the angle of the object on the image is generated by changing the vertical and horizontal dimensions so as to conform to the reference).

In this way, the image generation unit 18 generates the first image 1 and the second image 2 in which the angle of the object on the image is symmetrical to each other in the display state on the display 14 . Yes.

  Therefore, according to this embodiment, the angle adjustment of the object on the first image 1 and the second image 2 is performed in the vertical and horizontal directions with respect to the first original image and the second original image in the generation stage of the images 1 and 2. Since it can be performed by changing the dimensions, the processing required to generate the first image 1 and the second image 2 can be simplified, and the generation of the images 1 and 2 can be speeded up.

  In addition to the above configuration, in the present embodiment, the storage unit 16 is configured to function as a first display range storage device. 4 is stored, that is, information on the rectangular first display range 21 located on the left side in the display area 4 is stored.

  Further, the storage unit 16 functions as a second display range storage device, and the storage unit 16 functioning in this way has the second display range 22 shown in FIG. Information of a rectangular second display range 22 having the same vertical / horizontal dimension ratio as the first display range 21 located on the right side of the display area 4 (in the present embodiment, the dimensions themselves are also the same) is stored.

  Further, the storage unit 16 functions as a first display reference line segment storage device, and the storage unit 16 functioning in this way has a preset elliptical arc shape as shown in FIG. Information on the first display reference line segment 23 is stored.

Here, the first display reference line segment 23 has a shape (1 in a group of concentric elliptical shapes) set in advance as indicating the posture of the host vehicle included in the video that can be taken by the in-vehicle camera 11. Specifically, the shape is determined by the lens characteristics of the super-wide-angle lens , and is a line segment that is virtually located in the first display range 21, and as shown in FIG. The left end of the first display range 21 is divided into two parts by a: b (a and b are positive real numbers) as a predetermined ratio, and the right end is the right of the first display range 21. The line segment extends to the lower end (in other words, the lower right corner).

  Furthermore, the storage unit 16 functions as a second display reference line segment storage device, and the storage unit 16 functioning in this way has a preset elliptical arc shape as shown in FIG. The information of the second display reference line segment 24 is stored.

Here, the second display reference line segment 24 has a shape (1 in a group of concentric elliptical shapes) set in advance to indicate the posture of the host vehicle included in the video that can be taken by the in-vehicle camera 11. In particular, the shape is determined by the lens characteristics of the super-wide-angle lens , and is a line segment that is virtually located in the second display range 22, as shown in FIG. The right end of the second display range 22 is divided into two at a: b as a predetermined ratio similar to the first display reference line segment 23, and the left end is the lower left of the second display range 22. The line segment extends to the end (in other words, the lower left corner). The second display reference line segment 24 has a left-right symmetric (line symmetric) shape with respect to the first display reference line segment 23.

  Further, the storage unit 16 functions as a first angle range storage device, and the storage unit 16 functioning in this way has a preset imaging range of the in-vehicle camera 11 as shown in FIG. Information on the first angle range α for acquiring the first original image in the effective pixel region 26 (for example, the effective pixel region in the image sensor of the camera 11) is stored.

Here, the first angle range α is an angle range centered on the point P corresponding to the optical axis of the in-vehicle camera 11 in the imaging range 26. As shown in FIG. 5, the first angle range α is clockwise in FIG. 5, with the perpendicular L extending vertically from the point P toward the lower end side of the imaging range 26 as the angle reference 0 [°]. The angle is in the range from the angle θ ₁ [°] to the angle θ ₂ [°].

  Furthermore, the storage unit 16 functions as a second angle range storage device, and the storage unit 16 functioning in this way has a preset imaging range of the in-vehicle camera 11 as shown in FIG. The information of the second angle range β for acquiring the second original image at 26 is stored.

Here, the second angle range β is an angle range centered on the point P corresponding to the optical axis of the in-vehicle camera 11 in the imaging range 26. Further, as shown in FIG. 5, the second angle range β is a counterclockwise clock in FIG. 5 in which a perpendicular line L that is perpendicularly lowered from the point P toward the lower end side of the imaging range 26 is an angle reference of 0 °. The rotation angle is in a range from an angle θ ₃ [°] to an angle θ ₄ [°].

  As the first angle range α and the second angle range β, various values can be set according to the concept.

  Furthermore, as shown in FIG. 6, the posture detection unit 17 has a feature line extraction unit 29 as a feature line extraction device, and the captured video 3 of the in-vehicle camera 11 is stored in the feature line extraction unit 29. It is designed to be entered.

Then, the feature line extraction unit 29 extracts a plurality of feature lines that are regarded as the break of the object in the captured video 3 from the input captured video 3 of the in-vehicle camera 11, and outputs the extraction result to the subsequent stage. It has become. This feature line is a reference line for detecting the attitude of the vehicle, and is preferably sharp without any breaks. Therefore, the feature line extraction unit 29 may extract the feature lines through the processes shown in FIGS. 7A to 7D, for example. More specifically, an optimum elliptical arc is selected from among a large number of elliptical arcs as a characteristic line serving as a reference for detecting the attitude of the vehicle. As this optimal elliptical arc, for example, a plurality of elliptical arcs having a longer overall length may be selected, and an elliptical arc closer to the vehicle may be selected.

  That is, in FIG. 7A, after the captured video 3 of the in-vehicle camera 11 is acquired as it is, as shown in FIG. 7B, the adjacent pixels in the captured video 3 of FIG. The part where the color difference between the two is equal to or greater than the threshold is extracted as a feature line. In FIG. 7B, the characteristic line is indicated by a white line. Next, in FIG. 7C, the feature lines extracted in the process of FIG. 7B are once thickened to interpolate the breaks in the feature lines in FIG. 7B. Finally, in FIG. 7D, the feature line thickened in FIG. 7C is thinned and returned to the original thickness shown in FIG. (5 pixels) A sharp feature line with no following breaks is extracted. If the feature lines are extracted in this way, the posture detection unit 17 can detect the posture of the vehicle simply, appropriately and reliably by image processing.

  Further, as shown in FIG. 6, the posture detection unit 17 includes a first search unit 31 as a first search device and a second search unit as a second search device that are connected to the subsequent stage of the feature line extraction unit 29. 32. The extraction result output from the feature line extraction unit 29 is input to the first search unit 31 and the second search unit 32.

Then, based on the input extraction result of the feature line extraction unit 29, the first search unit 31 in the image 33 including the feature lines having the same range as the captured image 3 and the imaging range 26, as shown in FIG. A first elliptic arc within a first angular range α centered on a point p corresponding to the optical axis of the in-vehicle camera 11 taken on the image 33 made up of the characteristic line from the left end side toward the center side. the first ellipse arcuate segment showing a previously have been set shape as indicating the posture of the vehicle contained in the video that may be captured I likewise vehicle camera 11 and the first display reference line segment 23 of the Jo The first line segment in the feature line that can be approximated is sequentially searched. The first ellipse arc, although the center point is fixed at the point p, the group of an ellipse such that its size and shape with that search of the first line segment proceeds may vary It is said that. Further, the first search unit 31 may use the search start position as the left end part of the image 33 made up of feature lines and the search end position as the center part of the image 33.

  The search result of the first line segment by the first search unit 31 is output to the image generation unit 18 as a part of the detection result of the posture of the vehicle by the posture detection unit 17.

On the other hand, as shown in FIG. 9, the second search unit 32 performs a feature based on the input extraction result of the feature line extraction unit 29 from the right end side to the center side in the image 33 including the feature lines. Similar to the second elliptical arc-shaped second display reference line segment 24 in the second angle range β centered on the point p corresponding to the optical axis of the in-vehicle camera 11 taken on the line image 33. the second in the second ellipse arc characteristic line that can be approximated to a line segment showing the shape that is preset as indicating the orientation of the vehicle contained in the video that may be captured by the onboard camera 11 The line segments are searched sequentially. The second ellipse arc, although the center point is fixed at the point p, the group of an ellipse such that its size and shape with that search of the second line segment proceeds may vary It is said that. In addition, the second search unit 32 may use the search start position as the right end of the image 33 made up of feature lines and the search end position as the center of the image 33.

  The search result of the second line segment by the second search unit 32 is output to the image generation unit 18 as a part of the detection result of the posture of the vehicle by the posture detection unit 17.

  Therefore, in the present embodiment, the posture of the host vehicle is detected as a search result of the first search unit 31 and the second search unit 32.

  Furthermore, as shown in FIG. 10, the image generation unit 18 includes a first acquisition range extraction unit 34 as a first acquisition range extraction device, and a second acquisition range extraction unit 35 as a second acquisition range extraction device. The first acquisition range extraction unit 34 receives the search result of the first line segment output from the first search unit 31, and the second acquisition range extraction unit 35 stores the second search unit 32. The search results of the second line segment output from are respectively input.

Then, the first obtaining range extraction unit 34, on the first line segment showing search results of a first search unit 31 that is input is then approximated to the first ellipse arcuate segments, first after approximation 1 Based on the line segment and the first angle range α stored in the storage unit 16, a rectangular first acquisition range for acquiring the first original image is extracted from the image 33 including the feature lines. It is supposed to be.

More specifically, as shown in FIG. 11, the first acquisition range 37 has at its lower right end, as a first elliptical arc 38 included in the first elliptical arcs, corresponding to a first angle range α (In other words, it is separated from other parts of the first ellipse by α) and the right end part (right end part) which is the lower end part of the first elliptic arc 38 including the first line segment 39 after approximation And the lower end portion coincide with each other). Therefore, the right end and the lower end of the first acquisition range 37 are defined by the right end as the lower end of the first elliptic arc 38. Further, as shown in FIG. 11, the first acquisition range 37 is a range in which the left end side is divided by the above-described a: b with the first line segment 39 after approximation as a predetermined ratio. ing.

  Further, the first acquisition range 37 is a range extracted stepwise using the first angle range α and the approximated first line segment 39.

That is, in the present embodiment, when the first acquisition range extraction unit 34 extracts the first acquisition range 37, first, the first acquisition range 37 is used when searching for the first angle range α and the first line segment. on which indexes the first elliptic arc 38 using 1 of the elliptical arc, referred first elliptical arc 38 and the lateral width including the entire first elliptical arc 38 is the same range (hereinafter, a first elliptical same width range ) 40 are extracted. At this time, the extracted first elliptic arc same width range 40 is a range in which the vertical width is not determined.

  And the 1st acquisition range extraction part 34 is, when the 1st line segment 39 after approximation has reached | attained on the left end side in the extracted 1st elliptical arc same width range 40, ie, the 1st after approximation When the line segment 39 forms the left end portion of the first elliptic arc 38, the left end side of the first acquisition range 37 is divided by the first line segment 39 approximated to the first elliptic arc same width range 40. The first acquisition range 37 is extracted only by limiting the vertical width in consideration of the position.

  On the other hand, when the left end side in the first elliptic arc same-width range 40 is located on the left side of the left end portion of the approximated first line segment 39, the first acquisition range extraction unit 34 determines the first elliptic arc. With respect to the same width range 40, the width is limited (reduced) by removing the left-side range from the left end portion of the first line segment 39 after approximation, and the new width obtained by the width limitation is further obtained. The first acquisition range 37 is extracted by limiting the vertical width in consideration of the division position of the left end side of the first acquisition range 37 by the first line segment 39 after approximation with respect to the approximate range 40 ′. It is like that. In addition, the 1st acquisition range 37 in FIG. 11 is the limitation of the horizontal width with respect to the 1st elliptical arc same width range 40, and the limitation of the vertical width with respect to the new range 40 'after the limitation obtained by the limitation of the said horizontal width. It is the range extracted by doing.

  By extracting the first acquisition range 37 in this way, the range located to the left of the left end portion of the approximated first line segment 39 is set to the state of the region on the left side with respect to the front end portion of the host vehicle. It is possible to exclude from the first acquisition range 37 as a range unnecessary for confirmation.

  And the 1st acquisition range extraction part 34 outputs the extraction result of the 1st acquisition range 37 to a back | latter stage.

On the other hand, the second acquisition range extraction unit 35, on the second line segment showing search results of a second search unit 32 that is input is then approximated to a second ellipse arcuate segments, first after approximation 2 Based on the line segment and the second angle range β stored in the storage unit 16, a rectangular second acquisition range for acquiring the second original image is extracted from the image 33 including the feature lines. It is supposed to be.

More specifically, as shown in FIG. 11, the second acquisition range 42 to its lower left portion, as a second elliptical arcs 43 included in the second ellipse arc, corresponding to the second angular range β (In other words, it is separated from other parts in the second ellipse by β) and the left end part (left end part) which is the lower end part of the second elliptic arc 43 including the approximated second line segment 45 And the lower end portion coincide with each other). Therefore, the left end and the lower end of the second acquisition range 42 are defined by the left end as the lower end of the second elliptic arc 43. Further, as shown in FIG. 11, the second acquisition range 42 is a range in which the right end side is divided by the above-described a: b with the second line segment 45 after approximation as a predetermined ratio. ing.

  Furthermore, the second acquisition range 42 is a range extracted stepwise using the second angle range β and the approximated second line segment 45.

In other words, in the present embodiment, when the second acquisition range extraction unit 35 extracts the second acquisition range 42, the second acquisition range 42 is first used for searching the second angle range β and the second line segment. on which indexes the second elliptical 43 by using a second elliptical arcs, referred second elliptic arc 43 and the lateral width including the entire second elliptic arc 43 is the same range (hereinafter, a second elliptical same width range ) 46 is extracted. At this time, the extracted second elliptic arc same width range 46 is a range in which the vertical width is not determined.

  Then, the second acquisition range extraction unit 35, when the approximated second line segment 45 has reached the right end side in the extracted second elliptic arc same width range 46, that is, the approximated second segment When the line segment 45 forms the right end portion of the second elliptic arc 43, the right end side of the second acquisition range 42 is divided by the second line segment 45 approximated with respect to the second elliptic arc same width range 46. The second acquisition range 42 is extracted only by limiting the vertical width in consideration of the position. In addition, the 2nd acquisition range 42 in FIG. 11 is made into the range extracted by performing only limitation of the vertical width with respect to the 2nd elliptical arc same width range 46 like this.

  On the other hand, when the right end side in the second elliptic arc same width range 46 is located on the right side of the right end portion of the approximated second line segment 45, the second acquisition range extraction unit 35 selects the second elliptic arc. For the same width range 46, the lateral width is limited by removing the range on the right side of the right end portion of the approximated second line segment 45, and further, a new range obtained by the lateral width limitation is obtained. On the other hand, the second acquisition range 42 is extracted by limiting the vertical width in consideration of the division position of the right end side of the second acquisition range 42 by the second line segment 45 after approximation. .

  And the 2nd acquisition range extraction part 35 outputs the extraction result of the 2nd acquisition range 42 to a back | latter stage.

  Returning to FIG. 10, the image generation unit 18 is connected to a first original image size changing unit 47 as a first original image size changing device connected to the subsequent stage of the first acquisition range extracting unit 34. The extraction result of the first acquisition range 37 output from the first acquisition range extraction unit 34 is input to the first original image size changing unit 47.

Then, the first original image size changing unit 47, as shown in FIG. 12, based on the input extraction result of the first acquisition range extraction unit 34, the first acquisition range 37 in the captured video 3 of the in-vehicle camera 11. 3A (that belongs to the same XY coordinate range (pixel area) as the first acquisition range 37) is acquired as the first original image 3A.

  Further, the first original image size changing unit 47 matches the acquired first original image 3A in the vertical direction (Y direction) that fits (in other words, matches) the first display range 21 and The first image 1 is generated by appropriately changing the dimension in the horizontal direction (X direction).

  In addition, such a change in the vertical and horizontal dimensions with respect to the first original image 3A is such that the object in the first original image 3A is moved in the real world as if the vertical axis of rotation assumed in the real world is the center. This is processing corresponding to angle correction such as rotation.

  Although the first image 1 generated in this way is not illustrated, the position of the line segment corresponding to the first line segment 39 after the approximation matches the first display reference line segment 23. In this sense, the first display reference line segment 23 is a reference for displaying the first image 1. Here, the line segment corresponding to the first line segment 39 after approximation has the same shape and size as the first line segment 39 after approximation obtained on the image 33 including the characteristic lines described above. On the same XY coordinate position (position corresponding to the first acquisition range 37) as the first line segment 39 after approximation on the image 33 made up of feature lines on the captured video 3 of the in-vehicle camera 11 Further, the line segment defined in this manner is changed in size integrally with the first original image 3A as the first original image size changing unit 47 changes the size of the first original image 3A. It is a line segment that is virtually located on the first image 1 generated by the dimension change when it is assumed.

  Then, the first original image size changing unit 47 outputs the generated first image 1 to the subsequent image composition unit 20.

  Further, as shown in FIG. 10, the image generation unit 18 is connected to a second original image size changing unit 48 as a second original image size changing device connected to the subsequent stage of the second acquisition range extracting unit 35. The second original image size changing unit 48 receives the extraction result of the second acquisition range 42 output from the second acquisition range extraction unit 35.

Then, the second original image size changing unit 48, based on the input extraction result of the second acquisition range extraction unit 35, as shown in FIG. 12, the second acquisition range 42 in the captured video 3 of the in-vehicle camera 11. 2 (that belongs to the same XY coordinate range (pixel region) as the second acquisition range 42) is acquired as the second original image 3B.

  Further, the second original image size changing unit 48 matches the second display range 22 with respect to the acquired second original image 3B (in other words, the vertical direction (Y direction)). The second image 2 is generated by appropriately changing the dimensions in the horizontal direction (X direction).

  In addition, such a change in the vertical and horizontal dimensions with respect to the second original image 3B can be achieved by moving an object in the second original image 3B on the real world as if it were assumed to be a vertical axis of rotation assumed in the real world. This is processing corresponding to angle correction such as rotation.

  Although the second image 2 generated in this way is not shown, the line segment corresponding to the approximated second line segment 45 matches the second display reference line segment 24. In this sense, the second display reference line segment 24 serves as a reference for displaying the second image 2. Here, the line segment corresponding to the approximated second line segment 45 has the same shape and size as the approximated second line segment 45 obtained on the image 33 including the characteristic lines described above. On the same XY coordinate position (position corresponding to the second acquisition range 42) as the second line segment 45 after approximation on the image 33 made up of feature lines on the captured video 3 of the in-vehicle camera 11 In addition, the line segment defined in this way is changed in size integrally with the second original image 3B in accordance with the change in the size of the second original image 3B by the second original image size changing unit 48. It is a line segment that is virtually located on the second image 2 generated by the dimension change when it is assumed.

  Then, the second original image size changing unit 48 outputs the generated second image 2 to the subsequent image composition unit 20.

  As shown in FIG. 12, the first image 1 and the second image 2 output to the image composition unit 20 are synthesized with each other, and then displayed in the first display range 21 in the display area 4 of the display 14. The first image 1 and the second image 2 are displayed in the second display range 22, respectively.

  Therefore, according to the present embodiment, the first image 1 and the second image 2 suitable for checking the left and right sides with respect to the front end portion of the host vehicle can be easily and appropriately generated by image processing.

  The first search unit 31 searches for the first line segment in the search range on the image 33 corresponding to the first angle range α, and the second search unit 32 sets the second angle range β to the second angle range β. You may comprise so that a 2nd line segment may be searched in the search range on the image 33 which respond | corresponds.

  According to such a configuration, the search range of the first line segment by the first search unit 31 and the search range of the second line segment by the second search unit 32 can be limited to the minimum necessary ranges, respectively. Therefore, the first image 1 and the second image 2 can be generated quickly.

  Next, an embodiment of a vehicle driving support method according to the present invention will be described with reference to FIGS. 13 and 14.

  Note that the vehicle driving support method in the present embodiment is implemented using the vehicle driving support device 10 described above for convenience.

  That is, in this embodiment, first, in step 1 (ST1) of FIG. 13, the storage unit 16 stores the first display range 21, the second display range 22, the first display reference line segment 23, the second display range. The display reference line segment 24, the first angle range α, and the second angle range β are stored (set) in advance before the generation of the images 1 and 2 is started.

  Next, in step 2 (ST2), the control unit 15 determines whether or not the vehicle speed of the host vehicle is equal to or lower than a predetermined speed based on the vehicle speed pulse. If the predetermined speed is exceeded, the process is terminated.

Next, in step 3 (ST3), the control unit 15 controls the image display processing unit 12 to shift to display processing of the first image 1 and the second image 2. First, the feature line in the posture detection unit 17 is displayed. The extraction unit 29 extracts a plurality of feature lines from the captured video 3 of the in-vehicle camera 11 through, for example, the steps shown in FIGS.

  Next, in step 4 (ST4), by using the extraction result of the feature line in step 3 (ST3), the first search unit 31 in the posture detection unit 17 uses the feature line image 33, for example, in FIG. As shown in FIG. 9, the first line segment is searched and the second search unit 32 in the posture detection unit 17 searches the second line segment from the image 33 including the feature lines, for example, as shown in FIG. 9. I do.

  Next, in step 5 (ST5), the first acquisition range extraction unit 34 extracts the first acquisition range 37 based on the search result of the first line segment in step 4, and the second acquisition range extraction. The unit 35 extracts the second acquisition range 42 based on the search result of the second line segment in step 4.

That is, in step 5 (ST5), first, as shown in step 5-1 (ST5-1) of FIG. 14, the first line segment used in the search for the first line segment is used. thereby approximating the first elliptical arcuate segment, a second segment approximates a second ellipse arc line segments used during the search for the second segment.

Then, as shown in step 5-2 (ST5-2), the first angle range α and first first by using the elliptical arc of the elliptic arc 38 stored in the storage unit 16 (see FIG. 11) together determine the, determine a second elliptical 43 by using the second angle range stored in the storage unit 16 beta and the second ellipse arc (see FIG. 11).

  Next, as shown in step 5-3 (ST5-3), the first elliptical arc same-width range 40 is extracted and the second elliptical arc same-width range 46 is extracted (see FIG. 11).

  Next, as shown in Step 5-4 (ST5-4), the first line segment 39 after approximation on the left end side in the first elliptic arc same width range 40 extracted in Step 5-3 (ST5-3). Is reached and the approximated second line segment 45 arrives on the right end side in the second elliptic arc same width range 46 extracted in step 5-3 (ST5-3). It is determined whether or not.

  And as a result of determination of step 5-4 (ST5-4) with respect to the 1st elliptical arc same width range 40, the 1st line segment 39 after approximation has arrived on the left end side in the 1st elliptical arc same width range 40. In this case, the process proceeds to Step 5-5 (ST5-5), and if not reached, that is, the left end side in the first elliptic arc same width range 40 is located on the left side of the approximated first line segment 39. If yes, go to Step 5-6 (ST5-6).

  Further, as a result of the determination in step 5-4 (ST5-4) for the second elliptic arc same width range 46, the approximated second line segment 45 has reached the right end side in the second elliptic arc same width range 46. In this case, the process proceeds to Step 5-5 (ST5-5), and if not reached, that is, the right end side in the second elliptic arc same width range 46 is located on the right side of the approximated second line segment 45. If yes, go to Step 5-6 (ST5-6).

  Next, in step 5-5 (ST5-5), the vertical position in consideration of the division position of the left end side of the first acquisition range 37 by the first line segment 39 approximated with respect to the first elliptic arc same width range 40 is considered. The first acquisition range 37 is extracted by limiting only the width. Specifically, the lower end of the first elliptical arc same width range 40 is limited by the lower end of the first elliptical arc 38, and the first line segment 39 after approximating the left end side of the first elliptical arc same width range 40 has a predetermined value. The upper end of the first elliptic arc same width range 40 is limited so as to be divided at a ratio (a: b).

  Similarly, in Step 5-5 (ST5-5), the division position of the right end side of the second acquisition range 42 by the second line segment 45 after the approximation with respect to the second elliptic arc same width range 46 is considered. The second acquisition range 42 is extracted by limiting only the vertical width. Specifically, the lower end of the second elliptic arc same width range 46 is limited by the lower end portion of the second elliptic arc 43, and the second line segment 45 after approximating the right end side of the second elliptic arc same width range 46 has a predetermined value. The upper end of the second elliptic arc same width range 46 is limited so as to be divided at a ratio (a: b).

  On the other hand, in step 5-6 (ST5-6), the width is limited by removing the left-side range from the left end portion of the first line segment 39 after the approximation with respect to the first elliptic arc same width range 40. (Reduction) and step (reduction) of the width of the second elliptical arc same width range 46 by removing the range on the right side of the right end of the approximated second line segment 45 (reduction) Proceed to 5-7 (ST5-7).

Next, in step 5-7 (ST5-7), the vertical range similar to step 5-5 (ST5-5) is applied to the new range obtained by the width limitation in step 5-6 (ST5-6). to also when extracting a first acquisition range 37 performs limitation of the width, the second elliptical same width range 46, first performs a limitation of same vertical width as the step 5-5 (ST5-5) Two acquisition ranges 42 are extracted.

  Next, in step 6 (ST6) of FIG. 13, the first in-vehicle image size changing unit 47 in the image generating unit 18 of the in-vehicle camera 11 corresponding to the first acquisition range 37 extracted in step 5 (ST5). The video 3A in the captured video 3 is acquired as the first original image 3A.

  At this time, the second original image size changing unit 48 in the image generating unit 18 selects the video 3B in the captured video 3 of the in-vehicle camera 11 corresponding to the second acquisition range 42 extracted in step 5 (ST5). Obtained as the second original image 3B.

  Next, in step 7 (ST7), the first original image size changing unit 47 performs the first display stored in the storage unit 16 on the first original image 3A acquired in step 6 (ST6). The first image 1 is generated by changing the vertical and horizontal dimensions to match the range 21.

  At this time, the second original image size changing unit 48 conforms to the second display range 22 stored in the storage unit 16 with respect to the second original image 3B acquired in step 6 (ST6). The second image 2 is generated by changing the vertical and horizontal dimensions.

  Next, in step 8 (ST8), the image synthesizing unit 20 synthesizes the first image 1 and the second image 2 generated in step 7 (ST7), and displays the synthesized images 1 and 2 on the display. 14 to finish the processing. At this time, the first image 1 is displayed in the first display range 21 in the display area of the display 14, and the second image 2 is displayed in the second display range 22 in the display area 4 of the display 14.

  As described above, according to the present embodiment, the image display processing unit 12 determines the left and right side of the front end of the host vehicle based on the detection result of the posture detection unit 17 regardless of the position of the host vehicle. It is possible to display the first image 1 and the second image 2 that always include an area regarded as necessary for checking the situation and that the angle of the object on the image is bilaterally symmetric in the display state. It is possible to easily and appropriately confirm the situation of the left and right side regions with respect to the front end portion of the vehicle.

  In addition, this invention is not limited to embodiment mentioned above, A various change is possible as needed.

The block diagram which shows embodiment of the vehicle driving assistance device which concerns on this invention 1 is a block diagram illustrating a detailed configuration of an image display processing unit in FIG. 1 in an embodiment of a vehicle driving support apparatus according to the present invention. The figure which shows the display state of a 1st image and a 2nd image in embodiment of the vehicle driving assistance device which concerns on this invention. Explanatory drawing for demonstrating the 1st and 2nd display range and the 1st and 2nd display reference line in embodiment of the vehicle driving assistance device which concerns on this invention. Explanatory drawing for demonstrating the 1st and 2nd angle range in embodiment of the vehicle driving assistance device which concerns on this invention. FIG. 2 is a block diagram showing a detailed configuration of the posture detection unit in FIG. 2 in the embodiment of the vehicle driving support device according to the present invention. It is process drawing which shows the extraction process of the characteristic line in embodiment of the vehicle driving assistance device which concerns on this invention, Comprising: (a) is the acquisition process of the picked-up image of a vehicle-mounted camera, (b) is the characteristic line of an initial stage. (C) is an interpolation step of feature line breaks due to thickening of feature lines, and (d) is a final feature line in a state where the breaks due to thinning of feature lines are interpolated. Showing each extraction process Explanatory drawing for demonstrating the search process of a 1st line segment in embodiment of the vehicle driving assistance device which concerns on this invention. Explanatory drawing for demonstrating the search process of a 2nd line segment in embodiment of the vehicle driving assistance device which concerns on this invention. FIG. 2 is an explanatory diagram for explaining a detailed configuration of the image generation unit in FIG. 2 in the embodiment of the vehicle driving support device according to the present invention. Explanatory drawing for demonstrating the extraction process of the 1st and 2nd acquisition range in embodiment of the vehicle driving assistance device which concerns on this invention Explanatory drawing for demonstrating the expansion-contraction process of the 1st and 2nd original image in embodiment of the vehicle driving assistance device which concerns on this invention 1st flowchart which shows embodiment of the vehicle driving assistance method which concerns on this invention 2nd flowchart which shows embodiment of the vehicle driving assistance method which concerns on this invention The figure which shows the display state of a 1st image and a 2nd image Schematic diagram showing the attitude of the vehicle in the real world corresponding to FIG. The figure which shows the picked-up image of the vehicle-mounted camera used for the production | generation of the image of FIG. Schematic diagram showing the attitude of the vehicle in the real world when a conventional problem occurs The figure which shows the picked-up image of the vehicle-mounted camera corresponding to FIG. The figure which shows the display state of the 1st image and 2nd image corresponding to the case of FIG. 18 as a conventional problem. Schematic diagram showing the posture of the vehicle in the real world when the same problem as in FIG. 20 occurs

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 1st image 2 2nd image 3 Captured image 10 Vehicle driving assistance apparatus 11 Car-mounted camera 12 Image display processing apparatus 14 Display 17 Posture detection part 18 Image generation part 20 Image composition part

Claims (14)

An in-vehicle camera mounted on a vehicle and capable of photographing a wide viewing angle imaging region from the front of the vehicle to the left and right sides of the front end of the vehicle;
A first image for confirming the situation of the left side region with respect to the front end portion of the vehicle and a second state for confirming the situation of the right side region with respect to the front end portion of the vehicle based on the video image taken by the in-vehicle camera. An image display processing device that generates two images and simultaneously displays the generated first image and second image in a state of being arranged side by side on the display unit,
A vehicle driving support device capable of supporting driving of the vehicle by displaying the first image and the second image,
The image display processing device includes:
Approximate the shape shown in the captured image in the imaging area actually captured by the in-vehicle camera to the shape preset to indicate the posture of the vehicle included in the image that can be captured by the in-vehicle camera by a posture detection device for detecting the posture of the vehicle relative to the object included in the captured image,
Based on the detection result of the posture detection device, the first image and the second image always include a region that needs to be confirmed regardless of the posture of the vehicle, and the object on the image An image generating device for generating the first image and the second image, respectively, such that the angles are symmetrical with each other in the display state;
A vehicle driving support device comprising: an image composition device configured to compose the first image and the second image generated by the image generation device and display the synthesized image on the display unit. The image generation device includes:
A first original image that is a source of the first image that belongs to a range corresponding to a detection result of the posture detection device, and a range corresponding to the detection result of the posture detection device, from a captured image of the in-vehicle camera And a second original image that is a source of the second image as belonging to the first image, and the first image in the display unit that is set in advance with respect to the acquired first original image The vertical and horizontal dimensions are changed so as to fit the first display range in which is displayed, and the second image in the display unit that is set in advance with respect to the acquired second original image is By changing the vertical and horizontal dimensions so as to fit the second display range to be displayed, the first image and the second image in which the angle of the object on the image is symmetrical with each other in the display state Generate each image Vehicle driving support device according to claim 1, characterized in that it is urchin formed. The vehicle driving support device body is
A first display range storage device in which information on a first display range of the rectangle which is the first display range set in advance and is located on the left side in the display area of the display unit is stored;
Information on the second display range that is the second display range that is set in advance and that is the same as the first display range that is located on the right side of the display area of the display unit and that has the same vertical / horizontal ratio is stored. A second display range storage device;
As an indication of the posture of the vehicle included in a video that can be taken by the in-vehicle camera that serves as a reference when displaying the first image virtually located in the first display range set in advance A first display reference line segment having an elliptical arc shape indicating a set shape , the left end portion of the first display range is divided at a predetermined ratio, and the right end portion is the first display reference line. A first display reference line segment storage device that stores information about the first display reference line segment that reaches the lower right end of the display range;
As an indication of the posture of the vehicle included in a video that can be taken by the in-vehicle camera that serves as a reference when displaying the second image virtually positioned in the second display range set in advance A second display reference line segment having an elliptical arc shape indicating the set shape , the right end portion of which splits the right end side of the second display range at the predetermined ratio, and the left end portion of the second display reference line segment. A second display reference line segment storage device that stores information on the second display reference line segment that reaches the lower left end of the display range;
A first angle range for acquiring the first original image in the imaging range of the in-vehicle camera set in advance, the first angle range centering on a point corresponding to the optical axis of the in-vehicle camera in the imaging range A first angle range storage device storing information of one angle range;
A second angle range for acquiring the second original image in the imaging range of the in-vehicle camera set in advance, the second angle range being centered on a point corresponding to the optical axis of the in-vehicle camera in the imaging range A second angle range storage device in which information of two angle ranges is stored,
The posture detecting device is
A feature line extraction device that extracts a plurality of feature lines that are regarded as delimiters of objects in the captured video from the shape shown in the captured video actually captured by the in- vehicle camera;
Corresponding to the optical axis of the in-vehicle camera taken on the image of the feature lines of the feature lines from the left side to the center side in the image of the feature lines extracted by the feature line extraction device As the first elliptical arc-shaped first display reference line segment within the first angle range centered on a point, the vehicle posture included in the image that can be photographed by the in-vehicle camera is shown in advance. a first search unit that sequentially searches the first line segment which can be approximated to a first elliptical arcuate segment indicating the set to which the shape,
Corresponding to the optical axis of the in-vehicle camera taken on the image of the feature lines from the right side to the center side in the image of the feature lines extracted by the feature line extraction device As the second elliptical arc-shaped second display reference line segment within the second angle range centered on a point, the vehicle posture included in the image that can be photographed by the in-vehicle camera is shown in advance. comprising sequentially a search the second search unit of the second line segments that can be approximated to the second ellipse arcuate segment indicating the set to which the shape, the orientation of the vehicle, said first It is configured to detect as a search result of the search device and the second search device,
The image generation device includes:
On the first line segment retrieved by the first retrieval apparatus is approximated to the first elliptical arcuate segments, first the line segment first angular range storage device after approximation As a first acquisition range of a rectangle for acquiring the first original image from the image composed of the feature lines based on the first angle range stored in the A first elliptic arc included in the first ellipse, corresponding to the first angle range and including the first line segment after the approximation, the right end portion and the lower end portion of the first elliptic arc are positioned. , A first acquisition range extraction device for extracting a first acquisition range such that the first line segment after the approximation is divided at the predetermined ratio on the left end side;
Wherein after having the searched by the second search unit of the second line segments to approximate the second ellipse arcuate segment, a second of said line segment second angular range storage device after approximation As a second acquisition range of a rectangle for acquiring the second original image from the image composed of the feature lines based on the second angle range stored in the A second elliptic arc included in the second ellipse, corresponding to the second angle range and including the second line segment after the approximation, the left end portion and the lower end portion of the second elliptic arc are positioned. , A second acquisition range extraction device for extracting a second acquisition range such that the approximated second line segment is divided at the predetermined ratio on the right end side thereof;
The video corresponding to the first acquisition range extracted by the first acquisition range extraction device in the video captured by the in-vehicle camera is acquired as the first original image, and the acquired first original image is acquired. A first original image size changing device for generating the first image by changing the vertical and horizontal dimensions so as to fit the first display range;
A video corresponding to the second acquisition range extracted by the second acquisition range extraction device in the video captured by the in-vehicle camera is acquired as the second original image, and the acquired second original image is acquired. The apparatus further comprises a second original image size changing device that generates the second image by changing the vertical and horizontal dimensions to match the second display range. The vehicle driving support device according to claim 1. The said characteristic line extraction apparatus is formed so that the location from which the color difference with the adjacent pixel in the picked-up image of the said vehicle-mounted camera is more than a threshold value may be extracted as the said characteristic line. Vehicle driving support device. The feature line extraction device, after once thickening the extracted feature line and interpolating the discontinuity of the feature line, by thinning the thickened feature line and returning to the original thickness, The vehicle driving support device according to claim 4, wherein the vehicle is configured to extract the feature line in a state where there is no break having a predetermined number of pixels or less. The first search device includes:
Formed to search for the first line segment in a search range corresponding to the first angle range;
The second search device includes:
The vehicle according to any one of claims 3 to 5, wherein the vehicle is configured to search for the second line segment in a search range corresponding to the second angle range. Driving assistance device. The first acquisition range extraction device includes:
When extracting the first acquisition range, a range having the same horizontal width as the first elliptic arc including the entire first elliptic arc is extracted, and the extracted horizontal width is on the left end side in the same range. When the first line segment after the approximation has reached, the division position of the left end side of the first acquisition range by the first line segment after the approximation with respect to the range having the same horizontal width is considered. The first acquisition range is extracted only by limiting the vertical width, and the left end side in the extracted range having the same horizontal width is on the left side of the left end portion of the first line segment after the approximation. If it is located, the width is limited by removing the left-side range from the left end of the approximated first line segment with respect to the range having the same width, and further, the width limitation To the first line segment after the approximation with respect to the new range obtained by Is formed so as to extract the first acquiring range by performing a limitation of the vertical width in consideration of the division position of the left side of the first acquisition range that,
When the second acquisition range extraction device extracts the second acquisition range, the second acquisition range extraction device extracts a range having the same horizontal width as the second elliptic arc including the entire second elliptic arc, and the extracted horizontal width Is the second line segment after the approximation on the right end side in the same range, the second line segment by the second line segment after the approximation with respect to the range where the horizontal width is the same. The second acquisition range is extracted only by limiting the vertical width in consideration of the division position of the right end side of the acquisition range, and the right end side in the extracted range having the same horizontal width is the second after the approximation. Is located on the right side of the right end of the line segment, the width of the width is removed by removing the range on the right side of the approximated second line segment from the right end of the approximated second line segment. To the new range obtained by limiting the width. Thus, the second acquisition range is extracted by limiting the vertical width in consideration of the division position of the right end side of the second acquisition range by the approximated second line segment. The vehicle driving support device according to any one of claims 3 to 6, wherein An imaging region having a wide viewing angle that extends from the front of the vehicle to the left and right sides of the front end of the vehicle is captured using an in-vehicle camera mounted on the vehicle, and the front end of the vehicle is based on a captured image of the in-vehicle camera. A first image for confirming a situation of a region on the left side with respect to the part and a second image for confirming a situation of a region on the right side with respect to the front end portion of the vehicle, and the generated first image And a vehicle driving support method for supporting driving of the vehicle by simultaneously displaying the second image on the display unit in a state of being arranged in parallel on the left and right,
Approximate the shape shown in the captured image in the imaging area actually captured by the in-vehicle camera to the shape preset to indicate the posture of the vehicle included in the image that can be captured by the in-vehicle camera By detecting the posture of the vehicle with respect to the object included in the captured video,
Based on the detected attitude of the vehicle, the first image and the second image always include an area that is considered necessary for the situation confirmation regardless of the attitude of the vehicle, and the object on the image Generating the first image and the second image so that the angles are symmetrical with each other in the display state;
The vehicle driving support method, wherein the generated first image and the second image are combined and displayed on the display unit. A first original image that is a source of the first image that belongs to a range corresponding to the detected posture of the vehicle from a captured image of the in-vehicle camera, and a range corresponding to the detected posture of the vehicle And a second original image that is a source of the second image as belonging to the first image, and the first image in the display unit that is set in advance with respect to the acquired first original image The vertical and horizontal dimensions are changed so as to fit the first display range in which is displayed, and the second image in the display unit that is set in advance with respect to the acquired second original image is By changing the vertical and horizontal dimensions so as to fit the second display range to be displayed, the first image and the second image in which the angle of the object on the image is symmetrical with each other in the display state Generating each image The vehicle driving support method according to claim 8. A rectangular first display range located on the left side of the display area of the display unit, the first display range;
A second display range that is the second display range and has the same rectangular size ratio as the first display range located on the right side of the display area of the display unit;
It is set in advance to indicate the posture of the vehicle included in the video that can be taken by the vehicle-mounted camera serving as a reference when displaying the first image virtually located in the first display range . A first display reference line segment having an elliptical arc shape indicating a shape , the left end portion of the first display range dividing the left end side of the first display range at a predetermined ratio, and the right end portion being a right side of the first display range; A first display reference line segment that reaches the lower end,
It is set in advance as indicating the posture of the vehicle included in the video that can be taken by the in-vehicle camera that is a reference when displaying the second image virtually located in the second display range . A second display reference line segment having an elliptical arc shape indicating a shape , the right end portion of which is divided at the predetermined ratio by the right end side of the second display range, and the left end portion of the second display range A second display reference line segment that reaches the lower left corner,
A first angle range for acquiring the first original image in the imaging range of the in-vehicle camera, the first angle range centering on a point corresponding to the optical axis of the in-vehicle camera in the imaging range When,
A second angle range for acquiring the second original image in the imaging range of the in-vehicle camera, the second angle range centering on a point corresponding to the optical axis of the in-vehicle camera in the imaging range Set and in advance,
An image composed of the extracted feature lines after extracting a plurality of feature lines that are regarded as delimiters of objects in the captured video from the shape shown in the captured video actually captured by the in- vehicle camera . In the first angle range centered on a point corresponding to the optical axis of the in-vehicle camera taken on the image of the feature lines among the extracted feature lines from the left side to the center side , first ellipse arc showing the shape that is preset as indicating the posture of the vehicle included in the image can be captured by the first elliptic arc shaped first display reference line segment as well as the in-vehicle camera The first line segment that can be approximated to the line segment is sequentially searched, and the feature line of the extracted feature lines is moved from the right side to the center side in the image composed of the extracted feature lines. In the second range of angles about the point corresponding to the optical axis of the taken on an image consisting of symptoms line the vehicle camera, the as with the second elliptic arc shaped second display reference line sequentially searching for a second line segment which can be approximated to the second ellipse arcuate segment showing the shape that is preset as indicating the posture of the vehicle included in the video that may be captured by the onboard camera By detecting the posture of the vehicle as the search result of the first line segment and the second line segment,
Retrieved the first line segment on which is approximated to the first elliptical arcuate segments, based on the first angle range set in advance and the first segment after the approximation, As the first acquisition range of the rectangle for acquiring the first original image from the image composed of the feature lines, the lower right end of the first elliptic arc included in the first ellipse, A right end portion and a lower end portion of the first elliptical arc corresponding to the first angle range and including the first line segment after the approximation are positioned, and the left end side is defined as the first line segment after the approximation. Extract a first acquisition range that divides at the predetermined ratio,
Retrieved the second line segment on which is approximated to the second ellipse arcuate segments, based on the second angle range set in advance and a second line segment after approximation, As a second acquisition range of a rectangle for acquiring the second original image from the image composed of the feature lines, a second elliptic arc included in the second ellipse is provided at the lower left end of the second ellipse. A left end portion and a lower end portion of the second elliptic arc corresponding to the second angle range and including the approximated second line segment are located, and the right end side of the second elliptical arc is defined as the approximated second line segment. Extracts a second acquisition range that divides at the predetermined rate,
A video corresponding to the first acquisition range extracted in the video captured by the in-vehicle camera is acquired as the first original image, and the first display range is set in the first display range with respect to the acquired first original image. The first image is generated by changing the vertical and horizontal dimensions so as to fit, and the video corresponding to the second acquisition range extracted in the video captured by the in-vehicle camera is used as the second original image. The second image is generated by changing the vertical and horizontal dimensions so as to fit the second display range with respect to the acquired second original image. 9. The vehicle driving support method according to 9. The vehicle driving support method according to claim 10, wherein a portion where a color difference with an adjacent pixel in a captured image of the in-vehicle camera is equal to or greater than a threshold is extracted as the feature line. After the extracted feature line is thickened once and the cut of the feature line is interpolated, the thickened feature line is thinned and returned to the original thickness, thereby making a break of a predetermined number of pixels or less. 12. The vehicle driving support method according to claim 11, wherein the feature line in a state not having a mark is extracted. Searching the first line segment in a search range corresponding to the first angle range;
The vehicle driving support method according to any one of claims 10 to 12, wherein the second line segment is searched in a search range corresponding to the second angle range. When extracting the first acquisition range, a range having the same horizontal width as the first elliptic arc including the entire first elliptic arc is extracted, and the extracted horizontal width is on the left end side in the same range. When the first line segment after the approximation has reached, the division position of the left end side of the first acquisition range by the first line segment after the approximation with respect to the range having the same horizontal width is considered. The first acquisition range is extracted only by limiting the vertical width, and the left end side in the extracted range having the same horizontal width is on the left side of the left end portion of the first line segment after the approximation. If it is located, the width is limited by removing the left-side range from the left end of the approximated first line segment with respect to the range having the same width, and further, the width limitation To the first line segment after the approximation with respect to the new range obtained by That extracts the first acquiring range by performing a limitation of the vertical width in consideration of the division position of the left side of the first acquisition range,
When extracting the second acquisition range, a range having the same horizontal width as the second elliptic arc including the entire second elliptic arc is extracted, and the extracted horizontal width is on the right end side in the same range. When the approximated second line segment has arrived, the division position of the right end side of the second acquisition range by the approximated second line segment is considered with respect to the range having the same horizontal width. The second acquisition range is extracted only by limiting the vertical width, and the right end side in the extracted range having the same horizontal width is on the right side of the right end portion of the second line segment after the approximation. If it is located, the width is limited by removing a range on the right side of the right end portion of the second line segment after the approximation with respect to the range having the same width, and further, the width is limited. To the second line segment after the approximation to the new range obtained by The second acquisition range is extracted by limiting the vertical width in consideration of the division position of the right end side of the second acquisition range. The vehicle driving support method according to claim 1.

Images