JP5733196B2 - Lane guidance display system, method and program - Google Patents

Description

  The present invention relates to a lane guidance display system, method, and program for displaying a guide line indicating a lane image in a front image.

  2. Description of the Related Art Conventionally, a technique is known in which a guide line indicating an image of a traveling lane in which a vehicle is traveling is superimposed and displayed on a front image obtained by photographing the front of the vehicle with a camera provided in the vehicle (Patent Document). 1). In Patent Document 1, the shape vector of the image of the driving lane is recognized based on the result of recognizing the left and right white line images of the driving lane in the front image, and the shape of the guide line and the overlapping position are determined based on the shape vector. doing. Further, in Patent Document 1, if one of the left and right white line images of the traveling lane cannot be recognized, the reliability of the recognized shape vector of the traveling lane image is low. Further, when the reliability of the shape vector of the recognized traveling lane image is low, the shape vector of the traveling lane image is calculated based on the recognized white line image and the width of the lane registered in the map information. to correct.

JP 2008-309529 A

However, if one of the left and right white line images of the driving lane is not recognized in a short period of time, the guide line based on the corrected driving lane image shape vector and the uncorrected driving lane image shape There is a problem that the guide line based on the vector is switched in a short period of time. In other words, the guide line based on the corrected travel lane image shape vector and the guide line based on the uncorrected travel lane image shape vector are switched in a short period of time, so that the left and right white line images of the travel lane On the other hand, the guide line fluctuated, causing the driver to feel uncomfortable.
The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for preventing a sense of incongruity due to wobbling of a guide line.

  In order to achieve the above object, in the present invention, the front image acquisition means acquires a front image obtained by photographing the front of the vehicle. The center line specifying means specifies the center line indicating the center position of the lane in the width direction in the front image based on the image of the lane markings recognized in the front image. The display control unit generates a guide line including a center line portion with the center line as a reference, and displays a front image on which the guide line is superimposed on the display unit. Furthermore, when the reliability of the current center line, which is the center line specified in the current front image, is greater than or equal to the predetermined reliability, the display control means displays a guide line including the center line portion with the current center line as a reference. Generate. In addition, when the reliability of the center line is lower than the predetermined reliability, the display control means uses a center line portion based on a past center line that is a past center line whose reliability has finally become equal to or higher than the predetermined reliability. And a guide line in which the width of the center line portion is expanded in the direction of deviation of the current center line from the past center line is generated.

  When the reliability of the current center line (N) specified in the Nth (N is a natural number) forward image is equal to or higher than a predetermined reliability, the display control means determines the center line based on the current center line (N). A guide line including a part is generated. Thereby, when the current center line (N) specified in the latest front image can be trusted, a guide line including a center line portion based on the current center line (N) can be generated. A guide line having high reliability and high followability to the front image can be generated. Note that the characters in parentheses represent the acquisition order of the front image in which the center line is specified.

  Here, the following description will be given on the assumption that the reliability of the past center line (NM) specified in the (NM) -th (M is a natural number less than N) forward image is equal to or higher than a predetermined reliability. When the reliability of the current center line (N) is lower than the predetermined reliability in the N-th front image, the display control means generates a guide line including a center line portion based on the past center line (NM). To do. Accordingly, it is possible to prevent the driver from feeling uncomfortable by following the current center line (N) whose reliability is lower than the predetermined reliability and causing the center line portion to fluctuate. Further, when the reliability of the current center line (N) is lower than the predetermined reliability in the Nth front image, the display control means determines the width of the center line portion with reference to the past center line (NM), A guide line expanded in the direction of deviation of the current center line (N) with respect to the past center line (NM) is generated. Thus, even when the current center line (N) indicates a position closer to the center position of the lane than the past center line (NM) in the current front image, the position of the center line portion indicating the center position of the lane. It is possible to prevent the driver from giving an impression that the display is greatly deviated from the driver.

  Further, if the reliability of the current center line (N + 1) specified in the (N + 1) th forward image is equal to or higher than the predetermined reliability, the past center line (NM) is used as a reference in the Nth front image. From the state in which the center line portion to be generated is generated, the state shifts to a state in which the center line portion based on the current center line (N + 1) is generated in the (N + 1) th forward image. That is, between the Nth front image and the (N + 1) th front image, the reference of the center line portion changes from the past center line (NM) to the current center line (N + 1), and the guide line May fluctuate greatly and make the driver feel uncomfortable.

  Further, although the N-th center line (N) has a lower reliability than the predetermined reliability, the center line (N) specified in the N-th front image whose acquired timing is close to the (N + 1) -th front image. Therefore, it is considered that this is approximate to the (N + 1) th current center line (N + 1). That is, before the transition of the center line portion reference from the past center line (NM) to the current center line (N + 1) between the Nth front image and the (N + 1) th front image, The width of the line portion can be expanded to the Nth center line (N) side that approximates the current center line (N + 1) of the transition destination. Thereby, the fluctuation of the guide line when the reference of the center line portion transitions from the past center line (NM) to the current center line (N + 1) can be reduced.

  The forward image acquisition means only needs to acquire a forward image obtained by photographing the front of the vehicle. For example, the forward image may be acquired from a camera including the front of the vehicle in the field of view. The front of the vehicle means the front of the vehicle in the traveling direction. In particular, the expression “forward” in a certain lane means the forward direction of the vehicle in the lane (lane direction). Further, the position of the vehicle in the traveling direction in the lane appears in the vertical position in the front image displayed on the display unit. On the other hand, the position in the width direction in the lane appears in the position in the horizontal direction in the front image displayed on the display unit.

  The center line specifying means may specify the center line indicating the center position of the lane in the width direction in the front image, and may specify the center of the lane image recognized in the front image as the center line. For example, the center line identification means may identify the image of the recommended lane by recognizing the image of the lane marking by a known line recognition method such as Hough transform. The center line specifying means may specify the center line for the lane indicated by the guide line, may specify the center line of the traveling lane in which the vehicle is currently traveling, or the recommended lane in which the vehicle should travel. The center line may be specified.

  The display control means may generate a guide line including a center line portion with respect to the center line, and may generate a guide line including a center line portion having a certain geometric relationship with the center line. Further, the display control means may generate a guide line in which the width of the center line portion with respect to the past center line is expanded in the direction of deviation of the current center line with respect to the past center line. Regardless of the position, the width for expanding the center line portion may be constant. Further, the display control means may change the width for expanding the center line portion according to the position of the past center line in the length direction. The reliability of the central line this time may be the absolute reliability of the central line this time, or the relative reliability of the central line this time with respect to the past central line, or the absolute reliability. And relative reliability may be combined. The central line portion and the guide line can take various line shapes. The center line portion and the guide line are not limited to continuous lines, and may be broken lines or chain lines. Furthermore, the center line portion and the guide line are not limited to thin lines, and may be graphics such as arrows.

  In addition, when the reliability of the center line this time is lower than the predetermined reliability, the display control means may not expand the width in the shift direction for the portion indicating the vehicle guidance destination in the center line portion. Giving the driver the impression that the vehicle destination is ambiguous by keeping the width of the portion indicating the vehicle destination constant without expanding the width of the portion indicating the vehicle destination Can be prevented. On the other hand, the portion indicating the vehicle origin of the center line portion corresponds to a position in the vicinity of the vehicle that is formed larger in the front image than the portion indicating the guidance destination. Therefore, in the front image, when the current center line indicates a position closer to the center position of the lane than the past center line, the driver recognizes the shift of the portion indicating the vehicle guidance source with reference to the past center line. Cheap. Therefore, by extending the width of the portion indicating the vehicle guidance source, the driver is given the impression that the center line portion is displayed greatly deviated from the position indicating the center position of the lane in the current front image. It can be effectively prevented.

  Further, the display control means may generate a guide line including an arrow-shaped center line portion whose head is directed upward in the vertical direction of the front image. In this configuration, when the reliability of the center line this time is lower than the predetermined reliability, the display control means does not expand the width in the shifting direction for the head of the center line portion, and the head of the center line portion. About the part except a part, you may expand a width | variety in a shift | offset | difference direction. Since the width of the head of the arrow is maintained constant, it is possible to prevent the driver from giving an impression that the destination of the vehicle indicated by the head of the arrow is ambiguous.

  Further, when the reliability of the center line this time is lower than the predetermined reliability, the display control means may expand the width of the center line portion wider as the portion indicating the rear position in the lane direction of the center line portion. The portion indicating the rear position in the lane direction in the center line portion corresponds to the position in the vicinity of the vehicle that is largely formed in the front image. Therefore, in the front image, when the current center line indicates a position closer to the center position of the lane than the past center line, the portion indicating the position behind the lane in the center line portion based on the past center line. The driver can easily recognize the deviation. Accordingly, by expanding the width of the portion indicating the rear position in the lane direction, the impression that the center line portion is displayed greatly deviated from the position indicating the center position of the lane in the current front image is displayed to the driver. Can be effectively prevented.

  The display control means may determine that the reliability of the current center line is lower than the predetermined reliability when the edge amount of the image of the lane marking recognized in the current front image is smaller than the predetermined edge amount. An edge is a part where image feature values such as brightness and color in the front image change more than a predetermined reference. The smaller the recognized edge amount, the more parts of the contour of the lane line image cannot be recognized. Therefore, the reliability of the recognized lane line image and the reliability of the center line based on the lane line image are also It can be judged that it becomes lower. The edge amount may be simply an accumulated amount of edges or a length in which the edges are continuous without interruption. Further, the display control means recognizes when the contour line of the lane marking image recognized based on the edge in the front image is recognized as a pair with a distance corresponding to the width of the lane marking image. It may be determined that the edge amount is greater than or equal to a predetermined edge amount. That is, the display control means can recognize the edge amount of the recognized image of the partition line when the edge of the edge amount sufficient to recognize the pair of contour lines separated by a distance corresponding to the width of the image of the partition line cannot be recognized. May be determined to be smaller than the predetermined edge amount. When the contour line in the front image can be recognized as a pair with a distance corresponding to the width of the image of the lane marking, the line indicating the center position in the width direction of the lane marking is specified by the average of the paired contour lines The center line can be accurately identified based on the identified line.

  Further, the display control means, when the amount of deviation between the recognition position of the edge of the lane marking in the current front image and the recognition position of the edge of the lane marking in the previous front image is larger than a predetermined amount, This time, the reliability of the center line may be lower than the predetermined reliability. When the vehicle continuously travels on the road, it is unlikely that the shape of the lane captured in the front image will change significantly. When the recognition position of the edge recognized in the image deviates greatly, it can be considered that the reliability of the recognition position of the edge recognized in the current front image is low.

  Further, the display control means may determine that the reliability of the current center line is lower than the predetermined reliability when the deviation amount of the current center line with respect to the past center line is larger than a predetermined amount. When the vehicle travels continuously on the road, it is unlikely that the shape of the lane captured in the front image will change significantly. If they deviate, it can be considered that the reliability of the central line is low this time. The amount of deviation of the current center line relative to the previous center line may be the difference between the direction of the past center line and the direction of the current center line, or the difference between the position of the past center line and the position of the current center line. May be.

  Factors that reduce the reliability of the central line (for example, the loss of lane markings on the road and the presence of obstacles such as other vehicles) are considered to be temporary. When the deviation amount of the center line this time is larger than a predetermined amount, it is considered that the reliability of the center line this time is high in the current front image. Therefore, the display control means may determine that the reliability of the current center line is equal to or higher than the predetermined reliability when a period in which the reliability of the current central line is lower than the predetermined reliability continues for a predetermined period or longer.

  Furthermore, the method of pointing a lane by a guide line superimposed on the front image as in the present invention can also be applied as a program or method. In addition, the above lane guidance display system, program, and method may be realized as a single device or may be realized by using parts shared with each part provided in the vehicle. Including embodiments. That is, each means constituting the lane guidance display system may be distributed and provided in a plurality of substantial devices. When each means is distributed and provided in a plurality of substantive devices, a communication means for transmitting and receiving data necessary to make each means function may be provided. Furthermore, it is possible to provide a navigation apparatus, method, and program that include at least a part of the lane guidance display system as described above. Further, some changes may be made as appropriate, such as a part of software and a part of hardware. Furthermore, the present invention can be realized as a recording medium for a program for controlling the lane guidance display system. Of course, the software recording medium may be a magnetic recording medium, a magneto-optical recording medium, or any recording medium to be developed in the future.

It is a block diagram of a navigation apparatus. (2A)-(2D) are figures which show a front image. It is a figure which shows a front image. It is a flowchart of a lane guidance display process. It is a figure which shows a front image.

Here, embodiments of the present invention will be described in the following order.
(1) Configuration of navigation device:
(2) Lane guidance display processing:
(3) Other embodiments:

(1) Configuration of navigation device:
FIG. 1 is a block diagram showing a configuration of a navigation device 10 as a lane guidance display system according to an embodiment of the present invention. The navigation device 10 is provided in the vehicle C. The navigation device 10 includes a control unit 20 and a recording medium 30. The control unit 20 includes a CPU, a RAM, a ROM, and the like, and executes a program stored in the recording medium 30 or the ROM. The recording medium 30 records map information 30a. The map information 30a includes node data indicating nodes set corresponding to the end points (intersections) of roads, link data indicating information regarding roads between nodes, and shape interpolation for specifying the shape of roads between nodes. It includes point data and data indicating the features present on the road and its surroundings. The link data includes the number of lanes of the lane included in the road, the lane number of each lane counted from one of the width directions of the road (left side in the present embodiment), and the road that can be entered when the lane travels to the end point of the road. And lane information indicating the line type (line shape, color, etc.) of the lane marking that divides the lane.

  The vehicle C also includes a GPS receiver 41, a vehicle speed sensor 42, a gyro sensor 43, a camera 44, and a user I / F unit 45. The GPS receiver 41 receives radio waves from GPS satellites and outputs a signal for calculating the current position of the vehicle C via an interface (not shown). The vehicle speed sensor 42 outputs a signal corresponding to the rotational speed of the wheels provided in the vehicle C. The gyro sensor 43 outputs a signal corresponding to the angular acceleration acting on the vehicle C. The camera 44 is an image sensor that captures a front image in front of the vehicle C. The camera 44 has a field of view centered on the front front of the vehicle C, and captures a front image of the vehicle C viewed from above. The front image captured by the camera 44 is output to the control unit 20 via an interface (not shown). In the present embodiment, the camera 44 is provided on the back side of the rearview mirror provided at the center position in the width direction of the vehicle C.

  The user I / F unit 45 includes an output device that outputs various guides based on a control signal output from the control unit 20. The output device of the user I / F unit 45 of the present embodiment includes a speaker that outputs guidance by voice and a display as a display unit that displays guidance by image. The display is installed in a direction that the driver of the vehicle C can visually recognize, and is installed so that the vertical direction and the horizontal direction of the image displayed on the display are the vertical direction and the horizontal direction in the vehicle C, respectively. . In the present embodiment, the lane directions of the lanes captured by the camera 44 are all straight and parallel to each other.

The control unit 20 executes a lane guidance display program 21. The lane guidance display program 21 includes a navigation unit 21a, a front image acquisition unit 21b, a center line identification unit 21c, and a display control unit 21d.
The navigation unit 21a is a module that causes the control unit 20 to execute various functions necessary for guiding the planned travel route to the destination. The control unit 20 specifies the current position of the vehicle C based on output signals of the GPS receiving unit 41, the vehicle speed sensor 42, the gyro sensor 43, and the like by the function of the navigation unit 21a. And by the function of the navigation part 21a, the control part 20 specifies the road where the present position of the vehicle C exists as a traveling road with reference to the link data of the map information 30a. Further, the control unit 20 searches the planned travel route from the current position to the destination by a known route search method with reference to the map information 30a by the function of the navigation unit 21a. In the present embodiment, it is assumed that the planned travel route is searched in advance. Note that the control unit 20 may acquire the planned travel route by the function of the navigation unit 21a, and may acquire the planned travel route searched by an external computer through communication.

  The front image acquisition unit 21b is a module that causes the control unit 20 to execute a function of acquiring a front image obtained by photographing the front of the vehicle C. That is, by the function of the front image acquisition unit 21b, the control unit 20 continuously acquires the front image data captured by the camera 44 every predetermined time period.

  FIG. 2A is a diagram illustrating a front image. When the vehicle C travels horizontally, the upper part in the vertical direction on the bisector that bisects the front image in the horizontal direction indicates the front front of the camera 44 (the center position in the width direction of the vehicle C). A vanishing point exists on a bisector that bisects the front image in the horizontal direction. When the vehicle C travels horizontally, the lateral position of the front image corresponds to the horizontal position in the field of view of the camera 44, and particularly corresponds to the position of the lane in the width direction on the lane. On the other hand, the vertical position of the front image depends on the vertical position in the field of view and the distance from the camera 44. Therefore, the higher the position in the vertical direction of the front image, the farther the position on the front side in the traveling direction of the vehicle C is. In the present embodiment, the midpoint CC of the lower side of the front image indicates the center position of the vehicle C in the width direction.

  The center line specifying unit 21c is a module that causes the control unit 20 to execute a function of specifying the center line indicating the center position of the lane in the width direction based on the image of the lane markings recognized in the front image. It is. That is, by the function of the center line specifying unit 21c, the control unit 20 recognizes a part where an image feature value such as brightness and color changes by a predetermined threshold or more at a unit distance in the front image as an edge and distributes the edge in the front image. Get. The control unit 20 recognizes the contour line of the image of the partition line by performing the Hough transform on the edge distribution in the front image by the function of the center line specifying unit 21c.

  FIG. 3 is a diagram illustrating a state in which the center line of the lane is specified in the front image. In FIG. 3, a part of the outline (dashed line) of the image of the two partition lines in the broken line shape is recognized as an edge (thick line), and the two outlines EL are recognized by the Hough transform based on the distribution of the edge. ing. As for the image of the lane marking on the left side of FIG. 3, a pair of contour lines EL (one-dot chain lines) at both ends are recognized on the left and right sides of the image of the lane marking. Only the rightmost contour line EL is recognized. This is because at the left end of the image of the right partition line, an edge exceeding the predetermined edge amount necessary for recognizing the contour line EL in the Hough transform is not recognized.

  By the function of the center line specifying unit 21c, the control unit 20 extracts a pair of contour lines EL from the recognized contour lines EL, and a vector obtained by averaging the paired contour lines EL is approximated to an approximate line BL of the image of the partition line. It is specified as (solid line in the image of the left lane marking). The pair of contour lines EL means two contour lines that are substantially parallel to each other (the difference in direction is within a predetermined angle) and the difference in lateral position on the lower side of the front image is within a predetermined range. To do. The predetermined range is a range in which the width of the image of the lane marking that appears on the lower side of the front image when the lane marking is captured. On the other hand, with respect to the contour line EL for which the counterpart contour line EL could not be extracted by the function of the center line specifying unit 21c, the control unit 20 uses the contour line EL as it is as the approximate line BL (one point of the image of the partition line). Specified as a chain line).

  In addition, the control part 20 specifies the approximate line BL of the image of the division line which averaged the contour lines EL which become a pair by the function of the center line specific | specification part 21c in the following procedures. That is, by the function of the center line specifying unit 21c, the control unit 20 generates auxiliary lines (two-dot chain lines) that traverse the image of the recommended lane in the front image in the horizontal direction at a plurality of heights in the vertical direction. The intersection (white square) with the contour line EL is specified. And the control part 20 specifies the horizontal middle point (white circle) between the intersections of the auxiliary line and each of the paired contour lines EL for each auxiliary line by the function of the center line specifying part 21c. A straight line connecting the midpoints (white circles) is specified as an approximate line BL of a partition line image.

  By the function of the center line specifying unit 21c, the control unit 20 calculates a vector obtained by averaging the approximate lines BL of adjacent lane marking images in the horizontal direction among the approximate lines BL of the lane marking image, as a center position in the width direction of the lane. Is specified as a center line UC. With the function of the center line specifying unit 21c, the control unit 20 causes, for each auxiliary line, the horizontal midpoint between the intersections (white circles) between the auxiliary line and the approximate line BL of the image of the adjacent division line in the horizontal direction ( (White triangle) is specified, and a straight line connecting the midpoints (white triangles) is specified as the lane center line UC. Of the lane images, the lane image closest to the midpoint CC on the lower side of the front image is an image of the traveling lane in which the vehicle C is currently traveling. In the example of FIG. 2A, the center line UC of the traveling lane is Have been identified. In the present embodiment, the travel lane and the recommended lane coincide with each other, and a guide line for guiding that the travel lane should be traveled as it is is superimposed on the front image.

  The display control unit 21d generates a guide line including a center line portion with the center line UC as a reference, and causes the control unit 20 to execute a function of causing the user I / F unit 45 to display a front image on which the guide line is superimposed. It is a module. Specifically, by the function of the display control unit 21d, the control unit 20 performs a process of displaying the front image on which the guide line is superimposed on the user I / F unit 45 every time the front image is acquired at a predetermined time period. To do. Note that the midpoint CC of the lower side of the front image always indicates the center position in the width direction of the vehicle C in any front image taken at any time period. Further, when the vehicle C moves in the width direction of the lane, the image of the lane marking in the front image moves in the horizontal direction.

  When the reliability of the current center line, which is the center line UC identified in the current front image, is greater than or equal to a predetermined reliability, the control unit 20 uses the function of the display control unit 21d to determine the center line with the current center line as a reference. A guide line including a part is generated. Hereinafter, a guide line gl generated when the reliability of the current center line UC (N) specified in the Nth (N is a natural number) front image is equal to or higher than a predetermined reliability will be described with reference to FIGS. 2A and 2B. I will explain. It is assumed that the vehicle C is traveling in the center position in the width direction in the travel lane, and the current center line UC (N) intersects the lower side at the midpoint CC in the front images of FIGS. 2A and 2B.

  With the function of the display control unit 21d, the control unit 20 uses the current center line UC (N) as a reference when the reliability of the current center line UC (N) specified in the Nth forward image is equal to or higher than a predetermined reliability. A guide line gl is generated. That is, by the function of the display control unit 21d, the control unit 20 determines the predetermined value in the vertical direction of the front image when the reliability of the current center line UC (N) specified in the Nth front image is equal to or higher than the predetermined reliability. A front end point FE is generated at the intersection of the horizontal auxiliary line S generated at the position H and the current center line UC (N), and the rear side is located at the intersection of the lower side of the front image and the current center line UC (N). An end point BE is generated. With the function of the display control unit 21d, the control unit 20 generates a linear and thin guide line GL that connects the front end point FE and the rear end point BE on the current center line UC (N). Furthermore, the function of the display control unit 21d causes the control unit 20 to generate an arrow-shaped guide line gl that is line-symmetric with respect to the thin guide line GL and whose head is directed upward in the vertical direction of the front image. A front image on which the guide line gl is superimposed is displayed on the user I / F unit 45.

  On the other hand, when the reliability of the center line UC (N) is lower than the predetermined reliability this time, the control unit 20 uses the function of the display control unit 21d so that the past center line UC whose reliability finally becomes equal to or higher than the predetermined reliability. Including the center line portion based on the past center line UC (NM), and the width of the center line portion is the direction of deviation of the current center line UC (N) with respect to the past center line UC (NM) A guide line gl extended to is generated. It is assumed that the past center line UC (N−M) whose reliability is equal to or higher than the predetermined reliability in the (N−M) th front image is specified last. That is, it is assumed that the reliability of the center lines (N−M + 1) to UC (N) specified in the (N−M + 1) to Nth forward images is continuously lower than the predetermined reliability.

  Hereinafter, the guide line gl generated when the reliability of the current center line UC (N) specified in the Nth forward image is lower than the predetermined reliability will be described with reference to FIGS. 2C and 2D. 2C and 2D, the past center line UC (NM) intersects the lower side at the midpoint CC. When the reliability of the current center line UC (N) (two-dot chain line) specified in the Nth forward image is lower than the predetermined reliability by the function of the display control unit 21d, finally, the reliability Includes a center line portion based on a past center line UC (NM) (one-dot chain line) having a predetermined reliability or higher, and the width of the center line portion with respect to the past center line UC (NM) This time, a guide line gl extended in the direction of deviation of the center line UC (N) is generated.

  That is, by the function of the display control unit 21d, the control unit 20 generates the front end point FE at the intersection of the auxiliary line S and the past center line UC (NM), and the lower side of the front image and the past center line UC (N A rear end point BE is generated at the intersection with -M). Due to the function of the display control unit 21d, the control unit 20 causes the arrow-shaped guide line gl to be symmetric with respect to the guide line GL connecting the front end point FE and the rear end point BE on the past center line UC (NM). Is generated. Further, by the function of the display control unit 21d, the control unit 20 has an extended portion gle (hatching) that expands the width of the guide line gl in the direction of deviation of the current center line UC (N) with respect to the past center line UC (NM). The user I / F unit 45 displays a front image generated and superimposed with the guide line gl including the extended portion gle. In the present embodiment, the shift direction is from the intersection (BE, CC) between the lower side of the front image and the past center line UC (NM) to the intersection of the lower side of the front image and the current center line UC (N). The direction.

  With the function of the display control unit 21d, the control unit 20 has the current center line UC (N) with respect to the past center line UC (NM) for the head indicating the guidance destination of the vehicle C among the arrow-shaped guide lines gl. For the portion (the portion excluding the head and below the head in the vertical direction) indicating the vehicle C guidance source in the guide line gl without extending the width in the shift direction, the past center line UC The width is expanded in the shift direction of the current center line UC (N) with respect to (NM). Moreover, the control part 20 expands the width | variety of the guide line gl widely by the function of the display control part 21d, so that the part which shows the position back in the lane direction among the guide lines gl. That is, by the function of the display control unit 21d, the control unit 20 is a triangle that is adjacent to the past center line UC (N-M) in the direction of deviation of the current center line UC (N) with respect to the portion excluding the head of the guide line gl. The extended portion gle is generated with the lower end of the head of the guide line gl as the vertex and the lower part of the front image as the base. In the present embodiment, the function of the display control unit 21d causes the control unit 20 to start from the intersection (BE, CC) between the lower side of the front image and the past center line UC (NM) from the lower side of the front image and the current center line UC. Let the length to the intersection with (N) be the length of the base of the extended portion gle.

Next, the reliability of the center line UC will be described. With the function of the display control unit 21d, the control unit 20
(Condition 1) The edge amount of the lane marking image recognized in the current front image is smaller than the predetermined edge amount,
(Condition 2) The deviation amount of the current center line UC (N) with respect to the past center line UC (NM) is larger than a predetermined deviation amount.
(Condition 3) The period when the reliability of the center line UC (N) is lower than the predetermined reliability this time does not continue for a predetermined period or more,
If all of the above are satisfied, it is assumed that the reliability of the center line is lower than the predetermined reliability.

  (Condition 1) will be specifically described. In the present embodiment, when the control unit 20 specifies the center line UC (N) in the Nth forward image by the function of the display control unit 21d, the image of the left and right dividing lines sandwiching the center line UC (N) is displayed. When at least one of the paired contour lines LE is not recognized, it is determined that the edge amount of the lane marking image is smaller than the predetermined edge amount. In the example of FIG. 3, since the leftmost contour line LE is not recognized for the image of the lane marking on the right side of the travel lane, it is determined that the edge amount of the lane marking image is smaller than the predetermined edge amount.

  Next, (Condition 2) will be specifically described. In the present embodiment, the control unit 20 uses the function of the display control unit 21d to cause the past center line UC (NM), in which the reliability finally exceeds the predetermined reliability, as illustrated in FIG. 2C, and the Nth forward image. The acute angle θ that intersects with the current center line UC (N) specified in (1) is specified. Then, by the function of the display control unit 21d, the control unit 20 allows the current center line UC (N) as the current center line with respect to the past center line UC (NM) when the specified acute angle θ is larger than a predetermined angle. It is determined that the amount of deviation is larger than the predetermined amount of deviation. In the case of FIG. 2A, the past center line UC (NM) and the current center line UC (N) specified in the Nth forward image substantially coincide, and at least (Condition 1) is not satisfied at this time. It is determined that the reliability of the center line UC (N) is greater than or equal to the predetermined reliability.

  Next, (Condition 3) will be specifically described. By the function of the display control unit 21d, the control unit 20 causes the reliability of the current center line UC (N) when the reliability of the current center line UC (N) is lower than the predetermined reliability for a predetermined period or longer. Is lower than the predetermined reliability. In other words, the function of the display control unit 21d allows the control unit 20 to perform the reliability of the current center line UC (N) when the reliability of the current center line UC (N) is lower than the predetermined reliability for a predetermined period or longer. It is assumed that the degree is equal to or higher than a predetermined reliability. Specifically, the function of the display control unit 21d causes the control unit 20 that the number of times that the reliability of the center line UC (N) is determined to be continuously lower than the predetermined reliability has not reached the predetermined number of times. In this case, it is determined that a period in which the reliability of the center line UC (N) is lower than the predetermined reliability has not continued for a predetermined period.

  In the present embodiment described above, the function of the display control unit 21d allows the control unit 20 to use the current center line UC (N) as a reference when the reliability of the current center line UC (N) is equal to or higher than a predetermined reliability. A guide line gl is generated. Thereby, when the current center line UC (N) specified in the latest front image can be trusted, the guide line gl can be generated with the current center line UC (N) as a reference, and the reliability It is possible to generate a guide line gl that is high and has high followability to the front image.

  On the other hand, when the reliability of the current center line (N) is lower than the predetermined reliability in the Nth forward image, the control unit 20 uses the past center line (N−M) as a reference. A guide line gl is generated. Therefore, it is possible to prevent the guide line gl from fluctuating following the current center line (N) whose reliability is lower than the predetermined reliability. Further, the function of the display control unit 21d allows the control unit 20 to use the past center line (N−M) as a reference when the reliability of the current center line (N) is lower than the predetermined reliability in the Nth forward image. The width of the guide line gl is expanded in the shift direction of the current center line (N) with respect to the past center line (NM). Thus, even when the current center line (N) indicates a position closer to the center position of the lane than the past center line (NM) in the front image, the guide line gl increases from the position indicating the center position of the lane. It is possible to prevent the driver from giving the impression that the images are displayed in a shifted manner.

  Further, although the reliability of the Nth center line UC (N) is lower than the predetermined reliability, the center line UC () identified by the Nth front image whose acquired timing is close to the (N + 1) th front image N), it can be said to approximate the (N + 1) th current center line UC (N + 1). That is, before the reference of the guide line gl changes from the past center line UC (NM) to the current center line UC (N + 1) between the Nth front image and the (N + 1) th front image, The width of the guide line gl can be expanded in advance to the Nth center line UC (N) side that approximates the current center line UC (N + 1) as the transition destination. As a result, the fluctuation of the guide line gl when the reference of the guide line gl changes from the past center line UC (N−M) to the current center line UC (N + 1) can be reduced.

  Further, the function of the display control unit 21d allows the control unit 20 to use the arrow-shaped guide line gl as the past center line UC (NM) when the reliability of the current center line UC (N) is lower than the predetermined reliability. The head indicating the guidance destination of the vehicle C in the guide line gl is not expanded in the direction in which the current center line UC (N) deviates from the past center line UC (NM). Thus, by maintaining the head width indicating the guidance destination of the vehicle C constant, it is possible to prevent the driver from giving an impression that the guidance destination of the vehicle C is ambiguous. On the other hand, a portion of the guide line gl indicating the guidance source (a portion excluding the head and below the head in the vertical direction) is formed larger in the front image than the head indicating the guidance destination. Corresponds to a position near the vehicle. Therefore, in the case where the current center line UC (N) indicates a position closer to the center position of the lane in the front image than the past center line UC (N-M), the guidance source of the vehicle C in the current front image is indicated. The deviation of the part is easily recognized by the driver. Therefore, by expanding the width of the portion indicating the guidance source of the vehicle C, the driver is given an impression that the guide line gl is displayed greatly deviated from the position indicating the center position of the lane in the current front image. Can be effectively prevented.

  Furthermore, when the reliability of the center line UC (N) is lower than the predetermined reliability, the control unit 20 uses the function of the display control unit 21d to guide the guide line gl toward the portion indicating the rear position in the lane direction. The width of gl is expanded widely. The portion of the guide line gl that indicates the rear position in the lane direction corresponds to the position in the vicinity of the vehicle that is largely formed in the front image. Therefore, when the current center line UC (N) indicates a position closer to the center position of the lane than the past center line UC (NM) in the front image, the position behind the lane direction in the guide line gl is determined. The driver is more likely to recognize the deviation as shown. Accordingly, by expanding the width of the guide line gl in the portion indicating the rear position in the lane direction, the center line portion is displayed greatly deviated from the position indicating the center position of the lane in the current front image. It is possible to effectively prevent the driver from giving an impression.

  Further, the function of the display control unit 21d allows the control unit 20 to trust the current center line UC (N) when the edge amount of the image of the lane marking recognized in the Nth front image is smaller than the predetermined edge amount. It is assumed that the degree is lower than the predetermined reliability. The smaller the recognized edge amount, the more parts of the contour of the lane line image cannot be recognized. Therefore, the reliability of the recognized lane line image and the reliability of the center line based on the lane line image are also It can be judged that it becomes lower.

  Further, the function of the display control unit 21d allows the control unit 20 to determine the reliability of the current center line UC when the deviation amount of the current center line UC (N) with respect to the past center line UC (NM) is larger than a predetermined amount. Is lower than the predetermined reliability. When the vehicle C continuously travels on the road, it is unlikely that the shape of the lane photographed in the front image will change greatly. Therefore, the past center line UC (NM) having a reliability higher than a predetermined reliability. ) From the current center line UC (N) can be considered to be low in the reliability of the current center line UC (N).

  Further, the function of the display control unit 21d allows the control unit 20 to change the reliability of the current center line UC (N) when the reliability of the current center line UC (N) is lower than the predetermined reliability for a predetermined period or longer. It is assumed that the degree is equal to or higher than a predetermined reliability. Factors that decrease the reliability of the central line UC (N) this time (for example, the loss of lane markings on the road, the presence of obstacles such as other vehicles, etc.) are considered to be temporary, and therefore are If the deviation amount of the current center line UC (N) with respect to the past center line UC (NM) becomes larger than a predetermined amount over the past center line UC (NM), It can be determined that the reliability is high.

(2) Lane guidance display processing:
Next, the lane guidance display process executed by the function of the lane guidance display program 21 will be described. FIG. 4 is a flowchart of the lane guidance display process. The lane guidance display process is a loop process that is executed each time the control unit 20 acquires a front image at a predetermined time period by the function of the front image acquisition unit 21b. Here, the N-th process for the N-th acquired front image will be described. With the function of the front image acquisition unit 21b, the control unit 20 acquires a front image captured by the camera 44 (S100).

  Next, by the function of the center line specifying unit 21c, the control unit 20 specifies the current center line UC (N) of the image of the traveling lane in which the vehicle C is currently traveling in the front image (S110). Specifically, the control unit 20 recognizes the edge of the lane marking image in the front image by the function of the center line specifying unit 21c, and recognizes the contour line EL of the lane marking image by Hough transform based on the distribution of the edge. (Figure 3). Further, by the function of the center line specifying unit 21c, the control unit 20 specifies a vector obtained by averaging the paired contour lines EL among the recognized contour lines EL of the image of the dividing line as an approximate line BL of the dividing line image. For the contour line EL for which the paired partner contour line EL could not be recognized, the contour line EL is directly specified as the approximate line BL of the image of the partition line. Then, by the function of the center line identification unit 21c, the control unit 20 identifies, as a lane image, a region sandwiched between approximate lines BL of adjacent division line images in the front image, and out of the lane images. The vehicle lane image is identified as the lane image closest to the midpoint CC on the lower side indicating the center position in the width direction of the vehicle C. Further, a vector obtained by averaging the approximate lines BL of the left and right lane markings that divide the image of the travel lane is specified as the current center line UC (N) indicating the center position in the width direction of the travel lane.

  Next, by the function of the display control unit 21d, the control unit 20 determines whether or not the reliability of the current center line UC (N) is lower than the predetermined reliability (S115). Specifically, by the function of the display control unit 21d, the control unit 20 determines whether or not (Condition 1) to (Condition 3) described above are satisfied, and if all are satisfied, the center line UC ( It is determined that the reliability of N) is lower than the predetermined reliability.

  If it is not determined that the reliability of the center line UC (N) is lower than the predetermined reliability this time, the control unit 20 uses the function of the display control unit 21d to change the arrow shape with the center line UC (N) as a reference. A guide line gl is generated (S120). That is, by the function of the display control unit 21d, the control unit 20 sets the front end point FE and the rear end point BE on the current center line UC (N) and generates a thin guide line GL connecting them. To do. Then, by the function of the display control unit 21d, the control unit 20 generates a guide line gl that is symmetric with respect to the thin guide line GL (FIG. 2B).

  Next, the control unit 20 causes the user I / F unit 45 to display a front image on which the guide line gl is superimposed by the function of the display control unit 21d (S130). That is, when the current center line UC (N) of the travel lane specified in the current front image is reliable, the front image on which the guide line gl with the current center line UC (N) as a reference is superimposed is displayed as the user I / It is displayed on the F unit 45. Thereby, it is possible to display the guide line gl with high reliability and high followability to the front image.

  On the other hand, when it is determined that the reliability of the center line UC (N) is lower than the predetermined reliability, the control unit 20 finally determines that the reliability is equal to or higher than the predetermined reliability by the function of the display control unit 21d. Further, an arrow-shaped guide line gl with reference to the past center line UC (NM) is generated (S135). That is, by the function of the display control unit 21d, the control unit 20 sets the front end point FE and the rear end point BE on the past center line UC (NM), and the thin guide line GL connecting them. Is generated. Then, by the function of the display control unit 21d, the control unit 20 generates an arrow-shaped guide line gl that is line-symmetric with respect to the thin-line guide line GL (FIG. 2D). And the control part 20 expands the width | variety of the guide line gl by the function of the display control part 21d in the shift | offset | difference direction of this center line UC (N) with respect to the past center line UC (NM) (S145). That is, an extension portion gle (hatching) is generated, and a guide line gl including the extension portion gle adjacent to the guide line gl before expansion from the shift direction is generated (FIG. 2D).

  Next, the control unit 20 causes the user I / F unit 45 to display a front image on which the guide line gl including the extended portion gle is superimposed by the function of the display control unit 21d (S130). That is, when the current center line UC (N) of the travel lane specified in the current front image is not reliable, the reliability is determined not to be the current center line UC (N) but to be equal to or higher than the predetermined reliability. In addition, the user I / F unit 45 displays a front image in which the guide line gl is superimposed on the past center line UC (NM). Thereby, the guide line gl with high reliability can be displayed. Further, since the width of the guide line gl is expanded in the direction of deviation of the current center line UC (N) with respect to the past center line UC (NM), the current center line UC (N) is past the past center line UC (N Even when the position closer to the center position of the driving lane than -M) is shown, it is possible to prevent the driver from giving the impression that the guide line gl is displayed shifted in the front image.

(3) Other embodiments:
In the embodiment, only when all of the above (Condition 1) to (Condition 3) are satisfied, the reliability of the center line UC (N) is lower than a predetermined reliability (Condition 1). ), If the edge amount is smaller than the predetermined edge amount, the reliability of the current center line UC (N) is determined to be equal to or higher than the predetermined reliability, and the guide line gl with reference to the current center line UC (N) is determined. Will be generated. That is, even if the recognized edge amount is smaller than the predetermined edge amount and there is a possibility that an absolute error has occurred in the position and direction of the current center line UC (N), the current center line UC (N) Since the reference guide line gl is generated, the guide line gl may fluctuate due to an error.

  Therefore, the control unit 20 corrects the current center line UC (N) so as to approach the past center line UC (NM) when (condition 1) is satisfied by the function of the display control unit 21d, and the correction. The guide line gl based on the past center line UC (NM) may be generated. Thereby, the influence of the error of the current center line UC (N) due to the shortage of the edge amount can be reduced. For example, the function of the display control unit 21d causes the control unit 20 to convert a vector representing the past center line UC (N−M) by a coefficient α of 0 to 1 and a vector representing the current center line UC (N). The current center line UC (N) may be corrected so as to approach the past center line UC (N−M) by adding the vector multiplied by the coefficient (1−α). Note that by correcting the current center line UC (N) so as to approach the past center line UC (NM), the above-described (Condition 2) becomes difficult to be satisfied, and the reliability of the current center line UC (N). Is less than a predetermined reliability.

  When the vehicle C continuously travels on the road, it is unlikely that the shape of the lane captured in the front image will change significantly. If the recognition position of the edge recognized in the front image is greatly shifted, it can be considered that the reliability of the recognition position of the edge recognized in the current front image is low. Accordingly, the function of the display control unit 21d allows the control unit 20 to recognize the edge of the lane marking image in the Nth front image and the recognition position of the lane marking image edge in the (N-1) th front image. It may be added as a (condition 4) for determining that the reliability of the center line UC (N) is smaller than the predetermined reliability. The control unit 20 determines that the reliability of the current center line UC (N) is lower than the predetermined reliability when all of (Condition 1) to (Condition 4) are satisfied by the function of the display control unit 21d. Also good. Further, the function of the display control unit 21d allows the control unit 20 to satisfy both of (Condition 2) and (Condition 3), and when at least one of (Condition 1) and (Condition 4) is satisfied. This time, the reliability of the center line UC (N) may be lower than the predetermined reliability.

  Note that, by the function of the display control unit 21d, the control unit 20 causes the shift amount between the position of the edge before the Hough transform in the (N-1) th front image and the position of the edge before the Hough transform in the Nth front image. It may be determined whether or not is larger than a predetermined deviation amount. Furthermore, the function of the display control unit 21d allows the control unit 20 to calculate the contour line EL after the Hough transform in the (N-1) th front image and the edge contour line EL after the Hough transform in the Nth front image. It may be determined whether or not the deviation amount is larger than a predetermined deviation amount. The deviation amount of the contour line EL may be an angle difference between the contour lines EL, or may be a difference between positions of the contour line EL on the lower side of the front image.

  In the above embodiment, the case where the recommended lane and the traveling lane coincide with each other is illustrated. However, when the recommended lane and the traveling lane are different, the control unit 20 switches from the traveling lane to the recommended lane by the function of the display control unit 21d. A guide line gl to be changed in lane may be generated. For example, as shown in FIG. 5, a thin guide line GL including a recommended lane side guide line portion RL in the recommended lane image at the left end and a travel lane side guide line portion UL in the image of the center travel lane. And an arrow-shaped guide line gl with reference to the guide line GL may be generated. By the function of the display control unit 21d, the control unit 20 determines whether or not the reliability is lower than a predetermined reliability for each of the current center line UC (N) of the traveling lane and the current center line RC (N) of the recommended lane. judge.

  When the reliability of the current center line UC (N) (two-dot chain line) of the traveling lane is lower than the predetermined reliability, the control unit 20 finally has a reliability higher than the predetermined reliability by the function of the display control unit 21d. The travel lane side guide line portion UL is generated on the past center line UC (NM) (one-dot chain line) of the travel lane, and the current center line UC (N) with respect to the past center line UC (NM) It is only necessary to generate an extended portion gle1 in which the width of the guide line gl is extended in the shifting direction. On the other hand, when the reliability of the current center line UC (N) of the travel lane is equal to or higher than the predetermined reliability, the travel lane side guide line portion UL may be generated on the current center line UC (N). Similarly, when the reliability of the current center line RC (N) (two-dot chain line) of the recommended lane is lower than the predetermined reliability by the function of the display control unit 21d, the reliability is finally set to the predetermined reliability. The recommended lane side guide line portion RL is generated on the past center line RC (NK) of the recommended lane (the dashed line, K is a natural number equal to or less than N), and the past center line RC (NK). It is only necessary to generate an expanded portion gle2 in which the width of the guide line gl is expanded in the direction of deviation of the current center line RC (N). On the other hand, when the reliability of the current center line RC (N) of the recommended lane is equal to or higher than the predetermined reliability, the travel lane side guide line portion UL may be generated on the current center line RC (N).

  DESCRIPTION OF SYMBOLS 10 ... Navigation apparatus, 20 ... Control part, 21 ... Lane guidance display program, 21a ... Navigation part, 21b ... Front image acquisition part, 21c ... Centerline specific part, 21d ... Display control part, 30 ... Recording medium, 30a ... Map Information 41... Reception unit 42 42 vehicle speed sensor 43. Gyro sensor 44 44 camera 45 user I / F unit BE rear end point BL ... lane line approximation line C vehicle v CC ... mid point, FE ... front end point, GL, gl ... guide line, H ... fixed position, UC ... center line of driving lane, RC ... center line of recommended lane, UL ... driving lane side guide line part, RL ... recommended Lane side guide line part.

Claims (10)

Forward image acquisition means for acquiring a forward image obtained by photographing the front of the vehicle;
Based on the image of the lane markings recognized in the front image, center line specifying means for specifying the center line indicating the center position of the lane in the width direction in the front image;
Display control means for generating a guide line including a center line portion with respect to the center line and displaying the front image on which the guide line is superimposed on a display unit;
The display control means includes
When the reliability of the current center line, which is the center line specified in the current front image, is greater than or equal to a predetermined reliability, the guide line including the center line portion based on the current center line is generated. ,
If the reliability of the current center line is lower than the predetermined reliability, including the center line portion based on the past center line, which is the past center line whose reliability is equal to or higher than the predetermined reliability. And generating the guide line in which the width of the center line portion is expanded in the direction of deviation of the current center line with respect to the past center line,
Lanes guidance display system. When the reliability of the current center line is lower than the predetermined reliability, the display control means does not expand the width in the shift direction for the portion indicating the vehicle destination among the center line portions, For the portion indicating the vehicle origin of the center line portion, the guide line having an expanded width in the shift direction is generated.
The lane guidance display system according to claim 1. When the reliability of the current center line is lower than the predetermined reliability, the display control means includes the center line portion having an arrow shape with the head facing upward in the vertical direction of the front image, and the center For the head portion of the line portion, the width is not expanded in the shift direction, and for the portion other than the head portion of the center line portion, the guide line is generated with the width expanded in the shift direction.
The lane guidance display system according to claim 2. When the reliability of the current center line is lower than the predetermined reliability, the display control means expands the width of the center line portion wider as the portion indicating the rear position in the lane direction of the center line portion. Generate guide lines,
The lane guidance display system according to any one of claims 1 to 3. The display control means determines that the reliability of the current center line is lower than the predetermined reliability when the edge amount of the image of the lane marking recognized in the current forward image is smaller than the predetermined edge amount. ,
The lane guidance display system according to any one of claims 1 to 4. The display control means, when the amount of deviation between the edge recognition position of the lane marking in the current front image and the edge recognition position of the lane marking in the previous front image is larger than a predetermined amount. , Suppose that the reliability of the central line is lower than the predetermined reliability,
The lane guidance display system according to any one of claims 1 to 5. The display control means, when the deviation amount of the current center line with respect to the past center line is larger than a predetermined amount, the reliability of the current center line is lower than the predetermined reliability,
The lane guidance display system according to any one of claims 1 to 6. The display control means, when the reliability of the current center line is lower than the predetermined reliability continues for a predetermined period or longer, the reliability of the current central line is equal to or higher than the predetermined reliability.
The lane guidance display system according to any one of claims 1 to 7. A front image acquisition means for acquiring a front image obtained by photographing the front of the vehicle; and
A centerline specifying step for specifying a centerline indicating the center position of the lane in the width direction in the front image based on the image of the lane markings recognized in the front image by the centerline specifying means ;
A display control step for generating a guide line including a center line portion with respect to the center line, and displaying the front image on which the guide line is superimposed on a display unit,
The display control means includes
When the reliability of the current center line, which is the center line specified in the current front image, is greater than or equal to a predetermined reliability, the guide line including the center line portion based on the current center line is generated. ,
If the reliability of the current center line is lower than the predetermined reliability, including the center line portion based on the past center line, which is the past center line whose reliability is equal to or higher than the predetermined reliability. And generating the guide line in which the width of the center line portion is expanded in the direction of deviation of the current center line with respect to the past center line ,
Lane guidance display method. A front image acquisition means for acquiring a front image obtained by photographing the front of the vehicle; and
A center line specifying function , based on the image of the lane markings recognized in the front image, a center line specifying function for specifying a center line indicating the center position of the lane in the width direction in the front image;
The display control means generates a guide line including a center line portion with respect to the center line, and causes a computer to realize a display control function for displaying the front image on which the guide line is superimposed on a display unit,
The display control means includes
When the reliability of the current center line, which is the center line specified in the current front image, is greater than or equal to a predetermined reliability, the guide line including the center line portion based on the current center line is generated. ,
If the reliability of the current center line is lower than the predetermined reliability, including the center line portion based on the past center line, which is the past center line whose reliability is equal to or higher than the predetermined reliability. And generating the guide line in which the width of the center line portion is expanded in the direction of deviation of the current center line with respect to the past center line,
Lane guidance display program.

Images