JP2006268865A - Navigation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a navigation device capable of keeping safety of a travelling vehicle by imaging the forward side from the travelling vehicle and performing image processing. <P>SOLUTION: The navigation device images the forward side of the vehicle, determines a movement vector for showing the motion of an object in the picked up image from time t to time t+Δt by comparing it with an image obtained with a CCD camera 19 during the travel of the vehicle, determines an allowance degree to an end of the road under travel based on the detected movement vector, and informs a driver based on the determination result. The navigation device calibrates the imaging direction of the CCD camera 19 based on the detection result of the movement vector. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention belongs to a technical field such as a navigation device for obtaining a road edge based on an image from an imaging unit in a traveling vehicle.

  2. Description of the Related Art Conventionally, navigation apparatuses that detect a current position of a vehicle as a moving body, display the detected current position together with a surrounding road map on a display screen, and perform route guidance have been widely used. This type of navigation device includes a large-capacity recording medium in which map data is stored, and is used to generate display data on a display screen. The map data on the recording medium includes road width information associated with the road shape data and various display data. Therefore, the road width information corresponding to the road on which the vehicle is traveling can be read and the road width of the currently traveling road can be grasped.

  However, in the conventional navigation device described above, the road width information included in the map data does not indicate an accurate road width, but merely indicates an approximate road width of the road, for example, 5.5 m to 8 m. Therefore, detailed and accurate road width information corresponding to road points could not be acquired.

  In addition to simply knowing the width of a road, there is an increasing need for obtaining a distance to a road edge in real time and using it for safe driving of a vehicle. On the other hand, if you install multiple cameras and apply the principle of triangulation, you can measure the distance three-dimensionally, and you can find the distance to the road edge. It becomes a factor of up.

  Therefore, the present invention has been made in view of such a problem, and an image processing apparatus capable of obtaining a distance to a road edge such as a wall surface and a road width based on an image taken from a traveling vehicle with a simple configuration. Another object of the present invention is to provide a navigation device capable of giving a warning and guidance for ensuring safe driving to a driver according to the result of image processing.

  In order to solve the above-mentioned problem, the navigation device according to claim 1, an imaging unit that images the front of the vehicle and outputs an image, and a movement vector indicating a movement of the object in the image from time t to t + Δt A movement vector detection means obtained by comparing images obtained by the imaging means during traveling of the vehicle, and a margin to the road edge of the traveling road is determined based on the detected movement vector. A margin determination unit; and a control unit that controls the notification unit based on the determination result of the margin determination unit, and calibrates the imaging direction of the imaging unit based on the detection result of the movement vector detection unit. It has a calibration means.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments of the invention will be described with reference to the drawings. In the following embodiments, a case where the present invention is applied to a navigation device mounted on a vehicle will be described.

  FIG. 1 is a diagram illustrating a schematic configuration of a navigation device according to the present embodiment. 1 includes a control unit 11, a map data storage unit 12, a road width data storage unit 13, a sensor unit 14, a current position detection unit 15, a display 16, a speaker 17, and an image processing unit. 18 and a CCD camera 19.

  In the above configuration, the control unit 11 controls the operation of the entire navigation device. The control unit 11 includes a CPU and the like, reads out and executes a control program stored in a ROM (not shown), sends control signals to each component of the navigation apparatus, and inputs / outputs data.

  The map data storage unit 12 is a memory having a large storage capacity for storing map data. For example, a CD-ROM or a DVD-ROM is used. The map data stored in the map data storage unit 12 includes road shape data, name data, background data, and the like.

  The road width data storage unit 13 is a memory that records the road width data during traveling of the vehicle obtained along with the image processing according to the present invention in an updatable manner. The map data in the map data storage unit 12 includes simple road width information, but the road width data stored in the road width data storage unit 13 is more detailed data, and the road width that changes according to the position of the road is displayed. It is the data obtained accurately. A method for generating road width data will be described later.

  The sensor unit 14 includes various sensors necessary for detecting the current position of the vehicle. Specifically, it includes a vehicle speed sensor that detects the speed of the vehicle, a direction sensor that detects the amount of change in the direction of the vehicle, a GPS receiver that receives radio waves from a GPS (Global Positioning System) satellite, and the like. .

  The current position detection unit 15 calculates the current position of the vehicle based on the output signal from the sensor unit 14 and outputs current position data. The current position data is collated with the above-described map data by the control unit 11 and corrected by a map matching process or the like.

  On the display 16, map data is displayed in various modes under the instruction of the control unit 11, and the current position of the vehicle is displayed as a car mark superimposed on the map data. The display 16 is composed of, for example, a CRT, a liquid crystal display element, or the like. In addition, guidance information along the route of the vehicle is output from the speaker 17 by voice, and guidance voice to be described later is output in relation to the image processing according to the present invention.

  The image processing unit 18 is a unit that analyzes an image signal from the CCD camera 19 installed in the vehicle and performs image processing according to the present embodiment. Here, the configuration and operation of the image processing unit 18 will be described with reference to FIG.

  As shown in FIG. 2, the image processing unit 18 includes an A / D converter 21, a first image memory 22, a second image memory 23, a movement vector detection unit 24, and a movement vector processing unit 25. Has been. The image control unit 18 operates in accordance with a control signal from the control unit 11.

  In FIG. 2, an A / D converter 21 converts an analog image signal based on an image captured by the CCD camera 19 into a digital image signal. Usually, the digital image signal output from the A / D converter 21 constitutes one frame image data for every predetermined frame period, and consists of a plurality of continuous frame image data.

  The first image memory 22 and the second image memory 23 each store frame image data output from the A / D converter 21. The first image memory 22 stores the latest frame image data, and the second image memory 23 stores the previous frame image data. Therefore, the image processing unit 18 always holds the latest two frames of image data, and the processing described later is performed using these data.

  The movement vector detection unit 24 detects a movement vector for each area in the image based on the frame image data stored in the first image memory 22 and the second image memory 23. At this time, vehicle speed data output from the sensor unit 14 is used.

  The movement vector processing unit 25 obtains a horizontal distance to a road edge such as a wall surface on the front side of the traveling road based on the movement vector detected by the movement vector detection unit 24 and a road width based on the distance. Output as data.

  Details of the processes in the movement vector detection unit 24 and the movement vector processing unit 25 will be described later.

  Returning to FIG. 1, a CCD camera 19 as an imaging means of the present invention is installed at a predetermined position of the vehicle, images the front direction of the vehicle, and outputs an image signal. Here, FIG. 3 shows an installation state of the CCD camera 19 in the vehicle.

  FIG. 3A is a diagram illustrating the installation state of the CCD camera 19 as viewed from above the vehicle. As shown in FIG. 3A, the CCD camera 19 is attached near the center of the upper part of the vehicle interior. The CCD camera 19 is fixed at a position where the front of the vehicle can be imaged straight.

  FIG. 3B is a diagram showing the installation state of the CCD camera 19 viewed from the side of the vehicle together with the imaging range. As shown in FIG. 3B, the CCD camera 19 images the range of the angle of view θ toward the front of the vehicle. Usually, the center of the captured image is made to coincide with the road surface in the horizontal direction. The CCD camera 19 is attached at a position of height H from the road surface. For example, H = about 1.5 (m).

  Next, the principle of image processing according to this embodiment will be described with reference to FIGS.

  FIG. 4 is a diagram showing an image captured by the CCD camera 19 described above using a pinhole camera model. In FIG. 4, an image plane P0 corresponding to the light receiving element portion of the CCD camera 19 and a focal plane F corresponding to the lens portion are arranged in parallel with a focal length f therebetween. A lens center C of the focal plane F corresponds to a pinhole, and a spatial coordinate system (X, Y, Z) is considered around this lens center C. On the other hand, on the image plane P0, the camera coordinate system (x, y) is considered with the spatial coordinate (0, 0, −f) as the image center c.

Here, a point M (Xm, Ym, Zm) in the spatial coordinate system is considered. This point M can be made to correspond to a predetermined position on the road surface or wall surface within the field of view of FIG. Considering the central projection for this point M, the point M, the lens center C of the focal plane F, and the image plane P0 are connected by a straight line and projected onto the point m (xm, ym) of the camera coordinate system. At this time, the relationship between the point M (Xm, Ym, Zm) and the projected point m (xm, ym) is expressed by the following equation.
[Equation 1]
xm = f · Xm / Zm
ym = f · Ym / Zm

  Next, FIG. 5 is a diagram showing the relationship between the spatial coordinate system (X, Y, Z) and the pixel coordinate system (u, v) of the image data, corresponding to FIG. In FIG. 5, for simplicity, the image plane P0 in FIG. 4 is arranged in front of the focal plane F, and the x-axis and y-axis of the camera coordinate system (x, y) are inverted to obtain the u-axis and v-axis, respectively. Thus, the pixel image system (u, v) is obtained. Even if the arrangement in FIG. 4 is replaced with the arrangement in FIG. 5, the same relationship holds because they are equivalent to each other.

  In FIG. 5, the image plane P1 surface of the pixel image system corresponds to a display image having the number of pixels Nw in the horizontal direction and the number of pixels Nh in the vertical direction, and the image plane P1 includes a total of Nw × Nh pixels. It is. On the other hand, the image plane P1 coincides with the light receiving area of the CCD camera 19 having the size of the horizontal width w and the height h. Then, considering the point M (X, Y, Z) in the spatial coordinate system as in FIG. 4, as shown in FIG. 5, the lens on the focal plane F passes through the point m (u, v) on the image plane P1. A straight line is connected to the center C.

Here, in the same way as in FIG. 4, considering the central projection with respect to the point m, the relationship between the point M (X, Y, Z) and the projected point m (u, v) corresponding to Equation 1 is It is represented by
[Equation 2]
u = (f · X / Z) · Nw / w
v = (f · Y / Z) · Nh / h

  That is, Equation 2 represents the pixel position of the point m (u, v) on the image plane P1. In Equation 2, u corresponds to the horizontal pixel position, and v corresponds to the vertical pixel position.

In this embodiment, since an image captured from a running vehicle is processed, it is necessary to assume the movement of an object in the image plane P1. Therefore, it is necessary to obtain the optical flow in the image plane P1, that is, the movement vector V (u, v). When the brightness I of the same point in the three-dimensional space is kept constant, the following spatiotemporal differential equation holds.
[Equation 3]
dI / dt = Ix · u + Iy · v + It = 0
However, in the image plane P1, Ix is the partial differential in the horizontal direction, Iy is the partial differential in the vertical direction, It is the partial differential in the time axis direction, and u and v are the horizontal and vertical directions of the movement vector V described above. It is an ingredient.

  Next, a method for obtaining the movement vector V will be described with reference to FIGS. As shown in FIG. 6, a reference block R is defined in the image plane P1 at time t. This reference block R is a rectangular area starting from the upper left pixel pr and having a range of m pixels on the u axis and n pixels on the v axis, and includes a total of m × n pixels. Then, the image plane P1 is divided into a plurality of reference blocks R, and a movement vector V is obtained for each reference block R. In the example of FIG. 6, a reference block R in which m = 8 and n = 8 is shown.

  FIG. 7 is a diagram for explaining how to obtain the movement vector V for a predetermined reference block R in the image plane P1. FIG. 7 corresponds to the image plane P1 at time t + Δt when Δt has elapsed from time t in FIG. A predetermined area in the image plane P1 is determined in advance as the search range S based on vehicle speed data from the sensor unit 14 or the like. The search range S is a rectangular area having M pixels on the u axis and N pixels on the v axis, and normally M and N are sufficiently larger than m and n.

  Then, a comparison block C having the same size as the reference block R is defined in a predetermined position with respect to the search range S. Since the search range S is sufficiently larger than the reference block R, it is necessary to move the position of the comparison block C little by little within the search range S. In FIG. 7, the comparison block C has a size of m × n like the reference block R, and shows the upper left pixel pc of the comparison block C corresponding to the upper left pixel pr of the reference block. First, the upper left pixel of the search range S is set as the pixel pc of the comparison block C, and after that, within the search range S, the position of the pixel pc is moved by one pixel in the u-axis direction or the v-axis direction. The calculation described later is performed for all comparison blocks C that can be defined in S.

  Next, a correlation value is obtained between the reference block R at time t in FIG. 6 and the comparison block C at time t + Δt in FIG. Here, in the image data which is a set of pixels, the density value is made to correspond to all m × n pixels, and therefore the difference between the density values of the corresponding pixels of the reference block R and the comparison block C is obtained. The correlation value can be calculated by taking the sum of m × n pixels with respect to the difference in density value for each pixel.

  Then, the correlation value between all the comparison blocks C in the search range S is obtained for the reference block R, and the comparison block C that gives the minimum correlation value is searched. For example, assume that the comparison block C shown in FIG. 7 gives the minimum correlation value.

  In this case, as shown in FIG. 7, a vector from the reference block R at time t indicated by the dotted line toward the comparison block C at time t + Δt on the image plane P1 can be determined as the movement vector V.

  Although only one movement vector V is shown in FIG. 7, the movement vector V is actually obtained for all reference blocks R that can be defined on the image plane P1. Thereby, the spatial distribution of the movement vector V is obtained. Then, based on the spatial distribution of the movement vector V, processing described later is performed.

  Next, using FIG. 8 to FIG. 9, a method for calculating a movement vector in an image captured from a running vehicle, a method for calculating the distance from the vehicle to the road edge, and the road width of the running road will be described.

  FIG. 8 is a diagram illustrating an example of an image captured from the CCD camera 19. For the sake of simplicity, FIG. 8 shows an image in the case where a road that goes straight with a certain width has walls with a certain height at both road ends, and the vehicle is traveling a predetermined distance from the center line on the left side. The road surface is in contact with the left wall surface at the boundary line BL, and is in contact with the right wall surface at the boundary line BR.

  As shown in FIG. 8, the center of the image is made to correspond to the origin (0, 0) in the pixel image coordinate system (u, v). That is, the vanishing point at which the image capturing direction is parallel to the road surface and the wall surface and the boundary lines BL, BR, etc. meet is the origin (0, 0). In addition, the entire image includes an upper left point (-Nw / 2, Nh / 2), an upper right point (Nw / 2, Nh / 2), and a lower left point (-Nw / 2, -Nh / 2). And a rectangular area having four pixels at the lower right point (Nw / 2, -Nh / 2) as vertices. When performing image processing to be described later, it is necessary to accurately calibrate the imaging direction and the mounting height H of the CCD camera 19 in advance. However, even when the origin (0, 0) and the vanishing point do not match, the image processing according to the present invention can be performed if the position of the vanishing point can be accurately identified by calibration.

  In FIG. 8, a reference horizontal line L represented by v = VR is set in the image. The reference horizontal line L needs to be set at a position crossing the road surface and the left and right wall surfaces, that is, a position intersecting the road edge, and VR has a negative value. For example, the reference horizontal line L may be set at the image position corresponding to the relative position on the road surface 10 m ahead of the vehicle. Then, in FIG. 8, a movement vector V on the reference horizontal line L from time t to time t + Δt is obtained. When the vehicle advances forward by a certain distance until Δt elapses, the imaging object on the reference horizontal line L approaches the vehicle by that amount, so that the v component of the movement vector V on the reference horizontal line L is a negative value. Become.

  The relative position with respect to the vehicle on which the reference horizontal line L is to be set is not limited to 10 m and can be set to an appropriate distance. However, if it gets too close to the vehicle, it will become inappropriate because it does not cross the left and right wall surfaces. If the vehicle is too far away from the vehicle, the distance becomes too large even if the vehicle crosses the road surface and the wall surface, and errors due to changes in the shape of the road curve or the like are likely to occur. Therefore, it is desirable to set the relative position as close as possible to the vehicle within a range that sufficiently overlaps the road surface and the wall surface.

Here, it is assumed that imaging is performed by the CCD camera 19 at a frame period T (seconds) in a vehicle traveling at a speed v0 (m / second). In this case, the moving distance d of the vehicle between one frame is
[Equation 4]
d = v0 · T (m)
It can be expressed as. On the other hand, as the vehicle moves, the movement vector V of the reference horizontal line L becomes a vector that goes radially from the origin (0, 0) that is the vanishing point. And the magnitude | size of the movement vector V changes so that it may become large, so that the distance of the imaging target object on the reference | standard horizontal line L and a vehicle is near.

  At this time, when the distribution of the movement vector V in the reference horizontal line L is taken, the road surface and the left and right wall surfaces change according to different patterns. Therefore, by analyzing the pattern change of the movement vector V, the positions of the intersections of the reference horizontal line L and the boundary lines BL and BR are obtained, and based on this, the distances DL and DR from the vehicle position to the left and right road edges are calculated. Further, the road width RW of the traveling road is calculated.

Here, the relationship between the pixel image coordinate system (u, v) and the actual spatial coordinate system (X, Y, Z) in the reference horizontal line L in FIG. 8 will be described with reference to FIG. Z representing the depth of the object to be imaged in FIG.
[Equation 5]
Z = X · f · Nw / (u · w)

  FIG. 9A is a diagram illustrating a change in the depth Z as viewed from the imaging position with respect to u on the horizontal axis of the image. In FIG. 9A, the inflection point PL corresponds to the intersection of the reference horizontal line L and the boundary line BL, and the inflection point PR corresponds to the intersection of the reference horizontal line L and the boundary line BR. As shown in FIG. 9 (a), Z becomes a constant value on the road surface, whereas Z becomes smaller toward the outside on the wall surface. That is, as can be seen from Equation 5, since the reference horizontal line L is on the horizontal plane on the road surface, u and X are proportional and Z is a constant value. On the other hand, since the reference horizontal line L is on the vertical surface on the left and right wall surfaces, X becomes a fixed value, and Z changes in inverse proportion to u.

Next, the height Y in the reference horizontal line L in FIG. 8 is expressed by the following equation based on Equations 2 and 5.
[Equation 6]
Y = Z · v · h / (f · Nh)

  FIG. 9B is a diagram illustrating a change in the height Y with respect to u on the horizontal axis of the image. In FIG. 9 (b), Y is a constant value on the road surface between the inflection point PL and the inflection point PR, as in FIG. 9 (a). On the other hand, on the wall surface, Y increases as it goes outward. In Equation 6, since u is a negative value, the change is opposite to that in the case of FIG.

Next, consider a case where the vehicle has advanced by ΔZ (m) from the state of FIG. At this time, if the vertical position of the reference horizontal line L in FIG. 8 moves by Δv (pixels), the relationship between Δv and ΔZ is expressed by the following equation based on Equation 6.
[Equation 7]
Δv = Y · f · Nh / ((Z−ΔZ) · h) −Y · f · Nh / (Z · h)

  FIG. 9C is a diagram illustrating a change in Δv when the vehicle travels ΔZ (m) with respect to u on the horizontal axis of the image. In FIG. 9C, as in FIGS. 9A and 9B, Δv has a constant value on the road surface between the inflection point PL and the inflection point PR. On the other hand, on the wall surface, Δv becomes smaller toward the outside. Therefore, the calculation described later is performed based on the value of Δv shown in Equation 7.

  In the above example, the case where the wall surface is present on the entire road edge has been described. However, even in the case of a partial wall surface such as a guard rail, the movement vector V changes between the road surface and other portions. In this case, the distances DL and DR and the road width RW can be calculated.

  FIG. 10 is a flowchart showing a method for calculating the distances DL, DR and the like from the vehicle to the wall surfaces of the left and right road edges by evaluating the movement vector V based on Equation 7. In performing the following processing, it is assumed that the imaging direction and the mounting height H of the CCD camera 19 are calibrated in advance.

  In the process of FIG. 10, in step S1, the movement vector V in the reference horizontal line L is calculated as described above. That is, the movement vector V in the reference horizontal line L during the frame period T is calculated by comparing two consecutive frame image data images.

  In step S2, the v component of the movement vector V is evaluated, and the left inflection point PL and the right inflection point PR on the reference horizontal line L are extracted based on Equation 7. At this time, it is assumed that the u-direction position ul of the inflection point PL and the u-direction position ur of the inflection point PR are obtained.

In step S3, the u-direction positions ul and ur of the extracted inflection points PL and PR are converted into the X-direction positions of the spatial coordinates. Specifically, the following equation based on Equation 2 may be calculated.
[Equation 8]
X = u · Z · w / (Nw · f)
However, Z in Equation 8 is Z = −H · f · Nh / (v · h). X shown in Equation 8 is calculated for the left and right inflection points PL and PR.

In step S4, the distance DL to the left wall surface, the distance DR to the right wall surface, and the road width RW are obtained. That is, based on the calculation result in step S4, the X direction position of the left inflection point PL corresponds to the distance DL, and the X direction position of the right inflection point PR corresponds to the distance DR. By adding these, the road width RW is
[Equation 9]
RW = DL + DR
It can be asked. The vehicle position can be known based on the relative relationship between the distances DL and DR.

In step S5, assuming that the camera is installed at the center in the horizontal direction of the vehicle, a margin ML for the left wall surface and a margin MR for the right wall surface are obtained by the following equations.
[Equation 10]
ML = DL-W / 2
MR = DR-W / 2
However, W represents the width of the vehicle. The left and right margins ML and MR shown in Equation 10 serve as criteria for giving a warning to the driver as will be described later.

  The processes in steps S1 to S5 described above are performed by evaluating the v component of the movement vector V. However, the characteristics of the u component of the movement vector V change similarly at the inflection points PL and PR. You may make it evaluate the above and perform the above-mentioned processing.

  In addition, if the road width data corresponding to the calculated road width RW is associated with the map data and stored in the road width data storage unit 13 in an updatable manner, it can be used for various processes described later.

  Further, as specific examples of each numerical value in the above image processing, the focal length f = 4 (mm), the number of pixels in the horizontal direction Nw = 320 (pixels), the lateral width w of the light receiving region = 4 (mm), the frame period T = 1/30 (second).

  Next, specific image processing in the navigation device according to the present embodiment will be described with reference to FIGS. 11 and 12. FIG. 11 is a flowchart illustrating a process for preventing a traveling vehicle from approaching the road edge. FIG. 12 illustrates a vehicle traveling on a multi-lane road traveling in an appropriate lane with respect to the set route. It is a flowchart which shows the process for making it do. 11 and 12 are processes mainly performed according to the control of the control unit 11.

  When the process shown in FIG. 11 is started, the distances DL and DR to the wall surfaces on both sides are obtained by the process of the image processing unit 18 as described above, and based on this, the road width RW is calculated according to the equation (9). 10, margins ML and MR for the left and right wall surfaces are calculated (step S11). The road width data corresponding to the road width RW calculated in step S11 is written in the road width data storage unit 13 in association with the map data (step S12). At this time, road width data may be written at a predetermined point on the road set in advance.

  Next, based on the margins ML and MR obtained in step S11, it is determined whether or not the traveling vehicle is too close to the road edge (step S13). That is, a predetermined value serving as a reference is set, and when the margin ML or MR is smaller than the predetermined value, it may be determined that the vehicle is too close to the road edge.

  If the determination result in step S13 is “YES”, the driver is warned that the vehicle is too close to the road edge (step S14). For example, a predetermined message indicating that the vehicle is too close to the road edge may be output as sound from the speaker 17. Alternatively, a similar message or display symbol may be displayed on the display 16.

  When the determination result of step S13 is “NO”, or when step S14 is finished, it is determined whether or not the vehicle is traveling (step S15). If the vehicle is not running (step S15; NO), the processing is terminated because there is no need to perform image processing. On the other hand, when the vehicle is still running (step S15; YES), the processes of steps S11 to S15 are repeated.

  In the navigation device, the accurate road width RW can always be obtained in real time by performing the processes in steps S11 to S15. And when the distances DL and DR to the left and right wall surfaces of the road are monitored, when the running vehicle is too close to the road edge, that is, when the vehicle is too close to the road edge and has no room, Since the driver is warned, the driver's driving can be assisted to ensure safety during traveling. In addition, since the road width data is appropriately written in the road width data storage unit 13, the road width data can be effectively used when traveling on the same road later.

  Next, the process shown in FIG. 12 is performed in a situation where a desired destination is set in the navigation device and a route is determined. First, as in step S11, distances DL and DR to the wall surfaces on both sides are obtained, and a road width RW is calculated (step S21).

  And the lane information with respect to the running lane contained in the map data of the map data storage part 12 is read (step S22). Since the lane information is information indicating how many lanes the road is traveling on, it is determined based on the lane information whether or not the road being traveled is a plurality of lanes (step S23).

  When the determination result of step S23 is “YES”, it is determined which lane is currently traveling among the plurality of lanes based on the determination result of step S21 (step S24). This determination can be made based on the ratio between the distance DL to the wall surfaces on both sides of the vehicle obtained in step S11 and DR.

  Subsequently, it is determined whether or not the lane determined in step S24 is an appropriate lane corresponding to the route reaching the destination (step S25). In other words, for example, when driving on a three-lane road, the right lane is generally appropriate when making a right turn ahead, and the left lane is appropriate when making a left turn ahead. When going straight, it may be any lane. Or you may set the information of the appropriate lane at the time of the left turn and the right turn beforehand.

  If the result of determination in step S25 is that the vehicle is traveling in an appropriate lane (step S25; YES), processing is terminated. On the other hand, if the vehicle is traveling in an inappropriate lane (step S25; NO), the driver is instructed to change lanes (step S26). For example, when driving in the left lane in front of the right turn point, a message prompting the driver to move to the right of the second lane may be output as a voice from the speaker 17 or displayed on the display 16. .

  In the navigation device, by performing the processes in steps S21 to S26 described above, it is possible to determine in real time the lane in which the vehicle is located on a road with a plurality of lanes. And only when it is not in the proper lane to make a right turn or left turn, etc., the driver is instructed to change lanes, so the guidance is given only when necessary and the vehicle is already driving in the proper lane In this case, unnecessary guidance can be avoided. Therefore, it is possible to realize navigation that is highly convenient and more comfortable for the driver.

  In the example of FIG. 12, guidance for lane change is instructed only when the vehicle is traveling in an inappropriate lane, that is, guidance for lane change is not instructed when traveling in an appropriate lane. Although the case has been described, the present invention is not limited to this, and some guidance may be performed when the vehicle is traveling appropriately. For example, while traveling in an appropriate lane, guidance such as “Please keep the lane as it is” may be output by voice or the like. As a result, it is possible to give a driver a sense of security by grasping that the driver is driving in an appropriate lane.

  Here, a calibration method of the CCD camera 19 in the present embodiment will be described. As described above, the CCD camera 19 needs to be calibrated in order to perform image processing with good accuracy. In particular, if the imaging direction of the CCD camera 19 is shifted, the road width RW or the like cannot be obtained accurately. Therefore, when the CCD camera 19 is mounted, the imaging direction and the mounting height H are measured, and calibration is performed by setting these values. Alternatively, not only when the camera is mounted, but also using the CCD camera 19 for imaging, calibration may be performed as needed based on the image. In this case, it is necessary to statistically obtain each parameter to be calibrated.

  Further, the movement vector V in the image captured by the CCD camera 19 can be used for calibration of the imaging direction. That is, when the movement vector V is obtained over a wide range of the image, the direction is toward the vanishing point. The imaging direction can be calibrated by specifying the vanishing point and determining the corresponding pixel position by averaging in consideration of the error of the movement vector V and the like.

  As described above, the present invention has been described based on the embodiments, but the present invention is not limited to the above-described embodiments, and various improvements and modifications can be easily made without departing from the spirit of the present invention. It can be guessed.

  As described above, according to the navigation device according to the above-described embodiment, the front of the vehicle is imaged, and the movement vector indicating the movement from the time t to t + Δt of the object in the captured image is displayed while the vehicle is traveling. 19 is determined by comparing the images obtained, and based on the detected movement vector, the margin to the road edge of the running road is determined, and the driver is notified based on the determination result ( For example, if the determined margin is smaller than a predetermined reference value, the notification means is notified that the vehicle has approached the road edge), so the driver is given warnings and guidance to ensure safe driving. It can be carried out. Furthermore, since the imaging direction of the CCD camera 19 is calibrated (for example, the imaging direction is calibrated by determining a vanishing point) based on the detection result of the movement vector, the variation in the mounting state of the CCD camera 19 is taken into consideration. Since image processing is performed and a movement vector may be used for the configuration, a simple calibration method can be provided.

  The road edge of the running road detects the spatial distribution of the movement vector on the reference horizontal line provided in the image obtained from the CCD camera 19, and from the amount of change of each movement vector constituting the distribution. An inflection point corresponding to the left and right road edges on the reference horizontal line is specified, and obtained based on the inflection point. In other words, the movement vector on the reference horizontal line set in the image has different characteristics in the pixel parts corresponding to the road surface and the road edge structure, respectively, so that the road edge is determined by obtaining the inflection point of the movement vector. And the distance to the road edge can be calculated.

  Further, according to the navigation device, the road width of the running road is calculated based on the detected movement vector, and the road width data corresponding to the calculated road width is stored in association with the map data. Therefore, since the road width data of the road on which the vehicle has traveled is stored and can be used when the vehicle again passes through the same road, the function of the navigation device can be supported.

  Further, according to the navigation device, the vehicle travels based on the detected movement vector, information on the number of lanes included in the map data, the calculated road width to the road edge, and the route to the destination. Since an appropriate (appropriate) lane to be announced is announced, the driver can be alerted to drive the lane corresponding to the optimal route, and a comfortable navigation function can be realized.

  As described above, with a simple configuration, it becomes possible to improve the safety and convenience of the vehicle, such as ensuring the safety of the traveling vehicle and acquiring the road width data. It is possible to support the navigation device and improve the function by notifying the driver at the time, notifying the lane change based on the lane determination, storing the road width data, and the like.

It is a figure which shows schematic structure of the navigation apparatus which concerns on this embodiment. It is a figure which shows the structure of the image process part of a navigation apparatus. It is a figure which shows the installation state to the vehicle of a CCD camera, (a) is the installation state seen from the vehicle upper direction, (b) is a figure which shows the installation state of a vehicle side. It is the figure which represented the image imaged with the CCD camera with the model of the pinhole camera. It is a figure showing the relationship between a spatial coordinate system and the pixel coordinate system of image data. It is a figure explaining the reference block defined in an image plane when calculating | requiring a movement vector. It is a figure explaining the method of calculating | requiring a movement vector by defining a comparison block in a search range with respect to the predetermined | prescribed reference block in an image plane. It is a figure which shows an example of the image imaged from the CCD camera of this invention in the image processing which concerns on this embodiment. It is a figure explaining the relationship between the pixel image coordinate system and the space coordinate system in the reference | standard horizontal line in an image. It is a flowchart which shows the method of calculating the distance etc. from a vehicle to the wall surface of a right-and-left road edge based on a movement vector. 5 is a flowchart showing a process for preventing a traveling vehicle from approaching a road edge in response to road width detection in the navigation device according to the present embodiment. In the navigation device according to the present embodiment, it is a flowchart showing a process for a vehicle traveling on a road having a plurality of lanes to travel a proper lane with respect to a set route in response to road width detection.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Control part 12 ... Map data storage part 13 ... Road width data storage part 14 ... Sensor part 15 ... Current position detection part 16 ... Display 17 ... Speaker 18 ... Image processing part 19 ... CCD camera 21 ... A / D converter 22 ... 1st 1 image memory 23 ... second image memory 24 ... movement vector detection unit 25 ... movement vector processing unit V ... movement vector L ... reference horizontal line PL, PR ... inflection point

Claims (5)

Imaging means for imaging the front of the vehicle and outputting an image;
A movement vector detecting means for obtaining a movement vector indicating the movement of the object in the image from time t to t + Δt by comparing images obtained by the imaging means during traveling of the vehicle;
Based on the detected movement vector, margin determining means for determining a margin to the road edge of the traveling road,
Control means for controlling the notification means based on the determination result of the margin determination means;
With
A navigation apparatus comprising calibration means for calibrating an imaging direction of the imaging means based on a detection result of the movement vector detection means.   Road width calculating means for calculating a road width of the road that is running based on the detected movement vector; and road width data storage means for storing road width data corresponding to the calculated road width in association with map data. The navigation device according to claim 1, comprising:   Lane notification means for notifying an appropriate lane to travel based on the detected movement vector, information on the number of lanes included in the map data, the calculated road width, and a route to the destination The navigation device according to claim 2, further comprising:   The navigation apparatus according to claim 1, wherein the movement vector detection means sets a search range of the image of t + Δt to be compared based on vehicle speed data. The movement vector detection means detects a spatial distribution of the movement vector on a reference horizontal line provided in an image obtained from the imaging means,
The margin determination means identifies an inflection point corresponding to the left and right road edges on the reference horizontal line from the amount of change of each movement vector constituting the distribution, and based on the inflection point, The navigation device according to claim 1, wherein a margin to a road edge of the road is determined.

Images