JP6278790B2 - Vehicle position detection device, vehicle position detection method, vehicle position detection computer program, and vehicle position detection system - Google Patents

Description

  The present invention relates to a vehicle position detection device, a vehicle position detection method, a vehicle position detection computer program, and a vehicle position detection system that detect the position of the vehicle using an image and a map captured by a camera mounted on the vehicle. .

  2. Description of the Related Art Conventionally, for navigation or driving assistance, a position of the host vehicle while traveling is detected using a Global Positioning System (GPS) signal. However, when there is a structure that blocks the positioning signal from the GPS positioning satellite, such as a high-rise building, near the vehicle, it becomes difficult to receive the positioning signal in the vehicle, and the position of the host vehicle cannot be detected accurately. There was a thing. Therefore, a method for improving the accuracy of detecting the position of the host vehicle by associating an image of the surroundings of the vehicle, which is captured by the in-vehicle camera, with information such as a map prepared in advance has been proposed (for example, a patent). References 1-3 and Non-Patent Document 1).

  For example, the position positioning device described in Patent Document 1 specifies the current position based on a road marking in an image captured by an imaging unit that captures the front of the vehicle.

  In addition, the moving body position measuring device described in Patent Document 2 includes feature information extracted from an image captured by an imaging unit mounted on a moving body in the vicinity of a reference point on the map, and the vicinity of the reference point on the map. The current position of the moving object is estimated in accordance with the positions of a plurality of features near the reference point estimated by matching with the feature information extracted from the image.

  Moreover, the method described in Patent Document 3 determines the position of the vehicle with respect to the optical index identified in the image by applying template matching to the image data in the traveling direction of the vehicle.

  Furthermore, the method described in Non-Patent Document 1 detects the position of the host vehicle by matching a map including line segments representing curbs and road markings and feature points extracted from an image obtained by a stereo camera.

JP 2007-108043 A JP 2012-127896 A JP 2005-136946 A

Schreiber et al., "LaneLoc: Lane Marking based Localization using Highly Accurate Maps", Intelligent Vehicles Symposium (IV), 2013 IEEE, IEEE, 2013

  However, the device described in Patent Document 1 may not be able to accurately identify the position of the host vehicle when it is difficult to detect feature points of road markings due to environmental influences during shooting. In particular, the device described in Patent Document 1 extracts a road marking end point or the like as a feature point, and uses the feature point to determine the current position of the vehicle. Even if it is slightly drowned, the position of the extracted feature point is shifted, and the detection accuracy of the position of the host vehicle is lowered.

  The device described in Patent Document 2 uses an image photographed in the vicinity of a reference point on a map for specifying the position of the host vehicle, but rebuilding a building near the reference point or an environment at the time of photographing (for example, , Weather, shooting time, etc.), even if the image was taken at the same location, it appears in the scene taken in the image taken at the time of pre-registration and in the image taken when the vehicle position was detected The scenery can vary greatly. In such a case, there is a high possibility that matching between images will fail, and therefore the position of the host vehicle may not be accurately identified.

  Furthermore, the method described in Patent Document 3 uses template matching. In template matching, since it is necessary to repeatedly perform matching calculation while changing the position between the image to be compared and the template, a large amount of calculation is required for detecting the position of the host vehicle. Further, in the method described in Non-Patent Document 1, different algorithms are used for curb detection and road marking detection, and a stereo image is used for these detections. Therefore, in these methods, hardware resources necessary for detecting the position of the own vehicle increase, or time is required for detecting the position of the own vehicle, so that the update interval of the position of the own vehicle becomes long. .

  Therefore, an object of the present invention is to provide a vehicle position detection device, a vehicle position detection method, a vehicle position detection computer program, and a vehicle position detection system that can accurately detect the position of the host vehicle with a low processing load.

According to the first aspect of the present invention, a vehicle position detecting device is provided as one aspect of the present invention. The vehicle position detection device includes a map including positions of a plurality of map line segments representing the outline of an object on a road, and a map representing a distribution of map line segments for each locality and for each direction of the map line segment. A storage unit (41) for storing line segment distribution information and an imaging unit (2) mounted on the vehicle (10) are shown in the image obtained by photographing the surroundings of the vehicle (10). A line feature extraction unit (432, 532) that extracts a plurality of line segments representing the contour of an object on the road, and an image line segment that represents the distribution of line segments for each local area and for each line segment direction on the image The degree of coincidence between the map line segment distribution information and the image line segment distribution information at the virtual shooting point for each of a plurality of virtual shooting points set on the map and the line distribution information calculation unit (433, 534) for obtaining distribution information A virtual shooting point where the evaluation value is calculated and the evaluation value is maximized And a comparing section that the position of the vehicle.
Since the vehicle position detection device according to the present invention has the above-described configuration, the position of the host vehicle can be accurately detected with a low processing load.

According to the second aspect of the present invention, the storage unit (41) generates a virtual image obtained by virtually shooting a map with the imaging unit (2) from the virtual shooting point for each of the plurality of virtual shooting points. For each of the plurality of divided grids, it is preferable to store, as map line segment distribution information, the distribution for each direction of the map line segment represented on the virtual image included in the grid. In this case, the line distribution information calculation unit (433) divides the image into a plurality of lattices, and for each of the plurality of lattices, the distribution for each direction of the line segment represented on the image included in the lattice is imaged. It is preferable to use line segment distribution information.
By having such a configuration, the vehicle position detection device can obtain, for each virtual shooting point, a distribution of map line segments obtained from a virtual image that looks the same as an image obtained by actually shooting the periphery of the vehicle by the imaging unit. Since the information is used for comparison with the image line segment distribution information, the map line segment distribution information and the image line segment distribution information can be appropriately compared. Therefore, the vehicle position detection device can accurately specify the virtual shooting point closest to the vehicle position among the virtual shooting points.

According to a third aspect of the present invention, the vehicle position detection device performs bird's-eye conversion on each of the plurality of line segments, and determines the position of each of the plurality of line segments in the vehicle coordinate system based on the vehicle (10). It is preferable to further include a bird's-eye conversion unit (533) for obtaining a bird's-eye image to be represented. In this case, the storage unit (41) stores, for each of the plurality of first grids obtained by dividing the map, the distribution for each direction of the map line segment represented on the map included in the first grid. The line distribution information calculation unit (534) divides the bird's-eye image into a plurality of second grids, and the lines represented on the bird's-eye image included in the second grid for each of the plurality of second grids. The distribution for each minute direction is used as the image line segment distribution information, and the comparison unit (536) assigns each of the plurality of virtual shooting points to each of the plurality of second grids in the vehicle coordinate system according to the virtual shooting points. The corresponding first grid is specified, and the distribution for each map line direction of the first grid corresponding to each of the plurality of second grids is used as the map line segment distribution information for the virtual shooting point. It is preferable.
By having such a configuration, the vehicle position detection device can display image line segment distribution information obtained from a bird's-eye view image that looks like a map of an image obtained by actually photographing the periphery of the vehicle with an imaging unit. Used for comparison with minute distribution information. In addition, since the vehicle position detection device identifies the first grid corresponding to each of the second grids according to the virtual shooting point, the map line to be compared with the image line segment distribution information for each virtual shooting point. Minute distribution information can be set appropriately. Therefore, the vehicle position detection device can accurately specify the virtual shooting point closest to the vehicle position among the virtual shooting points.

Furthermore, according to the description of claim 4, for each of the plurality of map line segments, the direction of the map line segment is set according to the direction of the luminance gradient in the direction crossing the map line segment, and the line feature extraction unit (432, 532), for each detected line segment, it is preferable to obtain the direction of the luminance gradient in the direction crossing the line segment and set the direction of the line segment according to the direction of the luminance gradient.
With such a configuration, the vehicle position detection device can also evaluate the difference in the position of the contour with respect to the object on the road in the comparison between the image line segment distribution information and the map line segment distribution information. Therefore, this vehicle position detection system can detect the position of the vehicle more accurately.

Moreover, according to the description of Claim 5, the vehicle position detection method is provided as another form of this invention. In this vehicle position detection method, the contour of the object on the road shown in the image is obtained from the image obtained by photographing the periphery of the vehicle (10) by the imaging unit (2) mounted on the vehicle (10). A step of extracting a plurality of line segments to be represented; a step of obtaining image line segment distribution information representing a distribution of line segments for each local and direction of the line segment on the image; and an outline of an object on the road For each of the plurality of virtual shooting points set on the map including the positions of the plurality of map line segments to be represented, the distribution of the map line segments for each local area and for each direction of the map line segment at the virtual shooting point is calculated. Calculating an evaluation value representing the degree of coincidence between the map line segment distribution information to be represented and the image line segment distribution information, and setting the virtual shooting point having the maximum evaluation value as the position of the vehicle (10).
Since the vehicle position detection method according to the present invention includes the above steps, the position of the host vehicle can be accurately detected with a low processing load.

According to the sixth aspect of the present invention, a computer program for detecting a vehicle position is provided as still another aspect of the present invention. This computer program is a plurality of images representing the contour of an object on a road shown in an image obtained by photographing the periphery of the vehicle (10) by the imaging unit (2) mounted on the vehicle (10). A step of extracting a line segment of the image, a step of obtaining image line segment distribution information representing the distribution of the line segment for each local and direction of the line segment on the image, and a plurality of contours representing the contour of the object on the road For each of a plurality of virtual shooting points set on the map including the respective positions of the map line segments, a map representing the distribution of the map line segments for each local area and for each direction of the map line segments at the virtual shooting points Calculating an evaluation value representing the degree of coincidence between the line segment distribution information and the image line segment distribution information, and setting the virtual photographing point having the maximum evaluation value as the position of the vehicle (10). Execute on the installed processor (43) To include an instruction.
The computer program for detecting the vehicle position according to the present invention can accurately detect the position of the host vehicle with a low processing load by having the above-described command.

According to a seventh aspect of the present invention, a vehicle position detection system is provided as still another aspect of the present invention. This vehicle position detection system is mounted on a vehicle (10), captures an image of the periphery of the vehicle (10) and generates an image, and a plurality of map line segments representing the outline of an object on the road A storage unit (41) for storing a map including each of the positions, map line segment distribution information representing the distribution of map line segments for each local area and for each direction of the map line segment, and an image from the imaging unit (2). The communication unit (42) for acquiring the control unit and the control unit (43). The control unit (43) extracts a plurality of line segments representing the contour of the object on the road shown in the image, and represents the distribution of line segments for each local area and for each line segment direction on the image. The line segment distribution information is obtained, and for each of a plurality of virtual shooting points set on the map, an evaluation value indicating the degree of coincidence between the map line segment distribution information and the image line segment distribution information at the virtual shooting point is calculated and evaluated. A control unit (43) that sets the position of the vehicle (10) at the virtual shooting point having the maximum value.
The vehicle position detection system according to the present invention can accurately detect the position of the host vehicle with a low processing load by having the above-described configuration.

  The reference numerals in parentheses attached to the above-described parts are examples that show the correspondence with specific means described in the embodiments described later.

1 is a schematic configuration diagram of a vehicle position detection system according to one embodiment of the present invention. It is a figure which shows the relationship between a world coordinate system, a vehicle coordinate system, and a camera coordinate system. It is a figure which shows an example of map information. (A) is a figure which shows an example of the relationship between the brightness | luminance gradient of the direction crossing the line segment showing an outline, and the direction set to the line segment, (b) is the direction of each line segment set about a white line It is a figure which shows an example. It is a figure which shows the example of a setting of a virtual imaging | photography point. It is a figure which shows the example of the virtual image obtained by image | photographing the line segment shown on the map from the virtual imaging | photography point. It is a figure which shows an example of the grating | lattice set to a virtual image. It is a figure which shows an example of the histogram of the length of the line segment for every direction of the line segment calculated for every lattice. It is a functional block diagram of the control part of the vehicle position detection system which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the extracted line segment. It is a figure which shows an example of the grating | lattice set to an image. It is an operation | movement flowchart of the vehicle position detection process by 1st Embodiment. It is a figure which shows an example of the grid | lattice set to a map. It is a functional block diagram of the control part of the vehicle position detection system which concerns on 2nd Embodiment. It is an example of the bird's-eye view image obtained by carrying out bird's-eye view conversion of the extracted line segment. It is a figure which shows an example of the grating | lattice set to a bird's-eye view image. It is an operation | movement flowchart of the vehicle position detection process by 2nd Embodiment.

Hereinafter, a vehicle position detection system according to a plurality of embodiments will be described with reference to the drawings.
This vehicle position detection system detects line segments corresponding to contours such as lane markings or road markings from an image acquired from a camera that captures the periphery of the vehicle. This vehicle position detection system has a distribution of line segments for each local area on the image and for each direction of line segments, and map information including line segments corresponding to contours such as lane markings or road markings. The position of the vehicle is detected by finding the most matching position.

  In particular, in the first embodiment, the vehicle position detection system divides an image obtained from a camera that captures the periphery of the vehicle in units of grids, and for each grid, a histogram of the length of line segments for each line segment direction. Create In addition, this vehicle position detection system also sets a virtual shooting point on a map, and similarly divides the virtual image obtained by virtually shooting the map from the virtual shooting point into units of grids. In addition, a histogram of the length of the line segment for each line segment direction is stored in advance. The vehicle position detection system compares the histogram for each grid obtained from the image with the histogram for each grid obtained from the virtual image, thereby identifying the virtual image closest to the image, thereby Detect position.

  FIG. 1 is a schematic configuration diagram of a vehicle position detection system according to a first embodiment. As shown in FIG. 1, the vehicle position detection system 1 is mounted on a vehicle 10 and includes an in-vehicle camera 2 and a controller 4. The in-vehicle camera 2 and the controller 4 are connected to each other by a control area network (hereinafter referred to as CAN) 3. In FIG. 1, for convenience of explanation, each component of the vehicle position detection system 1 and the shape, size, and arrangement of the vehicle 10 are different from actual ones.

  The in-vehicle camera 2 is an example of an imaging unit, and captures a front area of the vehicle and generates an image of the front area. For this purpose, the in-vehicle camera 2 is present in front of the vehicle 10 on the two-dimensional detector composed of an array of photoelectric conversion elements having sensitivity to visible light, such as CCD or C-MOS. An imaging optical system that forms an image of the ground or a structure is included. The in-vehicle camera 2 is disposed in the vehicle interior of the vehicle 10 so that, for example, the optical axis of the imaging optical system is substantially parallel to the ground and faces the front of the vehicle 10. The in-vehicle camera 2 captures images at regular time intervals (for example, 1/30 seconds), and generates a color image capturing the front area of the vehicle 10. The in-vehicle camera 2 may include a two-dimensional detector having sensitivity to near infrared light, and may generate a gray image corresponding to the illuminance of the near infrared light within the imaging range.

FIG. 2 is a diagram illustrating a relationship among the world coordinate system, the vehicle coordinate system, and the camera coordinate system. In this embodiment, for convenience, a world coordinate system (Xw, Yw, Zw) having an arbitrary position in the real space as an origin, a vehicle coordinate system (Xv, Yv, Zv) having the vehicle 10 as an origin, and the in-vehicle camera 2 Use the camera coordinate system (Xc, Yc, Zc) with the origin at. In this embodiment, the vehicle coordinate system (Xv, Yv, Zv) has a midpoint between the left and right rear wheels of the vehicle 10 and a point on the ground as the origin. The traveling direction of the vehicle is the Zv axis, the direction orthogonal to the Zv axis and parallel to the ground is the Xv axis, and the vertical direction is the Yv axis. In the world coordinate system (Xw, Yw, Zw), the Xw axis and the Zw axis are set in a plane parallel to the ground, and the Yw axis is set in the vertical direction. In addition, in the camera coordinate system (Xc, Yc, Zc), for the sake of simplicity of explanation, the imaging surface is positioned at a height above the origin of the vehicle coordinate system along the vertical direction and at a height at which the in-vehicle camera 2 is installed. Assuming that there is a center, the center of the imaging surface is the origin. Similarly to the vehicle coordinate system, the traveling direction of the vehicle is the Zc axis, the direction orthogonal to the Zc axis and parallel to the ground is the Xc axis, and the vertical direction is the Yc axis.
In addition, although the position where the vehicle-mounted camera 2 is actually attached may be shifted from above the midpoint between the left and right rear wheels of the vehicle 10, this shift may be corrected by a simple parallel movement.

  The vehicle position detection system 1 may include a rear camera that captures the rear region of the vehicle as an image capturing unit instead of or together with the camera that captures the front region of the vehicle 10.

  The in-vehicle camera 2 sequentially transmits the generated image to the controller 4. In addition, when the vehicle-mounted camera which image | photographs the front area of a vehicle and the vehicle-mounted camera which image | photographs the back area | region of a vehicle are attached, the controller 4 selects only the image from the vehicle-mounted camera which image | photographs the direction which the vehicle is advancing. May be acquired automatically. For this purpose, the controller 4 acquires a shift position signal representing the position of the shift lever from the electronic control unit (ECU) 11 of the vehicle 10 via the CAN 3. And the controller 4 acquires an image from the vehicle-mounted camera which image | photographs the front area | region of a vehicle, when the shift position signal becomes the drive position etc. which show that the vehicle 10 moves forward. On the other hand, when the shift position signal is a reverse position indicating that the vehicle 10 moves backward, the controller 4 acquires an image from an in-vehicle camera that captures a rear region of the vehicle.

The controller 4 is an example of a vehicle position detection device, and includes a storage unit 41, a communication unit 42, and a control unit 43. The storage unit 41 includes, for example, a semiconductor memory such as an electrically rewritable nonvolatile memory and a volatile memory. The storage unit 41 includes various programs for controlling the vehicle position detection system 1, map information, position information of the vehicle-mounted camera such as the height of the vehicle-mounted camera 2 from the ground and the optical axis direction, the focal length of the imaging optical system, and the like. Various parameters such as camera parameters such as an angle of view, temporary calculation results by the control unit 43, and the like are stored. In addition, the storage unit 41 is a parameter for correcting distortion due to the imaging optical system of the in-vehicle camera 2, for example, a correction amount of distortion aberration for each pixel (that is, pixel movement on the image for canceling distortion aberration). Quantity and direction of movement) may be stored.
The communication unit 42 includes a communication interface that communicates with various sensors such as the in-vehicle camera 2, the ECU 11, and a wheel speed sensor (not shown) through the CAN 3 and a control circuit thereof.

Hereinafter, map-related information used for vehicle position detection will be described.
FIG. 3 is a diagram illustrating an example of map information. The map information 300 is associated with latitude / longitude information indicating the position of the area represented by the map information. The map information 300 includes information related to road markings such as steps due to curbs, lane lines such as white lines and yellow lines, and pedestrian crossings and stop lines. Specifically, as indicated by line 301, step, each of the contours of the lane line and the road displayed is approximated by a line segment, the coordinates of each line segment of the starting point o ^r _s and the end point coordinate o ^r _e ( (Expressed in the world coordinate system) is included in the map information 300. Further, each line segment is associated with a label indicating the type of object represented by the line segment (for example, step, white line, yellow line, pedestrian crossing, etc.). In the following, a line segment included in the map information is referred to as a map line segment.

For each line segment representing an outline such as a division line, a direction corresponding to a luminance gradient in a direction crossing the line segment is set.
FIG. 4A is a diagram illustrating an example of the relationship between the luminance gradient in the direction crossing the line segment representing the contour and the direction set for the line segment. Arrows 401 to 408 shown in FIG. 4A indicate the direction of the line segment, and are set at 45 ° intervals. The roots of the arrows 401 to 408 correspond to the start point s, and the tips of the arrows 401 to 408 correspond to the end point e. The arrows 401 to 408 indicate the relative luminances of the left and right of the line segment when the end point e is viewed from the start point s of the line segment. In this example, when viewing the end point e from the start point s, the direction of the line segment, that is, the start point and end point of the line segment is set so that the luminance decreases from the left to the right of the line segment. For example, as indicated by an arrow 401, when the line segment is in the vertical direction (that is, the Zw axis direction) and the left side luminance is higher than the right side luminance, the end point e is higher than the start point s. The start point s and the end point e are set so that the coordinate value of the Zw axis becomes larger. Conversely, as indicated by an arrow 405, when the line segment is in the vertical direction (that is, the Zw axis direction) and the right side luminance is higher than the left side luminance, the end point e is higher than the start point s. The starting point s and the ending point e are set so that the coordinate value of the Zw axis becomes smaller in this case.

FIG. 4B is a diagram illustrating an example of the direction of each line segment set for the white line. The outline of the white line 410 is represented by four line segments 411 to 414. The direction of each line segment is represented by an arrow. The luminance of the white line 410 is higher than the surrounding luminance. For this reason, the line segment 411 representing the right contour of the white line 410 corresponds to the arrow 408 in FIG. 4A, so that the end point closer to the lower end of the Zw axis is the start point s and the end closer to the upper end of the Zw axis The end point is the end point e. Conversely, the line segment 413 representing the left outline of the white line 410 corresponds to the arrow 404 in FIG. 4A, so that the end point closer to the lower end of the Zw axis is the end point e, and the end closer to the upper end of the Zw axis The end point of is the starting point s. Similarly, since the arrow 412 corresponds to the arrow 406 in FIG. 4A, the right end point is the start point s and the left end point is the end point e. Also, since the arrow 414 corresponds to the arrow 402 in FIG. 4A, the left end point is the start point s, and the right end point is the end point e.
In this way, by referring to the direction of the line segment, it can be understood which side of the line line represents the contour, such as a division line.

  Note that the direction may not be set for a line segment whose magnitude of the luminance gradient in the direction orthogonal to the line segment is smaller than a predetermined value, such as a step or an outline of a manhole. In this case, for example, a flag indicating that the direction is not set for the line segment is associated. Any of both end points of the line segment may be the starting point.

  Next, a description will be given of a descriptor that is an example of map line segment distribution information that represents the distribution of line segments for each local area and for each direction of line segments, created from map information. As described above, in the present embodiment, for each virtual shooting point set on the map, the distribution of the line segment for each local area and for each line segment direction is represented from the virtual image corresponding to the virtual shooting point. A descriptor is created.

  FIG. 5 is a diagram illustrating an example of setting virtual shooting points. In the map 500, a plurality of virtual shooting points 501 are set in the world coordinate system. The interval between adjacent virtual shooting points is set to an interval (for example, 1 m to 5 m) according to the required detection accuracy of the position of the vehicle, for example. In addition, a plurality of shooting directions may be set for each virtual shooting point 501 as indicated by an arrow 502. This shooting direction corresponds to the azimuth of the virtual vehicle 10. Note that the shooting direction may be set only for the assumed traveling direction of the vehicle 10. For example, on a two-lane road, for a virtual shooting point set on the left lane, the shooting direction is set only for the direction in which the value of the Zw axis is positive, while it is set on the right lane. For the virtual shooting point, the shooting direction may be set only for the direction in which the value of the Zw axis is negative.

  When a virtual shooting point and shooting direction are set, a virtual image is generated by virtually shooting a map line segment representing an object on the road described on the map toward the shooting direction set from the virtual shooting point. Is done. For example, the virtual image is preferably the same size as the image generated by the in-vehicle camera 2.

ここで、回転行列R^vg _kは、k番目の仮想撮影点（ただし、個々の仮想撮影点は、世界座標系で表される位置だけでなく、撮影方向も含めて設定される）に対応する回転行列であり、θ_kは、その撮影方向である。また並進行列t^vgは、世界座標系でのk番目の仮想撮影点の位置を表す。また、(c_u,c_v)、(f_u,f_v)は内部パラメータであり、具体的に、(c_u,c_v)は、車載カメラ２の光軸に対応する画像上の点、すなわち、画像中心の座標である。また(f_u,f_v)は、水平方向と垂直方向の1画素あたりのサイズの違いを考慮した、車載カメラ２の焦点距離に対応する値である。またy_cは、路面から車載カメラ２までの高さを表す。そして(β^r _skx,β^r _skz)、(β^r _ekx,β^r _ekz)は、それぞれ、車両１０がk番目の仮想撮影点の位置にいて撮影方向がθ_kである場合の地図線分(o^r _s, o^r _e)を車両座標系で表したときの始点及び終点の座標である。なお、この例では、仮想撮影を行うカメラの光軸が地図上の路面と平行であると仮定して、（１）式の各パラメータが設定されている。 The starting point of the rth (r = 1,2, ..., R) map line segment on the map is o ^r _s = (o ^r _sx , o ^r _sz ) ^t and the end point is o ^r _e = (o ^r _ex , o ^r _ez ) ^t , the starting point l ^r _s = (l ^r _su , l ^r _sv ) ^t and end point l _re = (l ^r _eu , l ^r _ev ) ^t on the virtual image Is calculated by the following equation.

Here, the rotation matrix R ^vg _k corresponds to the k-th virtual shooting point (however, each virtual shooting point is set including not only the position expressed in the world coordinate system but also the shooting direction). It is a rotation matrix, and θ _k is its photographing direction. The ^{parallel progression} row t ^vg represents the position of the kth virtual shooting point in the world coordinate system. Further, (c _u , c _v ) and (f _u , f _v ) are internal parameters. Specifically, (c _u , c _v ) is a point on the image corresponding to the optical axis of the in-vehicle camera 2, That is, the coordinates of the center of the image. Further, (f _u , f _v ) is a value corresponding to the focal length of the in-vehicle camera 2 in consideration of the difference in size per pixel in the horizontal direction and the vertical direction. Y _c represents the height from the road surface to the in-vehicle camera 2. (Β ^r _skx , β ^r _skz ) and (β ^r _ekx , β ^r _ekz ) are the map line segments when the vehicle 10 is at the position of the k-th virtual shooting point and the shooting direction is θ _k ( o ^r _s , o ^r _e ) are the coordinates of the start point and end point when expressed in the vehicle coordinate system. In this example, the parameters of equation (1) are set on the assumption that the optical axis of the camera that performs virtual shooting is parallel to the road surface on the map.

  Note that, for map line segments in which both end points are separated from the virtual shooting point by a predetermined distance or more, the corresponding line segments on the virtual image need not be obtained. This is because such a map line segment appears only small on the virtual image, and if used for vehicle position detection, the error increases and the accuracy of position detection may decrease. The predetermined distance is set to 30 m, for example.

  FIG. 6 is a diagram illustrating an example of a virtual image obtained by virtually photographing a map line segment from a virtual photographing point. A virtual image 602 is an image obtained by virtually photographing from a virtual photographing point 601 on the map 600.

In order to examine the distribution of line segments for each local area and for each line segment direction, each virtual image is divided into a grid pattern.
FIG. 7 is a diagram illustrating an example of a lattice set in the virtual image illustrated in FIG. As shown in FIG. 7, the virtual image 700 is divided so that each grid 701 can include a plurality of map line segments. The size of one grid is set to 30 pixels × 30 pixels, for example. For convenience, numbers j = 1, 2,..., J are set for each grid in the raster scan order.

  A histogram representing the distribution of map line segments for each grid and for each line segment direction is generated. In this histogram, individual bins are set at equiangular intervals, for example, π / 8 (= 45 °) intervals. Furthermore, in order to cope with the case where no map line segment is included in the grid, a bin corresponding to no line segment is added.

Further, in this example, the frequency of the line segment for each bin is represented by the total length of the line segments having the direction corresponding to the bin and included in the lattice. For example, a bin corresponding to a counterclockwise angle between the reference direction and the map line segment, with the reference direction (for example, the horizontal direction) set to 0 ° and the starting point of the map line segment as the rotation center, This is the bin to which the length of the map line should be added. By defining the angle in this way, for map line segments that represent the contours of objects that are brighter than the surroundings, such as lane markings or road markings, the direction adjacent to the object is distinguished, and the distribution of map line segments Is required.
For map line segments with no orientation, such as line segments representing steps, a bin corresponding to the angle formed by the map line segment and the reference direction is added to or subtracted from that angle by 180 °. It is only necessary to add half the length of the map line to the bin corresponding to the angle.

  When the lengths of all the map line segments in the grid are added, the histogram is normalized so that the total length of the line segments in each bin is 1.

FIG. 8 is a diagram illustrating an example of a normalized histogram of line segment lengths for each line segment direction calculated for each lattice.
In FIG. 8, a normalized histogram 811 is a histogram obtained for the grid 801 on the virtual image 800, and a normalized histogram 812 is a histogram obtained for the grid 802 on the virtual image 800. Since the grid 801 includes two map line segments having different directions, the sum of the values of the two bins 821 and 822 corresponding to the direction of these line segments is 1 in the normalized histogram 811. Other bin values are 0. Further, since the grid 802 does not include a map line segment, the bin value corresponding to “none” is 1 in the normalized histogram 812, and the values of other bins are 0.

Here, a normalized histogram for the j-th lattice on the virtual image corresponding to the virtual shooting point y _k = (x _k , z _k , θ _k ) ^t is η _{k, j} . At this time, the matrix a _k = (η _{k, 1} , η _{k, 2} , ..., η _{k, J} ) ^{t in} which the normalized histograms are arranged in the order of the grid numbers over the entire virtual image becomes the virtual shooting point y _k . The corresponding map line segment distribution for each local area and for each line segment direction is represented. Hereinafter, for convenience, a _k is referred to as a descriptor.

The storage unit 41 stores the descriptor a _k (k = 1, 2,..., K) for each virtual shooting point y _k (k = 1, 2,..., K) and the world of the virtual shooting point. The coordinates (x _k , z _k ) in the coordinate system and the shooting direction θ _{k are} stored.

The control unit 43 includes one or a plurality of processors (not shown) and their peripheral circuits. And the control part 43 controls the vehicle position detection system 1 whole.
FIG. 9 shows a functional block diagram of the control unit 43 according to the first embodiment. As illustrated in FIG. 9, the control unit 43 includes an initial value setting unit 431, a line feature extraction unit 432, a line distribution information calculation unit 433, a search range setting unit 434, and a comparison unit 435. Each of these units included in the control unit 43 is implemented as a functional module realized by a computer program executed on a processor included in the control unit 43, for example.

The initial value setting unit 431 sets the initial value of the position of the host vehicle when the detection of the position of the vehicle 10, that is, the detection of the position of the host vehicle is started according to the present embodiment. For example, the initial value setting unit 431 receives the GPS positioning signal received at a different time immediately before starting the detection of the position of the host vehicle from the position obtained from the positioning signal when the GPS positioning signal was last received. The azimuth of the host vehicle calculated from the change in the position of the host vehicle obtained from the above is received from, for example, a navigation system. Then, the obtained position (x _t , z _t ) and direction θ _t are set to the initial value μ _t = (x _t , z _t , θ _t ) ^t of the host vehicle.
Note that the initial value set by the initial value setting unit 431 may be a value obtained with detection accuracy lower than the detection accuracy of the position of the host vehicle according to the present embodiment.

  The initial value of the position of the host vehicle set by the initial value setting unit 431 is stored in the storage unit 41.

The line feature extraction unit 432 extracts a line segment from an image generated at a time closest to the current time t among images generated by the in-vehicle camera 2. At that time, the line feature extraction unit 432 extracts a line segment by applying a detector called a line segment detector (LSD) to the image. Regarding LSD, for example, Rafael Grompone von Gioi et al., `` LSD: A Fast Line Segment Detector with a False Detection Control '', Pattern Analysis and Machine Intelligence, IEEE Transactions, vo. 32, no. 4, pp. 722- See 732, April 2010.
Note that the line feature extraction unit 432 may extract any line segment by applying any of various other methods for detecting a line segment from the image to the image acquired from the in-vehicle camera 2. Good.
Further, the line feature extraction unit 432 may correct image distortion due to distortion or the like on the image before extracting the line segment from the image acquired from the in-vehicle camera 2.

  FIG. 10 is a diagram illustrating an example of a line segment extracted from an image. In the image 1000, a line segment 1001 is a line segment extracted by the line feature extraction unit 432. As shown in FIG. 10, the line segment 1001 is extracted along the contour of a section line, the contour of a road marking, a step, or the like.

  In the present embodiment, the vehicle position detection system 1 uses information on the road such as road markings, which is relatively rarely changed, and is affected by the rebuilding of a building beside the road. Information is not used. Therefore, the line feature extraction unit 432 deletes a line segment in which the vertical coordinate values of both end points of the line segment are less than a predetermined threshold Th among the extracted line segments. However, on the image, it is assumed that the upper left pixel is the origin, and the vertical coordinate value increases as it goes downward. That is, the line segment in which the vertical coordinate values of the both end points of the line segment are located above the predetermined threshold Th is deleted. In this embodiment, since the in-vehicle camera 2 is installed so that the optical axis of the in-vehicle camera 2 is parallel to the road surface, the horizontal line including the image center corresponds to the horizon. Therefore, the predetermined threshold Th can be a value obtained by subtracting a predetermined offset (for example, 5 to 10) from the height of the image center.

  As described above, lane markings, contour lines of road markings, and the like are extracted as line segments. Therefore, the luminance may change along the direction crossing the extracted line segment. Therefore, for example, the line feature extraction unit 432 can identify which side of the bright line (white line, yellow line, road marking) is adjacent to the extracted line segment. The direction of the extracted line segment is set according to arrows 401 to 408 indicated by (). That is, the line feature extraction unit 432 sets one of the end points of the extracted line segment as the start point and the other as the end point. At that time, for example, the line feature extraction unit 432 generates a luminance gradient image from the image, and calculates a median value in the direction of the luminance gradient of each pixel on the line segment. Then, the line feature extraction unit 432 may specify the higher one of the left side and the right side of the line segment based on the median value. In addition, when the average value of the magnitude of the luminance gradient for each pixel on the line segment is smaller than a predetermined threshold, the line feature extraction unit 432 may use any one of both ends of the line segment as a starting point. . The line feature extraction unit 432 associates a flag indicating that the direction is not set with the line segment.

  The line feature extraction unit 432 stores the coordinates of the start point and end point of the extracted line segment in the storage unit 41.

Similar to the descriptor calculated for the virtual image, the line distribution information calculation unit 433 divides the image into grid units, and calculates a normalized histogram of the length of the line segment for each line segment direction for each grid. . Also in this case, the normalized histogram includes individual bins set at equiangular intervals (for example, π / 8 intervals), and bins without line segments so as to correspond to grids that do not include line segments. Provided. Also, the line distribution information calculation unit 433 calculates the total length of the line segments for each bin for each lattice in the same manner as when calculating the descriptor for the virtual image, and the total length for each bin is 1 Normalize the histogram so that
Here, a normalized histogram for the j-th lattice on the image is denoted by κ _j . The line distribution information calculation unit 433 creates a descriptor λ = (κ ₁ , κ ₂ ,..., Κ _J ) ^{t in} which normalized histograms are arranged in the order of lattice numbers over the entire image. This descriptor λ is an example of image line segment distribution information representing the distribution of line segments such as the contour of an object on the road surface, which is reflected in the image for each local area and for each direction of the line segment.

  FIG. 11 is a diagram illustrating an example of a grid set in an image. In the image 1100, the same number of lattices 1101 as the number of lattices set for each of the horizontal direction and the vertical direction of the virtual image is set in each of the horizontal direction and the vertical direction. In addition, a number is assigned to each grid in the same order as the grid set for the virtual image, for example, in the raster scan order. In other words, the j-th grid set for the image 1100 and the same j-th grid set for the virtual image correspond to the same position on the image, respectively. Therefore, the vehicle position detection system 1 compares the normalized histograms for the lattices with the same number to obtain the distribution of the line segment for each local area and for each line segment direction between the image and the virtual image. Can be compared.

  The line distribution information calculation unit 433 stores the descriptor λ in the storage unit 41.

  The search range setting unit 434 sets, as a search range on the map, a range where the host vehicle may be located at the current time t based on the position of the temporary host vehicle. For example, when the vehicle position detection process is performed for the first time, the temporary position of the host vehicle can be the initial value of the position of the host vehicle set by the initial value setting unit 431. In addition, when the vehicle position detection process is performed at the previous time (t-1), the position of the provisional own vehicle is set as the position of the own vehicle detected at the previous time (t-1). Can do. In addition, the search range setting unit 434 predicts the position of the host vehicle detected at the previous time (t-1) based on the vehicle speed or the steering angle received from the vehicle speed sensor or the steering angle sensor attached to the vehicle 10. The predicted position of the host vehicle at the current time t may be the temporary host vehicle position.

The search range setting unit 434 sets a predetermined range centered on the position of the temporary host vehicle as the search range. For example, in a navigation system using GPS or GPS, an accuracy of about 20 m can be expected as a vehicle position detection accuracy. Accordingly, the search range setting unit 434 sets, for example, a range having a radius of 20 m centered on the position of the temporary host vehicle as the search range. Alternatively, the search range setting unit 434 has a relatively wide search range within ± 90 ° with respect to the direction θ _k of the traveling direction of the host vehicle at the position of the temporary host vehicle. For example, the position of the temporary host vehicle Within 20 m from the search range, while the search range is relatively narrow in a direction deviating from ± 90 ° with respect to the azimuth θ _k . For example, the search range may be within 10 m from the position of the provisional host vehicle. . Thereby, since the search range becomes narrow, the number of virtual photographing points included in the search range is reduced, and as a result, the amount of calculation by the comparison unit 435 described later is reduced.

The comparison unit 435 identifies a virtual shooting point that has the best line segment distribution for each region and for each direction of the line segment among the virtual shooting points in the search range. Then, the comparison unit 435 sets the specified virtual shooting point as the position of the host vehicle at the current time t.
For this purpose, the comparison unit 435 reads the descriptor a _k for the virtual shooting point included in the search range from the storage unit 41. Then, for each of the read descriptors a _k , the comparison unit 435 calculates the inner product γ _k = a _k ^t λ of the descriptors λ obtained from the image as an evaluation value. This evaluation value represents the degree of coincidence between the map line segment distribution information and the image line segment distribution information at the virtual shooting point. The comparison unit 435 sets the virtual shooting point where the evaluation value γ _k is the maximum as the position of the host vehicle at the current time t.

  FIG. 12 is an operation flowchart of the vehicle position detection process. The vehicle position detection system 1, for example, every predetermined period, for example, every imaging period of the in-vehicle camera 2 until it is notified from the navigation system that the position of the host vehicle can be detected based on GPS. Then, the position of the host vehicle is detected according to the following operation flowchart.

  The line feature extraction unit 432 extracts a line segment on the road from the image generated at the time closest to the current time t among the images generated by the in-vehicle camera 2 (step S101). Then, the line feature extraction unit 432 sets the direction of the line segment, that is, the start point and the end point of each extracted line segment according to the luminance gradient in the direction crossing the line segment (step S102).

  The line distribution information calculation unit 433 determines the descriptor λ by dividing the image into grids and calculating a normalized histogram representing the total length of the line segments for each line segment direction for each grid ( Step S103).

Further, the search range setting unit 434 sets a search range on the map based on the position of the temporary host vehicle (step S104). The comparison unit 435 calculates, for each virtual shooting point in the search range, the inner product of the descriptor a _k that is the map line segment distribution information of the virtual shooting point and the descriptor λ that is the image line segment distribution information. Then, an evaluation value representing the degree of coincidence of the distribution of the line segment for each local area and for each direction between the descriptor a _k and the descriptor λ at the virtual photographing point is calculated (step S105). Then, the comparison unit 435 detects the virtual shooting point with the maximum evaluation value as the position of the host vehicle at the current time t (step S106). And the control part 43 complete | finishes a vehicle position detection process. In addition, the control part 43 may replace the order of the process of step S101-103 and the process of step S104, or may perform it in parallel.

  As described above, the vehicle position detection system according to the first embodiment of the present invention can detect the position of the host vehicle without the process of repeated trials as in template matching. Therefore, this vehicle position detection system can suppress processing load. In addition, this vehicle position detection system represents an object on a road on a virtual image virtually obtained from a map and a distribution for each local and orientation of a line segment representing the object on the road on the image. By comparing the local distribution and the distribution for each direction of the line segment, and the virtual shooting point with the most consistent distribution is set as the position of the own vehicle, the position of the own vehicle can be accurately detected. Also, the direction of the line segment representing the object on the road is less affected by the wear of a part of the object than the feature point such as the end point of the object on the road. Therefore, this vehicle position detection system can detect the position of the host vehicle with high robustness against changes in the environment of the road surface. Furthermore, since this vehicle position detection system also expresses information indicating the luminance gradient in the direction across the line segment as the direction of the line segment, the difference in the position of the contour relative to the object on the road is also evaluated in the comparison of the line segment distribution. can do. Therefore, this vehicle position detection system can accurately detect the position of the host vehicle.

  Next, a vehicle position detection system according to the second embodiment will be described. In this vehicle position detection system, a grid is set on the map itself, and a normalized histogram of the length of the line segment for each line segment direction is created in advance for each grid. On the other hand, this vehicle position detection apparatus sets a grid in the bird's-eye conversion image obtained by performing bird's-eye conversion on each line segment extracted from the image obtained by the in-vehicle camera, and for each grid, the line segment for each line segment direction. Create a normalized histogram of length of. The vehicle position detection device compares the normalized histogram for each grid calculated from the map with the normalized histogram for each grid calculated from the bird's-eye conversion image, and matches the position on the map that best matches the bird's-eye conversion image. By detecting this, the position of the host vehicle is detected.

  Compared with the vehicle position detection system according to the first embodiment, the vehicle position detection system according to the second embodiment stores information related to the map stored in advance in the storage unit 41 and processing performed by the control unit 43. Some of them are different. Therefore, hereinafter, information related to the map and the control unit 43 stored in advance in the storage unit 41 will be described. For other components of the vehicle position detection system, refer to the description of the corresponding components of the vehicle position detection system according to the first embodiment.

  In the present embodiment, the storage unit 41 includes, as information related to the map, a normalized histogram that represents a distribution of lengths of line segments for each line segment direction for each grid set directly on the map, and each grid Remember the range.

  FIG. 13 is a diagram illustrating an example of a grid set in a map. As shown in FIG. 13, a plurality of grids 1301 are set on the map 1300. Each lattice is set so that one side is along the Xw axis and the side perpendicular to the side is along the Zw axis. The size of each grid is set to a size, for example, 1 m × 1 m, for example, so that the road marking extends over a plurality of grids and a part of the left and right contours of one lane marking is included in one grid. The Each grid is assigned a number in order of raster scanning, for example, starting from the upper left corner.

For each grid set in the map, a normalized histogram of the length of the line segment for each direction of the line segment included in the grid is generated. The method for generating the normalized histogram is the same as the method for generating the normalized histogram of the length of the line segment for each line segment direction for the virtual image or the grid set for the image in the first embodiment. Hereinafter, for convenience, the normalized histogram of the j-th lattice is denoted as η _j (j = 1, 2,..., J).

  FIG. 14 shows a functional block diagram of the control unit 43 according to the second embodiment. As illustrated in FIG. 14, the control unit 43 includes an initial value setting unit 531, a line feature extraction unit 532, a bird's eye conversion unit 533, a line distribution information calculation unit 534, a search range setting unit 535, and a comparison unit 536. And have. Each of these units included in the control unit 43 is implemented as a functional module realized by a computer program executed on a processor included in the control unit 43, for example.

  Compared with the control unit 43 according to the first embodiment, the control unit 43 according to the second embodiment includes a point having a bird's-eye conversion unit 533, a line distribution information calculation unit 534, a search range setting unit 535, and a comparison unit 536. The processing performed depends on. Therefore, hereinafter, the bird's-eye conversion unit 533, the line distribution information calculation unit 534, the search range setting unit 535, and the comparison unit 536 will be described.

ここで(uⁱ _mt,vⁱ _mt)は、現時刻tに最も近い時刻に撮影された画像から抽出された線分のうちのi番目の線分の始点(m=s)または終点(m=e)についての画像上の水平方向の座標値及び垂直方向の座標値を表す。またbⁱ _mt=(bⁱ _m,x,t,bⁱ _m,z,t)^tは、抽出された線分のうちのi番目の線分の始点(m=s)または終点(m=e)が鳥瞰変換された位置の車両座標系での位置を表す。そして(c_u,c_v)、(f_u,f_v)は内部パラメータであり、具体的に、(c_u,c_v)は、車載カメラ２の光軸に対応する画像上の点、すなわち、画像中心の座標である。また(f_u,f_v)は、水平方向と垂直方向の1画素あたりのサイズの違いを考慮した、車載カメラ２の焦点距離に対応する値である。またycは、路面から車載カメラ２までの高さを表す。 The bird's-eye conversion unit 533 compares the line segments extracted from the image by the line feature extraction unit 532 with the line segments represented in the map information stored in the storage unit 41 so that the road surface around the vehicle 10 is Assuming that the plane is a plane, bird's-eye view transformation is performed in the vehicle coordinate system. For example, the bird's-eye conversion unit 533 performs bird's-eye conversion of the start point and the end point of each line segment according to the following equation. For the sake of convenience, a line segment that has been bird's-eye converted will be referred to as an image line segment.

Here, (u ⁱ _mt , v ⁱ _mt ) is the start point (m = s) or end point (m of the i-th line segment extracted from the image taken at the time closest to the current time t. = e) represents the horizontal coordinate value and the vertical coordinate value on the image. B ⁱ _mt = (b ⁱ _{m, x, t} , b ⁱ _{m, z, t} ) ^t is the start point (m = s) or end point (m = s) of the i-th line segment of the extracted line segments. e) represents the position in the vehicle coordinate system of the position subjected to bird's-eye conversion. (C _u , c _v ) and (f _u , f _v ) are internal parameters. Specifically, (c _u , c _v ) is a point on the image corresponding to the optical axis of the in-vehicle camera 2, that is, , The coordinates of the image center. Further, (f _u , f _v ) is a value corresponding to the focal length of the in-vehicle camera 2 in consideration of the difference in size per pixel in the horizontal direction and the vertical direction. Yc represents the height from the road surface to the in-vehicle camera 2.

  FIG. 15 is an example of a bird's-eye view image obtained by performing bird's-eye conversion on the extracted line segment. As shown in FIG. 15, it can be seen that in the bird's-eye view image 1500, each image line segment 1501 represents a lane marking, a road marking, and the like as viewed from directly above.

  The bird's eye conversion unit 533 stores the coordinates of the start point and end point of each image line segment in the storage unit 41.

  The line distribution information calculation unit 534 obtains image line segment distribution information based on the bird's-eye view image. In the present embodiment, the line distribution information calculation unit 534 divides the bird's-eye view image into a plurality of grids.

FIG. 16 is a diagram illustrating an example of a grid set in the bird's-eye view image. The bird's-eye view image 1600 is divided by a grid 1601 having the same size as the grid set on the map. For example, the grids are numbered in the order of raster scanning. Then, the line distribution information calculation unit 534 calculates, for each grid set in the bird's-eye image, a normalized histogram of the length of the image line segment for each direction of the line segment included in the grid. The method for calculating the normalized histogram is the same as the method for calculating the normalized histogram of the length of the line segment for each line segment direction for the virtual image or the grid set for the image in the first embodiment. Similarly to the first embodiment, the line distribution information calculation unit 433 has descriptors λ = (κ ₁ , κ ₂ ,..., Κ _J ) in which normalized histograms are arranged in the order of the grid numbers over the entire bird's-eye view image. ^t is created as image line segment distribution information. Here, κ _j represents a normalized histogram for the j-th lattice on the bird's-eye view image. The line distribution information calculation unit 534 sets the coordinates of the center point of each lattice as χ _m = (χ _mx , χ _mz ) ^t (m = 1, 2,..., M).

  Similar to the search range setting unit 434 according to the first embodiment, the search range setting unit 535 sets a predetermined range centered on the position of the temporary host vehicle as the search range on the map. The position of the temporary host vehicle and the predetermined range may be set similarly to the search range setting unit 434.

Further, in the present embodiment, the search range setting unit 535 sets one or more virtual shooting points within the search range. For example, the search range setting unit 535 is arranged at regular intervals within the search range so that the interval between adjacent virtual shooting points is approximately the same as the accuracy of the required detection position of the vehicle (for example, 1 m to 5 m). Set multiple virtual shooting points. Similarly to the first embodiment, the search range setting unit 535 may set a plurality of shooting directions for one virtual shooting point. Hereinafter, the virtual shooting point including the shooting direction is expressed as y _k = (x _k , z _k , θ _k ) ^t (k = 1, 2,..., K).

  The comparison unit 536 identifies the position on the map that most closely matches the bird's-eye view image by comparing the image line segment distribution information obtained for the bird's-eye view image and the map line segment distribution information, and determines the position at the current time t. It detects as the position of the own vehicle.

ここで、回転行列R^vg _kは、k番目の仮想撮影点に対応する回転行列であり、θ_kは、その仮想撮影点についての撮影方向である。また並進行列t^vgは、世界座標系でのk番目の仮想撮影点の位置を表す。またφ_kmは、k番目の仮想撮影点に対応する回転並進行列にて座標変換することにより世界座標系で表された、m番目(m=1,2,...,M)の格子の中点の座標である。 For this purpose, the comparison unit 536 uses, for each virtual imaging point, the position of the center point of each grid set for the bird's-eye view image using the rotation parallel ^{progression sequence} R ^vg _k and t ^vg _k determined from the virtual imaging point, Convert to world coordinate system according to the following formula.

Here, the rotation matrix R ^vg _k is a rotation matrix corresponding to the k-th virtual shooting point, and θ _k is the shooting direction for the virtual shooting point. The ^{parallel progression} row t ^vg represents the position of the kth virtual shooting point in the world coordinate system. Φ _km is the m-th (m = 1,2, ..., M) lattice represented in the world coordinate system by transforming the coordinates with the rotation parallel progression corresponding to the k-th virtual imaging point. The coordinates of the midpoint.

The comparison unit 536 identifies a grid set on the map including the midpoint φ _km for each virtual shooting point. The comparison unit 536 then arranges the normalized histograms of the grids including the midpoints in the order of the numbers of the midpoints φ _km (m = 1, 2,..., M). As an example, assume that descriptor a _k = (η _{k, 1} , η _{k, 2} ,..., Η _{k, J} ) ^t .
Further, the comparison unit 536 excludes bins corresponding to no line segments so as to cancel the deviation of the direction of the line segment due to the imaging direction θ _{k at the} virtual imaging point in the normalized histogram η _{k, j} of each grid. The bin value is shifted by the number of bins corresponding to that θ _k .

The comparison unit 536 calculates the inner product γ _k = a _k ^t λ of the descriptor a _k corresponding to each virtual shooting point with the descriptor λ obtained from the bird's-eye view image, the image line segment distribution information and the map line segment. It is calculated as an evaluation value representing the degree of coincidence of distribution information. The comparison unit 536 sets the virtual shooting point at which the evaluation value γ _k is the maximum as the position of the host vehicle at the current time t.

  FIG. 17 is an operation flowchart of a vehicle position detection process according to the second embodiment.

  The line feature extraction unit 532 extracts a line segment on the road from the image generated at the time closest to the current time t among the images generated by the in-vehicle camera 2 (step S201). Then, the line feature extraction unit 532 sets the direction of the line segment, that is, the start point and the end point of each extracted line segment according to the luminance gradient in the direction crossing the line segment (step S202).

  The bird's-eye conversion unit 533 performs bird's-eye conversion on the start point and end point of each line segment, and identifies the position on the bird's-eye image represented in the vehicle coordinate system for each line segment (step S203).

  The line distribution information calculation unit 534 obtains the descriptor λ by dividing the bird's-eye view image for each grid and calculating a normalized histogram representing the total length of the line segments for each direction of each line. (Step S204).

  Further, the search range setting unit 535 sets a search range on the map based on the position of the temporary host vehicle (step S205). Then, the search range setting unit 535 sets a plurality of virtual shooting points within the search range (step S206).

For each virtual imaging point in the search range, the comparison unit 536 uses the rotation parallel progression sequence corresponding to the virtual imaging point to set the position of the midpoint of each grid set in the bird's-eye image to the position of the world coordinate system. Conversion is performed (step S207). Then, the comparison unit 536 specifies a grid on the map including the midpoint of each grid for each virtual shooting point, and the normalized histogram of the specified grid is the same as the grid order described in the descriptor λ. By arranging them in order, a descriptor a _k for each virtual shooting point is _obtained (step S208). Further, the comparison unit 536 corrects the deviation of the direction of the line segment in the normalized histogram of each grid according to the shooting direction θ _k at the virtual shooting point (step S209). Then, for each virtual shooting point, the comparison unit 536 calculates the inner product of the descriptor λ that is the image line segment distribution information obtained from the bird's-eye image and the descriptor a _k that is the map line segment distribution information at the virtual shooting point. Thus, in the map of the position corresponding to the bird's-eye view image and the virtual shooting point, the evaluation value representing the degree of coincidence of the distribution of the line segment for each region and for each direction between the descriptor a _k and the descriptor λ is calculated. (Step S210). Then, the comparison unit 536 detects the virtual shooting point having the maximum evaluation value as the position of the host vehicle at the current time t (step S211). And the control part 43 complete | finishes a vehicle position detection process. In addition, the control part 43 may replace the order of the process of step S201-204 and the process of step S205-S206, or may perform it in parallel.

  As described above, the vehicle position detection system according to the second embodiment of the present invention has the same effects as the vehicle position detection system according to the first embodiment. Furthermore, in the vehicle position detection system according to the second embodiment, since the virtual shooting point is set only in the search range, the amount of information prepared in advance and stored in the storage unit is the first embodiment. Less than. Therefore, the vehicle position detection system according to the second embodiment can reduce the storage capacity of the storage unit as compared with the first embodiment.

  The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. According to the modification, the information about the direction of the luminance gradient in the direction across the line segment extracted from the image by the map line segment and the line feature extraction unit may be expressed in another format instead of the direction of the line segment. Good. For example, the direction of the luminance gradient in the direction across the line segment may be represented by a flag associated with the line segment. For example, if the luminance decreases along the direction crossing the line from left to right or from top to bottom, the flag value is set to '0' and the line is moved from right to left or down. If the luminance decreases along the direction crossing upward from, the flag value may be set to “1”.

  The position of the host vehicle output from the vehicle position detection system according to the above embodiment or modification is transmitted to a control circuit (not shown) of the driving support system via, for example, CAN3. The control circuit of the driving support system compares, for example, the position of the host vehicle and the surrounding information, and within a predetermined distance range from the host vehicle, for example, a specific structure (for example, a highway toll booth, a route being navigated) If there is an intersection that requires a left turn or a right turn, the driver is notified that the structure is near through a display or a speaker installed in the vehicle. Alternatively, the control circuit of the driving support system may output a command to reduce the speed to the ECU 11.

  As described above, those skilled in the art can make various modifications in accordance with the embodiment to be implemented within the scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Vehicle position detection system 2 Car-mounted camera 3 Control area network (CAN)
4 Controller (vehicle position detection device)
10 Vehicle 11 ECU
41 Storage Unit 42 Communication Unit 43 Control Unit 431 Initial Value Setting Unit 432 Line Feature Extraction Unit 433 Line Distribution Information Calculation Unit 434 Search Range Setting Unit 435 Comparison Unit 531 Initial Value Setting Unit 532 Line Feature Extraction Unit 533 Bird's-eye View Conversion Unit 534 Line Distribution Information calculation unit 535 Search range setting unit 536 Comparison unit

Claims (7)

A map including each position of a plurality of map line segments representing the outline of an object on the road, and map line segment distribution information representing the distribution of the map line segments for each locality and for each direction of the map line segment. A storage unit (41) for storing;
A plurality of line segments representing the contour of an object on the road shown in the image are obtained from an image obtained by photographing the periphery of the vehicle (10) by the imaging unit (2) mounted on the vehicle (10). Line feature extraction units (432, 532) to be extracted;
A line distribution information calculation unit (433, 534) for obtaining image line segment distribution information representing the distribution of the line segment for each local area and for each direction of the line segment on the image;
For each of a plurality of virtual shooting points set on the map, an evaluation value representing the degree of coincidence between the map line segment distribution information and the image line segment distribution information at the virtual shooting point is calculated, and the evaluation value is A comparison unit (435, 536) that sets the maximum virtual shooting point as the position of the vehicle (10);
A vehicle position detection device comprising: For each of the plurality of virtual shooting points, the storage unit (41) includes a plurality of grids obtained by dividing a virtual image obtained by virtually shooting the map with the imaging unit (2) from the virtual shooting points. For each of the above, the distribution for each direction of the map line segment represented on the virtual image included in the grid is stored as the map line segment distribution information,
The line distribution information calculation unit (433) divides the image into a plurality of grids, and for each of the plurality of grids, a distribution for each direction of the line segments represented on the image included in the grids. The vehicle position detection device according to claim 1, wherein the image line segment distribution information is set as the image line segment distribution information. A bird's-eye conversion unit (533) that performs bird's-eye conversion on each of the plurality of line segments and obtains a bird's-eye image representing the position of each of the plurality of line segments in the vehicle coordinate system with reference to the vehicle (10). Have
The storage unit (41) stores, for each of a plurality of first grids obtained by dividing the map, a distribution for each direction of the map line segment represented on the map included in the first grid. And
The line distribution information calculation unit (534) divides the bird's-eye view image into a plurality of second lattices, and each of the plurality of second lattices is included on the bird's-eye image included in the second lattice. The distribution for each direction of the represented line segment is the image line segment distribution information,
The comparison unit (536) specifies, for each of the plurality of virtual shooting points, the first grid corresponding to each of the plurality of second grids in the vehicle coordinate system according to the virtual shooting point. The distribution for each direction of the map line segment of the first grid corresponding to each of the plurality of second grids is used as the map line segment distribution information for the virtual imaging point. The vehicle position detection device according to 1. For each of the plurality of map line segments, the direction of the map line segment is set according to the direction of the luminance gradient in the direction across the map line segment,
For each of the detected line segments, the line feature extraction unit (432, 532) obtains the direction of the luminance gradient in the direction across the line segment, and sets the direction of the line segment according to the direction of the luminance gradient. The vehicle position detection device according to any one of claims 1 to 3. A plurality of line segments representing the contour of an object on the road shown in the image are obtained from an image obtained by photographing the periphery of the vehicle (10) by the imaging unit (2) mounted on the vehicle (10). Extracting, and
On the image, obtaining image line segment distribution information representing the distribution of the line segment for each local area and for each direction of the line segment;
For each of a plurality of virtual shooting points set on a map including the positions of a plurality of map line segments representing the outline of an object on the road, for each local area at the virtual shooting point and the map line segment An evaluation value representing the degree of coincidence between the map line segment distribution information representing the map line segment distribution for each direction and the image line segment distribution information is calculated, and the virtual shooting point having the maximum evaluation value is determined as the vehicle ( 10) the position of the step;
The vehicle position detection method characterized by including. A plurality of line segments representing the contour of an object on the road shown in the image are obtained from an image obtained by photographing the periphery of the vehicle (10) by the imaging unit (2) mounted on the vehicle (10). Extracting, and
On the image, obtaining image line segment distribution information representing the distribution of the line segment for each local area and for each direction of the line segment;
For each of a plurality of virtual shooting points set on a map including the positions of a plurality of map line segments representing the outline of an object on the road, for each local area at the virtual shooting point and the map line segment An evaluation value representing the degree of coincidence between the map line segment distribution information representing the map line segment distribution for each direction and the image line segment distribution information is calculated, and the virtual shooting point having the maximum evaluation value is determined as the vehicle ( 10) the position of the step;
The computer program for vehicle position detection characterized by including the command which makes the processor (43) mounted in the said vehicle (10) execute. An imaging unit (2) mounted on the vehicle (10), which captures an image of the surroundings of the vehicle (10) and generates an image;
A map including each position of a plurality of map line segments representing the outline of an object on the road, and map line segment distribution information representing the distribution of the map line segments for each locality and for each direction of the map line segment. A storage unit (41) for storing;
A communication unit (42) for acquiring the image from the imaging unit (2);
A control unit,
Extracting a plurality of line segments representing the contour of an object on the road in the image;
Finding image line segment distribution information representing the distribution of the line segment for each local area and for each direction of the line segment on the image,
For each of a plurality of virtual shooting points set on the map, an evaluation value representing the degree of coincidence between the map line segment distribution information and the image line segment distribution information at the virtual shooting point is calculated, and the evaluation value is A control unit (43) which sets the maximum virtual shooting point as the position of the vehicle (10);
A vehicle position detection system comprising:

Images