JP2018194417A - Position estimation device, mobile device - Google Patents

Abstract

To achieve highly accurate position estimation without using an expensive instrument.SOLUTION: A self-driving vehicle 1 comprises: a camera 22; and the other camera. Two cameras are comprised of a stereo cameras, and are used for measuring a distance to each position of forward road surfaces. The distance to the road surface is measured, and thereby an approximation expression of the road surface with a camera coordinate system set as a reference is obtained. A posture of the self-driving vehicle 1 can be estimated using this approximation expression. The stereo camera is virtually arranged on three-dimensional map data by the estimated posture, and distance data is obtained that is acquired by the virtually arranged stereo camera. Distance data generated by photographing and the distance data generated based on the three-dimensional map data are caused to match with each other, thereby estimating a position.SELECTED DRAWING: Figure 6

Description

  The present disclosure relates to position estimation.

  Patent Document 1 discloses a technique for real-time positioning using an in-vehicle camera. This technique measures the position of the vehicle by extracting features such as straight lines and corner points from the real image area and matching them with map data.

JP 2012-185011 A

  In the case of the above prior art, since the physical quantity used for matching is only the luminance value, the measurement accuracy is low. For example, RTK-GPS is known as a technique capable of highly accurate positioning, but the receiver is expensive. Based on the above, it is an object of the present disclosure to realize highly accurate position estimation without using expensive equipment.

  One form of the present disclosure is a mobile device (1, 2) having a function of moving on a road surface and a distance measuring device (21, 22) for acquiring a distance value to a surrounding target including the road surface. A position estimation device (10, 10a) for estimating a position; an attitude determination unit that determines an attitude of the moving device with respect to a surrounding road surface based on distance data that is a set of distance values to the road surface around the moving device. (S200); using the posture determined by the posture determination unit, the distance value obtained by simulating the measurement by the distance measuring device virtually arranged in the space defined by the three-dimensional map data A calculation unit (S310, S320) that calculates distance data that is a set for each of a plurality of candidate positions; from among the plurality of candidate positions; A position estimation device comprising: a measurement result by the location selection unit that selects based on the matching between the calculated results by the calculating section and (S400). According to this embodiment, it is possible to estimate the position with high accuracy by using a normal distance measuring device.

The block block diagram of an autonomous vehicle. The flowchart which shows a position estimation process. The flowchart which shows an attitude | position determination process. An example of a captured image. The graph which shows the distance value along each y value. The graph which shows the relationship between y value and distance value in the road surface in the case of a reference state. The graph which shows the relationship between y value and distance value in the road surface at the time of pitching. The flowchart which shows the preparation process of matching. The figure which shows a mode that the value of the low-order area | region was removed. The figure which shows a mode that the distance value of the high level area | region was removed. The figure which shows a mode that a position estimation apparatus and several autonomous driving vehicles communicate. FIG. 3 is a block diagram of an automatic driving vehicle according to a second embodiment.

  Embodiment 1 will be described. An automatic driving vehicle 1 shown in FIG. 1 is an automatic driving vehicle of level 1 or higher. Specifically, the autonomous driving vehicle 1 has a function that can automatically perform at least steering.

  The automatic driving vehicle 1 is equipped with an ECU 10. The ECU 10 is a camera ECU. ECU10 acquires the RGB value of each pixel as imaging data from the camera 21 and the camera 22, and performs the position estimation process mentioned later.

  The cameras 21 and 22 are mounted so that the front of the autonomous driving vehicle 1 is within the imaging range. The cameras 21 and 22 acquire the luminance value of each pixel and input it to the ECU 10. The cameras 21 and 22 constitute a stereo camera arranged side by side in the vehicle width direction for distance measurement using parallax. The ECU 10 can acquire a distance value for each imaged pixel by using the focal length and the base line length of the cameras 21 and 22. The distance value is a value indicating the distance from the cameras 21 and 22 to the imaged object.

  As shown in FIG. 1, the autonomous driving vehicle 1 includes a yaw rate sensor 30, a vehicle speed sensor 40, a GNSS receiver 50, and a map data storage device 60. The yaw rate sensor 30 measures the yaw rate of the autonomous driving vehicle 1 and inputs it to the ECU 10. The vehicle speed sensor 40 measures the vehicle speed of the autonomous driving vehicle 1 and inputs it to the ECU 10.

  The GNSS receiver 50 receives navigation signals to perform GNSS surveying. GNSS is an acronym for Global Navigation Satellite System, which is a global navigation satellite system.

  The map data storage device 60 stores 3D map data. In the three-dimensional map data here, the presence or absence of a target is associated with each three-dimensional lattice point, and a luminance value is further associated with the lattice point associated with the presence of the target. Data structure. The position of each three-dimensional grid point is defined by each value of latitude, longitude, and altitude.

  The target mentioned here means all objects that are recognized as having some object by data collection using radar, images, or the like. The target includes a road surface. In this embodiment, since the position estimation target is the autonomous driving vehicle 1, the road surface here is the surface of the roadway. As the luminance value data, values to be stored in the three-dimensional map data are determined based on image data captured in the daytime.

  The map data storage device 60 extracts a part of data from the three-dimensional map data using the signal input from the GNSS receiver 50 and inputs the extracted data to the ECU 10.

  The ECU 10 estimates the position of the autonomous driving vehicle 1 by executing the position estimation process shown in FIG. 2 using the image data obtained by the cameras 21 and 22, the yaw rate, the vehicle speed, and the three-dimensional map data. For this reason, the position estimation process in this embodiment is a self-position estimation process.

  The ECU 10 repeatedly executes the position estimation process while the automatic driving function is set to ON. The ECU 10 inputs the estimated position to the ECU that controls automatic driving. Note that the ECU that controls the automatic operation is not shown.

  As shown in FIG. 2, when the position estimation process is started, the ECU 10 acquires distance data and luminance data from the imaging data as S100. The distance data is data having a structure in which a distance value is associated with each pixel. Luminance data is data having a structure in which a luminance value is associated with each pixel.

  Subsequently, the ECU 10 executes a posture determination process as S200. When the posture determination process is started, the ECU 10 executes road surface estimation in S210. Hereinafter, S210 will be described by taking the captured image shown in FIG. 4 as an example. As shown in FIG. 4, the horizontal direction of the imaging range is defined as the x direction and the vertical direction is defined as the y direction.

  S210 is a step of determining an expression representing the road surface based on the distance data. The formula determined here is defined in a coordinate system whose conversion relationship with the camera coordinate system is known. For this reason, S210 is equivalent to calculating | requiring the formula of the road surface defined in a camera coordinate system.

  As shown in FIG. 4, the distance values acquired along y0 to y8 are extracted and used for the description. y0 is a value corresponding to the lower side of the imaging range, and y8 is a value corresponding to the upper side of the imaging range.

  FIG. 5 shows a state in which distance values are plotted for each of y0 to y8 on a graph in which the vertical axis is the distance value z and the horizontal axis is the horizontal direction x. y7 is not shown in FIG. 5 because it covers the sky and a high-rise building located far away and is outside the range of the z-axis shown in FIG. y8 is not shown in FIG. 5 because it covers the sky and is outside the range of the z-axis shown in FIG.

  FIG. 6 is a graph showing the relationship between the y value and the distance value z for a certain x value in the reference state. The reference state is a state in which the autonomous driving vehicle 1 is not pitching or rolling at all and is running or stopped on a horizontal road surface.

  If the road surface is flat as well as horizontal, as shown in FIG. 6, the relationship between the y value and the distance value z is a straight line. On the other hand, when the road surface is not flat due to a slope ahead, the relationship between the y value and the distance value z is approximated by a curve. Hereinafter, a curve in the relationship between the y value and the distance value z includes a straight line.

  A similar relationship can be obtained for each x value within an area corresponding to the road surface. Therefore, by using the distance data, an equation of a surface that approximates the road surface in the coordinate system with the cameras 21 and 22 as a reference can be obtained.

  However, in this embodiment, it is assumed that the road surface is flat in the width direction. The width direction is a direction along the road surface and a direction orthogonal to the traveling direction defined by the lane. With this assumption, the road surface can be represented by a curve in a two-dimensional space. Therefore, in this embodiment, a representative value of the distance value z is determined for each y value, and one approximate curve is obtained. The representative value in this embodiment is determined by the mode value.

  Note that the width direction and the x direction are not always parallel. For example, when there is a difference between the direction in which the autonomous driving vehicle 1 is actually traveling and the traveling direction of the road surface, or when the autonomous driving vehicle 1 is rolling, the x direction is relative to the width direction. It is not parallel. The correction may be performed in consideration of such a shift, or a curve in a three-dimensional space may be obtained. However, in this embodiment, such a shift is ignored in order to reduce the calculation load. Find a curve in space.

  In the case of the example shown in FIGS. 4 and 5, the road surface is not included in the regions y4 to y8. For this reason, it is better not to use the region y4 to y8 for estimating the road surface. However, even if the region of y4 to y8 is included, the road surface estimation result often does not have a great influence. Therefore, in this embodiment, the road surface is estimated using all y values as objects of calculation. It should be noted that the relationship between the y value and the distance value z in the y4 to y8 region varies greatly from the relationship between the y value and the distance value z in the y0 to y4 region, so that it is possible to separate them. Therefore, in another embodiment, a curve representing a road surface is obtained after excluding an area where the relationship between the y value and the distance value z has changed significantly compared to an area where the y value is small.

  Next, the ECU 10 calculates the height h and the pitch angle θ of the cameras 21 and 22 as S220. As shown in FIG. 6, in the present embodiment, a line of sight where the y value is y3 is used as a reference, and an angle formed by the line of sight and the road surface is defined as a pitch angle. In the reference state, the distance value z when the y value is y3 is known, and is shown as the distance value z3 in FIG. The height h in the reference state is also known because it is measured in advance.

  The height h and the pitch angle θ are obtained by using the above known values and the approximate expression of the y value and the distance value z. In addition, the height in this embodiment is a distance along the direction orthogonal to the road surface. Accordingly, the height direction is different from the vertical direction, for example, when traveling on a hill.

  Finally, a roll angle is estimated as S230. Distance data is used for estimating the roll angle. The relationship learned in advance is used for S230. The relationship learned in advance is a relationship obtained by examining what characteristics the distribution of distance values has in many cases where rolling has occurred.

  After completing the posture determination process, the ECU 10 executes a matching preparation process in S300. As shown in FIG. 8, the ECU 10 first determines a candidate position as S310. The candidate positions are a plurality of positions that are candidates for the current position of the autonomous driving vehicle 1. In the present embodiment, the position of the autonomous driving vehicle 1 includes the yaw angle in addition to the latitude and longitude. The yaw angle is an angle indicating which direction the front of the autonomous driving vehicle 1 is facing.

  FIG. 7 shows two candidate positions ta and tb as candidate positions. In practice, a larger number of candidate positions are determined. The candidate positions are determined at random after narrowing down to a certain range in consideration of the estimated position t-1, the vehicle speed, and the yaw rate estimated in the previous position estimation process.

  Next, ECU10 produces the distance data corresponding to each determined candidate position as S320. In S320, the 3D map data stored in the map data storage device 60 is used. Specifically, when creating distance data corresponding to a certain candidate position, the cameras 21 and 22 are virtually arranged at the candidate position on the three-dimensional map data. The posture determined by the posture determination process is applied to this virtual arrangement. Then, distance data is created from the three-dimensional map data by simulating imaging by the virtually arranged cameras 21 and 22.

  Next, the ECU 10 creates luminance data corresponding to each determined candidate position in S330. S330 has the same contents as S320 except for the difference between the distance value and the luminance value.

  Next, ECU10 removes the value which belongs to a low-order field from distance data as S340. The data targeted in S340 are data created by imaging as S100 and distance data created by S320. The lower region is a region lower than a predetermined height. However, the region lower than the road surface is excluded from the lower region so that the distance value indicating the road surface is not removed.

  Next, ECU10 removes the value which belongs to a low-order field from luminance data as S350. The data to be targeted in S340 are data created by imaging as S100 and luminance data created in S330. The definition of the low level region is as described in the description of S340.

  As shown in FIG. 9, the value corresponding to the preceding vehicle or the pedestrian is removed by removing the lower region. As described above, the purpose of removing the low-order region is to exclude a target that is included in the data created by the imaging but is not included in the three-dimensional map data from the target of matching. For this reason, in the present embodiment, the predetermined height that serves as a reference for the low-level region is determined such that a target moving on or near the road surface is excluded. Specifically, it is determined to be slightly higher than the vehicle height of a large vehicle such as a truck.

  Finally, ECU10 removes the value which belongs to a high-order field from distance data as S360. The data to be targeted in S360 are the distance data created by imaging as S100 and the distance data created by S320, as in S340. The high level area is an area higher than a predetermined height in the vertical direction. The predetermined height here is higher than the predetermined height for the lower region.

  As shown in FIG. 10, the value of the target located far away is removed by removing the high-level region. When comparing FIG. 9 and FIG. 10, it seems that only a high-rise building located far away has been removed, but in reality, the area corresponding to the sky also belongs to the high-order area, so the distance data Removed from.

  The purpose of removing the value belonging to the higher region from the distance data is to remove the distance value with low measurement accuracy. Even if the distance is measured against the sky, the measurement itself is difficult. Even if it is not empty, a target that is too far away has a small parallax value, so that it is difficult to measure with high accuracy. For this reason, it is preferable to exclude regions that are too far from the target of matching. However, if it is attempted to determine whether the distance is too far or not by the distance value, the accuracy of the distance value itself is low, and therefore the accuracy of the determination itself may be lowered. Therefore, in this embodiment, the height is used as a reference. This is based on the fact that it is possible to estimate that a certain high region is a far region in the imaged range.

  Note that the luminance data is not subject to removal even if it is an area that is too far for the distance data, because it does not affect the accuracy so much. For example, a distant high-rise building shown in FIG. 9 can obtain sufficient accuracy as a target for matching luminance data.

  When finishing the matching preparation process, the ECU 10 executes matching as S400. That is, the most likely current position is selected from the candidate positions based on the degree of coincidence between the luminance data and the distance data.

  S400 is performed based on cost calculation. Specifically, the difference is calculated for each of the distance data and the luminance data, and it is determined that the difference is more likely as the difference is smaller. Note that the luminance data created based on the imaging data largely depends on the time zone, weather conditions, and the like, and therefore the gradient of the luminance value (also referred to as an edge) is used as a difference calculation target instead of the luminance value itself.

  In the present embodiment, the difference between the candidate position and the position by odometry is also added to the cost calculation. Odometry is a method for estimating the position from the vehicle speed and the yaw rate.

  Then, ECU10 performs filtering as S500. In S500, a particle filter is used in order to estimate a likely self-position in time series.

  Finally, as S600, the ECU 10 estimates the self position from the result of the above-described steps.

  According to the embodiment described above, high-precision self-position estimation can be realized without using a particularly expensive device. The main reasons for achieving high-precision estimation are that the posture with respect to the road surface is determined and used for matching, and that two types of data, distance data and luminance data, are used for matching, and distance data The lower region and the higher region are removed from the image data, and the lower region is removed from the luminance data.

  Furthermore, according to the present embodiment, the processing load for the position estimation process is reduced. Since the position estimation process is used for automatic driving, it is a process that is repeated in a short cycle. For this reason, a light processing load is an important effect. The main reason for reducing the processing load is that the road surface is two-dimensionalized in the estimation of the road surface.

  Making the road surface two-dimensional has the following advantages in addition to being able to reduce the independent variables in the equations, and according to the method of the present embodiment. The advantage is that it is not necessary to specify where the road surface is in the x direction or the width direction. For this reason, the two-dimensional road surface greatly contributes to the reduction of the processing load.

  Moreover, even if the road surface is two-dimensionalized, the estimation accuracy of the road surface does not deteriorate so much. This is because the road surface is often made almost flat in the width direction so that it can be easily driven by a four-wheeled vehicle, so even if the information is compressed in the width direction, the substantial information is almost the same. . In the present embodiment, the mode value is taken along the x direction. Although strictly different from the compression in the width direction, even if information is compressed along the x direction, there is practically no problem. Can be estimated.

  A second embodiment will be described. The description of the second embodiment mainly focuses on differences from the first embodiment. Points that are not particularly described are the same as those in the first embodiment.

  In the second embodiment, a position estimation system is configured by the position estimation device 10a and the plurality of autonomous driving vehicles 2.

  Each of the plurality of autonomous driving vehicles 2 is not equipped with a position estimation device, and instead communicates with the position estimation device 10a as shown in FIG. The position estimation device 10 a includes a processing unit 12, a map data storage device 60, and a communication interface 70.

  As shown in FIG. 12, the autonomous driving vehicle 2 includes an ECU 80. The ECU 80 transmits information acquired from the cameras 21 and 22, the vehicle speed sensor 40, and the GNSS receiver 50 to the position estimation device 10 a via the communication interface 90.

  The processing unit 12 acquires information from the autonomous driving vehicle 2 via the communication interface 70. The processing unit 12 executes the position estimation process described in the first embodiment using information acquired from the autonomous driving vehicle 2 and information stored in the map data storage device 60. The processing unit 12 transmits the position information estimated by the position estimation process to the autonomous driving vehicle 2.

  The ECU 80 uses position information acquired from the position estimation apparatus 10a via the communication interface 90 for automatic steering.

  A correspondence between the embodiment and the claims will be described. The autonomous vehicles 1 and 2 are mobile devices, the ECUs 10 and 10a are position estimation devices, the camera 21 is a distance measurement device and an imaging device, the camera 22 is a distance measurement device and an imaging device, S200 is an attitude determination unit, and S310. , S320 correspond to a calculation unit, S310 and S330 correspond to an acquisition unit, S340 and S360 correspond to a distance value removal unit, S350 corresponds to a luminance value removal unit, and S400 corresponds to a selection unit. The height of the grazing on the road corresponds to the first brightness height and the first distance height. The predetermined height for the lower region corresponds to the second luminance height and the second distance height. The predetermined height for the higher region corresponds to a predetermined height.

  The present disclosure is not limited to the embodiments, examples, and modifications of the present specification, and can be realized with various configurations without departing from the spirit of the present disclosure. For example, the technical features in the embodiments, examples, and modifications corresponding to the technical features in the embodiments described in the summary section of the invention are to solve some or all of the above-described problems, or In order to achieve part or all of the effects described above, replacement or combination can be performed as appropriate. If the technical feature is not described as essential in this specification, it can be deleted as appropriate. For example, the following are exemplified.

  If the estimation of the road surface in the posture determination process fails, the most recently estimated road surface may be handled as the current road surface.

  An object to be estimated by the position estimation device is not limited to an automobile, but may be a moving device that moves on a road surface. For example, another transportation device (for example, a two-wheeled vehicle) or a robot may be used. In the case of a robot, for example, it may be grounded on the road surface with wheels like an automobile, or may be a type that walks on two legs.

  The distance measuring device may not be a stereo camera. For example, LIDAR or millimeter wave radar may be used. LIDAR is an acronym for Light Detection And Ranging or Laser Imaging Detection And Ranging.

  The luminance value may not be used for matching. In this case, luminance value data is not required even in the three-dimensional map data.

  The preconditions for estimating the road surface may be changed as appropriate. For example, the road surface may be obtained as a curved surface in a three-dimensional space.

  Luminance data may be used for estimating the roll angle. Even in the case of using luminance data, a previously learned relationship may be used as in the case of using distance data.

  In the above embodiment, some or all of the functions and processes realized by software may be realized by hardware. In addition, some or all of the functions and processes realized by hardware may be realized by software. As the hardware, for example, various circuits such as an integrated circuit, a discrete circuit, or a circuit module obtained by combining these circuits may be used.

1 automatic driving vehicle, 2 automatic driving vehicle, 10 ECU, 10a ECU, 21 camera, 22 camera

Claims (12)

Position estimating device for estimating the position of a moving device (1, 2) having a function of moving on a road surface and a distance measuring device (21, 22) for acquiring distance values to surrounding targets including the road surface (10, 10a),
An attitude determination unit (S200) that determines an attitude of the mobile device with respect to a surrounding road surface based on distance data that is a set of distance values to the road surface around the mobile device;
Distance data that is a set of distance values obtained by simulating measurements by the distance measuring device virtually arranged in the space defined by the three-dimensional map data using the posture determined by the posture determining unit Calculating unit (S310, S320) for calculating each of a plurality of candidate positions;
A selection unit (S400) that selects a plausible position as the position of the mobile device from the plurality of candidate positions based on matching between a measurement result by the distance measuring device and a calculation result by the calculation unit;
A position estimation apparatus comprising: The moving device has an imaging device (21, 22) for measuring a luminance value of a surrounding target
In the three-dimensional map data, a luminance value at a three-dimensional position is stored,
Using the attitude determined by the attitude determination unit, obtain a luminance value obtained by simulating imaging by the imaging device virtually arranged on three-dimensional map data for each of the plurality of candidate positions. It further includes an acquisition unit (S310, S330),
The position estimation device according to claim 1, wherein the selection unit performs the selection in consideration of matching between a measurement result obtained by the imaging device and an acquisition result obtained by the acquisition unit. The luminance value removing unit (S350) for removing a value belonging to an area not used for the matching from the luminance value acquired by the acquiring unit and the luminance value measured by the imaging device. Position estimation device. The position estimation apparatus according to claim 3, wherein the luminance value removing unit performs the removal on a region from a first luminance height to a second luminance height with reference to a road surface. The distance value removal part (S340, S360) which removes the value which belongs to the area | region which is not used for the said matching from the distance value measured by the said ranging device, and the distance value computed by the said calculation part. The position estimation apparatus according to any one of claims 1 to 4. The position estimation device according to claim 5, wherein the distance value removal unit (S340) performs the removal on a region from a first distance height to a second distance height on the basis of a road surface. The position estimation device according to any one of claims 5 to 6, wherein the distance value removal unit (S360) performs the removal on a region having a predetermined height or more. The attitude determination unit determines a value representative of a predetermined direction to two-dimensionally measure distance data measured by the distance measuring device, and determines a relative position of the road surface with respect to the distance measuring device based on the two-dimensional distance data. The position estimation apparatus according to any one of claims 1 to 7, wherein the posture is determined. The distance measuring device is an imaging device configured as a stereo camera,
The predetermined direction is a horizontal direction in imaging,
The position estimation device according to claim 8, wherein the posture determination unit determines a pitch angle of the moving device and a height from a road surface of the imaging device as the posture. The posture determination unit executes the determination of the posture using a position determined in the past when the determination of the relative position of the road surface fails. Position estimation device. The position estimating device according to any one of claims 1 to 10, wherein the moving device is an autonomous driving vehicle (1). A mobile device (1) according to any one of claims 1 to 11, comprising:
A movement apparatus comprising the position estimation device (10) according to any one of claims 1 to 11.

Images