JP7228219B2 - Lateral position estimation device and lateral position estimation method - Google Patents

Description

The present invention relates to a lateral position estimating device and a lateral position estimating method for estimating the lateral position of a moving body with respect to the running direction.

Currently, companies, universities, and the like are actively conducting research on automatic driving of moving bodies such as vehicles (see Non-Patent Documents 1 and 2, etc.). In order for a vehicle to operate automatically on public roads, it is necessary to accurately estimate the position of the vehicle on the road. In particular, accurate estimation of the vehicle's lateral position with respect to the direction of travel is necessary to achieve safe automated driving.

For example, in Non-Patent Document 1, the position of a vehicle is estimated by calculating the degree of similarity between a map generated in advance and the observation value of LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) from the vehicle. Techniques are disclosed.

Daiki Yamamoto, Naoki Suganuma, "Self-position estimation of self-driving car using high-resolution infrared reflection image", Japan Society of Mechanical Engineers, 23rd Transportation and Logistics Division Conference Proceedings 2014 (23), p. 329-330 J. Levinson and S. Thrun, "Robust vehicle localization in urban environments using probabilistic maps", in Proceedings of IEEE Conference on robotics and automation, pp. 4372-4378, 2010

In order for a vehicle to operate automatically on public roads, it is necessary to accurately estimate the lateral position of the vehicle regardless of the road environment, weather, and other conditions. However, for example, when the road surface condition changes due to snowfall, the map generated before the road surface condition change and the LIDAR observation value acquired after the road surface condition change may not match. For this reason, the conventional techniques including the technique described in Non-Patent Document 1 cannot always perform accurate position estimation.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a lateral position estimating device and a lateral position estimating method capable of accurately estimating the lateral position of a moving object.

In order to achieve the above object, a lateral position estimation device according to one aspect of the present invention is a lateral position estimation device for estimating a lateral position of a moving object with respect to a running direction, the moving object running. 1 shows a relationship between an acquisition unit that acquires image data obtained by capturing an image of a road on which the vehicle is located from the moving body, the lateral position in the image data, and an edge strength that indicates a rate of change in luminance in the image data with respect to the lateral position; In the edge profile, a reconstruction unit that reconstructs the edge profile by emphasizing the edge strength; and the reconstructed edge profile and the lateral direction obtained from road map information corresponding to the image data. a matching unit for calculating the lateral position of the moving object by matching a map edge profile showing the relationship between the position and the edge strength.

In order to achieve the above object, a lateral position estimation method according to one aspect of the present invention is a lateral position estimation method for estimating a lateral position of a moving object with respect to a running direction, the method comprising: an acquisition step of acquiring image data obtained by capturing an image of a road on which the vehicle is moving from the moving body; the relationship between the horizontal position in the image data and edge strength indicating a rate of change in luminance in the image data with respect to the horizontal position; a reconstruction step of reconstructing the edge profile by emphasizing the edge strength; and the reconstructed edge profile and the road map information corresponding to the image data. and a collation step of calculating the lateral position of the moving body by collating a map edge profile indicating the relationship between the lateral position and the edge strength.

With such a lateral position estimating device and lateral position estimating method, even if the edge intensity in the edge profile obtained from the image data is insufficient for accurately estimating the lateral position, the By enhancing the edge intensity in the edge profile by the reconstructor, the lateral position can be estimated with high accuracy.

In addition, these general or specific aspects may be realized by a system, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media.

According to the lateral position estimating device and lateral position estimating method of the present invention, the lateral position of a moving object can be accurately estimated.

FIG. 1 is a schematic diagram illustrating the lateral position of a vehicle. FIG. 2A is a diagram showing an example of road image data obtained from LIDAR observation values. FIG. 2B is a diagram showing map information of the road shown in FIG. 2A. FIG. 2C is a diagram showing a result of comparing the image data shown in FIG. 2A and the map information shown in FIG. 2B. FIG. 3A is a photograph showing an example of a road on which vehicles travel. FIG. 3B is a diagram showing map information of the road shown in FIG. 3A. FIG. 3C is a diagram showing road image data obtained from LIDAR observations installed on a vehicle traveling on the road shown in FIG. 3A. 4 is a block diagram showing an example of the configuration of the lateral position estimation device according to Embodiment 1. FIG. FIG. 5 is a diagram showing an example of the road on which the vehicle V10 travels and the position estimation result on the road. FIG. 6 is a diagram showing another example of the road on which the vehicle V10 travels and the position estimation result on the road. 7 is a graph showing the behavior of eigenvectors according to Embodiment 1. FIG. FIG. 8A is a graph showing placement of first and second eigenvectors of a map edge profile obtained from map information. FIG. 8B is a graph showing an example of a profile with out-of-range peaks. FIG. 8C is a graph showing a profile obtained by reconstructing the profile shown in FIG. 8B. FIG. 9 is a graph showing the relationship between the eigenvectors used in the reconstruction according to Embodiment 1 and the magnitude of change in the profile caused by encoding using the eigenvectors. FIG. 10 is a flowchart showing a lateral position estimation method according to Embodiment 1. FIG. 11 is a diagram showing a first operation example of the lateral position estimation device according to Embodiment 1. FIG. 12 is a diagram showing a second operation example of the lateral position estimation device according to Embodiment 1. FIG. 13A and 13B are diagrams showing a third operation example of the lateral position estimation device according to Embodiment 1. FIG. 14 is a diagram showing a fourth operation example of the lateral position estimation device according to Embodiment 1. FIG.

(Knowledge on which the present invention is based)
Prior to the description of the present invention, the knowledge on which the present invention is based will be described.

FIG. 1 is a schematic diagram illustrating the lateral position of a vehicle 1000. FIG. As shown in FIG. 1, the lateral position of the vehicle 1000 means the position in the direction transverse to the direction of travel of the vehicle 1000 traveling on the road R0. By estimating the lateral position, it is possible to determine whether the vehicle 1000 is in an appropriate position in the driving lane of the road R0 in automatic driving. For example, on a road R0 in Japan, as shown in FIG. 1, a center line Lc is shown near the center of two edges Er, which are ends in the transverse direction. Further, on the road R0, a roadway outside line Ls indicating the outer edge of the roadway is shown. On the road R0, the appropriate lateral position of the traveling vehicle 1000 is on the left side of the center line Lc and between the center line Lc and the roadway outer line Ls.

Here, a method for estimating the lateral position of a vehicle using LIDAR will be described with reference to FIGS. 2A to 2C. FIG. 2A is a diagram showing an example of road image data obtained from LIDAR observation values. FIG. 2B is a diagram showing map information of the road shown in FIG. 2A. FIG. 2C is a diagram showing a result of comparing the image data shown in FIG. 2A and the map information shown in FIG. 2B.

As shown in FIG. 2A, LIDAR can detect road edges Er and road markings such as road boundary lines (central line Lc, roadway outer line Ls) indicated on the road surface. By comparing this information with the roadway boundary lines and the like in the map information shown in FIG. 2B, the relationship between the image data obtained from LIDAR and the map information can be estimated as shown in FIG. 2C. In the example shown in FIG. 2C, in the diagram showing the map information, the inside surrounded by the white line frame matches the image data shown in FIG. 2A. This makes it possible to estimate the position of the area indicated by the image data on the map. Therefore, the lateral position of the vehicle can be estimated from the vehicle position information in the image data obtained in advance.

However, such an estimation method may not accurately estimate the lateral position of the vehicle. An example thereof will be described below with reference to FIGS. 3A to 3C.

FIG. 3A is a photograph showing an example of a road on which vehicles travel. FIG. 3B is a diagram showing map information of the road shown in FIG. 3A. FIG. 3C is a diagram showing road image data obtained from LIDAR observations installed on a vehicle traveling on the road shown in FIG. 3A.

As shown in FIG. 3A, the road may be covered with snow. FIG. 3B shows map information when the road shown in FIG. 3A is not covered with snow. As shown in FIG. 3C, it is recognized that linear patterns formed by snow are formed in the image data obtained from the observation values of the LIDAR installed on the vehicle traveling on the snow-covered road. be done. Also, in FIG. 3C, the roadway outside line is hidden by snow. Therefore, when comparing FIG. 3B and FIG. 3C, the position of the image data of FIG. 3C on the map of FIG. 3B cannot be estimated with high accuracy. As described above, the road conditions can change dynamically depending on the weather and the like, so there is a problem that the above method cannot accurately estimate the position.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a lateral position estimating device and a lateral position estimating method capable of accurately estimating the lateral position of a moving object such as a vehicle. and

Embodiments of the lateral position estimation device and the lateral position estimation method of the present invention will be specifically described below with reference to the drawings.

It should be noted that the embodiments described below are all comprehensive or specific examples. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in independent claims representing the highest concept will be described as arbitrary constituent elements.

Each figure is a schematic diagram and is not necessarily strictly illustrated. Moreover, in each figure, the same code|symbol is attached|subjected about the same component.

(Embodiment 1)
A lateral position estimation device and an estimation method according to Embodiment 1 will be described.

[1-1. Device configuration]
First, the configuration of the lateral position estimation device according to this embodiment will be described with reference to FIG.

FIG. 4 is a block diagram showing an example of the configuration of lateral position estimation apparatus 100 according to the present embodiment.

A lateral position estimating apparatus 100 according to the present embodiment is a device that is mounted on a mobile body such as a vehicle V10 and estimates the lateral position of the mobile body with respect to the running direction. The lateral position estimation device 100 includes an acquisition unit 10 , a reconstruction unit 20 , a collation unit 30 , a position detection unit 40 and a storage unit 50 .

The acquisition unit 10 acquires image data of a road on which a mobile object such as the vehicle V10 is traveling. In the present embodiment, the acquisition unit 10 has a LIDAR mounted on the vehicle V10, and acquires two-dimensional image data from the measured reflectance, which is the LIDAR observation value of the road on which the vehicle V10 is traveling. The acquiring unit 10 acquires, for example, image data of a 32 m×32 m area around the vehicle V10 at 10 Hz. The number of pixels in image data is, for example, 192×192 pixels. More specifically, three-dimensional image data can be acquired by LIDAR, but in the present embodiment, two-dimensional image data at a height of 30 cm from the road surface is cut out from the three-dimensional image data and used. As a result, landmarks common to general roads, such as road boundary lines, road markings, road edges, and curbs, can be represented as road information about 30 cm or less from the road surface using two-dimensional image data. .

The reconstruction unit 20 enhances the edge strength in the edge profile showing the relationship between the horizontal position in the image data acquired by the acquisition unit 10 and the edge strength indicating the luminance change rate in the image data with respect to the horizontal position. to reconstruct the edge profile. By using the edge profile, the lateral characteristics of the image data can be extracted. Thus, edge profiles can be an effective tool for estimating lateral position. In this embodiment, the reconstruction unit 20 generates an edge profile based on the image data acquired by the acquisition unit 10, and reconstructs the edge profile. Edge profiles and their reconstruction are described later.

The position detection unit 40 detects the approximate position of the vehicle V10. As the position detection unit 40, for example, a GNSS (Global Navigation Satellite System)/INS (Inertial Navigation System) composite navigation device can be used. As a result, the three-dimensional position and orientation of the vehicle V10 can be measured at approximately 100 Hz.

Storage unit 50 stores map information used in lateral position estimation device 100 . Storage unit 50 may store other information used in lateral position estimation device 100 . Here, the map information is information indicating road structures and signs. The map information includes, for example, information indicating the structure of the road, such as the position of the edge of the road, the position of the gutter on the road, the position of the guardrail, and the like. In addition, the map information indicates, for example, information indicating the positions of road signs such as center lines, roadway outer lines, crosswalks, and the like indicated on the road. The map information may be generated, for example, by observation using LIDAR or the like when the weather is fine, or may be generated by other methods.

A collation unit 30 collates the reconstructed edge profile with a map edge profile indicating the relationship between the lateral position and the edge strength obtained from the road map information corresponding to the image data acquired by the acquisition unit 10. By doing so, the lateral position of the moving object is calculated. In the present embodiment, based on the approximate position of vehicle V<b>10 detected by position detection unit 40 , matching unit 30 acquires map information corresponding to the approximate position from storage unit 50 . Here, the map information is information on an area sufficiently wider than the area indicated by the image data acquired by the acquisition unit 10 . The matching unit 30 estimates the lateral position of the vehicle V10 by matching the map edge profile obtained from the map information with the edge profile obtained from the image data. More specifically, the deviation between the lateral position of the vehicle V10 obtained by the position detection unit 40 and the lateral position obtained based on the collation result is estimated. The details of the matching method in the matching unit 30 will be described later.

[1-2. Reconstruction method and collation method]
Next, a method for reconstructing an edge profile in the reconstruction unit 20 and a matching method in the matching unit 30 will be described. First, examples of edge profiles to be reconstructed by the reconstruction unit 20 will be described with reference to FIGS. 5 and 6. FIG. FIG. 5 is a diagram showing an example of the road on which the vehicle V10 travels and the position estimation result on the road. FIG. 6 is a diagram showing another example of the road on which the vehicle V10 travels and the position estimation result on the road. FIG. 5 shows an example of position estimation results on a road in fine weather in summer. On the other hand, FIG. 6 shows an example of position estimation results for the same road and lateral position as in FIG. 5 during snowfall in winter. Each photograph (a) in FIGS. 5 and 6 is a photograph showing the appearance of the road on which position estimation was performed. Each graph (b) in FIGS. 5 and 6 shows the map edge profile obtained from the map information of the road on which position estimation was performed. Each graph (c) in FIGS. 5 and 6 shows an edge profile obtained from image data (that is, an image obtained by LIDAR) obtained by the obtaining unit 10 . Each graph (d) in FIGS. 5 and 6 is a profile showing the degree of similarity obtained by comparing the map edge profile of each graph (a) and the edge profile of each graph (b). Details of the degree of similarity will be described later. Each graph (e) in FIGS. 5 and 6 is a graph obtained by superimposing the map edge profile of each graph (b) and the edge profile of each graph (c) based on the similarity of each graph (d). . The solid line in each graph (e) indicates each map edge profile, and the dotted line indicates the edge profile. Each image (f) in FIGS. 5 and 6 is an image in which each map information and the image data acquired by the acquisition unit 10 are superimposed based on the similarity of each graph (d). In each image (f), image data surrounded by a white frame is superimposed on the map information. The vertical axis of each graph (b) to (e) in FIGS. 5 and 6 indicates the horizontal position, and the horizontal axis of each graph (b) to (c) and (e) indicates the edge strength. Note that the horizontal axis of each graph (d) indicates the degree of similarity R _Edge , which will be described later.

The edge profile shown in each graph (c) of FIGS. 5 and 6 indicates the relationship between the lateral position and the edge strength obtained from the image data acquired by the acquisition unit 10, as described above. Specifically, the edge profile can be detected using, for example, a Sobel filter in the image data acquired by the acquisition unit 10 . By applying the Sobel filter, edge strength E(u, v) and direction θ _E (u, v) are represented by the following equations (1) and (2).

where E _U and E _V are the gradient components in the u and v directions, respectively. In this embodiment, only gradients (rates of change) steeper than 20 degrees in each direction are extracted as edges. As a result, edge profiles as shown in graphs (c) of FIGS. 5 and 6 can be generated. In this embodiment, the number of pixels of the edge profile is 192 at the lateral position of the vehicle V10 with respect to the road (the vertical position in each graph of FIGS. 5 and 6). Therefore, the number of pixels on the vertical axis of the graphs (c) in FIGS. 5 and 6 is 192. FIG.

Further, when the traveling direction (Longitudinal Direction) and the lateral direction (Lateral Direction) of the vehicle V10 are the u direction and the v direction, respectively, the map edge profile Ed _Map ( v) and the edge profile Ed _Obs (v) as shown in graphs (c) of FIGS. 5 and 6 are matched using equation (3) below.

where η is a normalization constant, v _c is the scanning step, and V is the number of pixels in the edge profile Ed _Obs (v). The position on the map where the similarity R _Edge represented by Equation (3) is maximized is estimated to be the position on the map of the image data. In the examples shown in FIGS. 5 and 6, the number of pixels in the horizontal direction of the map edge profile (the vertical axis direction of each graph (b) in FIGS. 5 and 6) is 256, so the matching position candidate is There are 64 (=256-192) points (see graphs (d) in FIGS. 5 and 6). Among these candidate points, the point with the highest similarity R _Edge is presumed to be the point where the edge profile and the map edge profile match (circle mark points shown in graphs (d) of FIGS. 5 and 6). be done.

For example, as shown in the image (f) of FIG. 5, the horizontal relative position of the image data acquired by the acquisition unit 10 with respect to the map information can be accurately determined based on the similarity R _Edge represented by Equation (3). can be estimated to Thus, by using the edge profile, it is possible to extract the characteristics of the image data in the horizontal direction. Edge profiles are an effective method for estimating lateral position, as they allow the extraction of the position of road edges, the positions of road signs such as carriageway boundaries, and the strength of their corresponding edges. can be a tool.

However, the image data acquired by the acquisition unit 10 may fluctuate according to environmental conditions such as weather and conditions such as road construction. For example, as shown in FIG. 6, when the road is covered with snow, the edge profile obtained from the image data also changes.

Comparing the graph (c) of FIG. 5 showing the edge profile during fine weather in summer with the graph (c) of FIG. 6 showing the edge profile during snowfall in winter, graph (c) of FIG. While there are about three portions with high values, graph (c) in FIG. 6 has only one portion with a high peak value. This is probably because the edge of the edge profile corresponding to the line outside the roadway is obscured because the line outside the roadway is hidden by the snow when it snows.

When lateral position estimation is performed by calculating the similarity between the edge profile shown in graph (c) in FIG. 6 and the map edge profile shown in graph (b), graph (e) and image (f) in FIG. , the relative position of the image data to the map information is estimated. Graph (c) showing the edge profile in FIG. 6 was obtained at the same position as graph (c) in FIG. , should be superimposed at the same position as the image data surrounded by the white frame in the image (f) of FIG. However, due to the effect of snow on the image data and edge profile, the position of the image data in image (f) of FIG. Not estimated.

Not only in the case of snowfall as described above, but also due to rainfall, road construction, vegetation around the road, etc., even with the lateral position estimation method using the edge profile, the lateral position cannot be estimated correctly. Sometimes.

Therefore, in the reconstruction unit 20 according to the present embodiment, among the edge profiles shown in the graph (c) of FIG. By reducing the edges that are not in , an edge profile close to the map edge profile shown in graph (c) of FIG. 5 is reconstructed.

A reconstruction method by the reconstruction unit 20 according to the present embodiment will be described below.

The reconstruction unit 20 determines a reconstruction algorithm in advance by machine learning. As a result, it is possible to easily improve the accuracy of lateral position estimation by sufficiently preparing training data used in machine learning. In this embodiment, PCA (Principal Component Analysis) is used in machine learning. Note that although PCA is used in machine learning in this embodiment, the method that can be used in machine learning is not limited to PCA. For example, a neural network or the like, which will be described later, may be used.

The general configuration of PCA is shown below. First _, regarding the raw data Ed _{map_i} (i=1, 2, .

Here, the covariance matrix C of the data whose average value is zero is calculated as shown in the following equation (5).

Also, the eigenvector matrix Ω and the eigenvalue λ of the covariance matrix C are obtained by solving the following equation (6).

As a result, the data Ed _{map_i} is represented by a linear combination of eigenvectors relating to the variable Ed' _{map_i} , as shown in Equation (7) below.

where B _i is a 1×N vector containing the projection values of each eigenvector.

In this embodiment, the edge profiles of various roads are represented by data consisting of a set of M vectors. Edge profiles are extracted from map information to indicate road edges, signs, and the like. The map information includes information on roads on which the lateral position estimation device 100 is used and roads having the same configuration as the roads. The map information is acquired, for example, by a device similar to LIDAR provided in lateral position estimation device 100 according to the present embodiment.

The diagonal entries of the covariance matrix C described above indicate the change of each of the M pixels of the edge profile. Also, the non-diagonal components of the covariance matrix C indicate correlations between different pixels in the entire edge profile.

One of the key properties of PCA is that it implicitly models the structure of data. The eigenvectors of the covariance matrix indicate the direction of the edge profile that varies the most. Such variations are due to variations in edge peak placement, intensity, and road structure characteristics. Thus, each eigenvector is assumed to exhibit different patterns of these properties according to the corresponding eigenvalues.

An example of the eigenvectors described above will be described with reference to FIG. FIG. 7 is a graph showing the behavior of eigenvectors according to this embodiment. FIG. 7 shows the behavior of five eigenvectors with large eigenvalues from −3λi to + _3λi . The vertical axis of each graph in FIG. 7 indicates the lateral position on the road, and the horizontal axis indicates the strength of each element of the eigenvector.

The first (1st) eigenvector shown in the top graph (a) of FIG. 7 encodes the change in placement of multiple peaks in the edge profile. Over the entire range shown in FIG. 7, the peaks shift equally by two pixels each. Here, 2 pixels corresponds to 25 cm in real space. As shown in graph (a) of FIG. 7, the plurality of peaks correspond to the road edge Er, the center line Lc, the roadway outer line Ls, and the like. The second (2nd) eigenvector shown in graph (b) on the second row from the top of FIG. 7 encodes a change in intensity approximately proportional to the peak of the road boundary line. The third (3rd) eigenvector shown in the third row graph (c) of FIG. 7 encodes the intensity relationship between the peak corresponding to the centerline of the road and the peaks at the edges of the road. . The fourth (4th) eigenvector shown in graph (d) in the fourth row from the top of FIG. 7 indicates the relationship between the strengths of the two roadway outside lines. The fifth (5th) eigenvector shown in graph (e) at the bottom of FIG. 7 indicates the range in which the central peak varies.

Next, specific procedures for reconstructing the edge profile will be described with reference to FIGS. 8A to 8C. FIG. 8A is a graph showing placement of first and second eigenvectors of a map edge profile obtained from map information. FIG. 8B is a graph showing an example of a profile with out-of-range peaks. FIG. 8C is a graph showing a profile obtained by reconstructing the profile shown in FIG. 8B.

As shown in FIG. 8A, the distribution of eigenvectors is generally concentrated within the region enclosed by the dash-dotted ellipse. This elliptical region is believed to be the platform for reconstructing the canonical profile based on the pattern encoded in each eigenvector. For example, the profile shown in FIG. 8B has an unusual negative peak. The profile shown in FIG. 8B corresponds to the points indicated by the triangles in FIG. 8A. The position of the profile in eigenspace is determined by Equation (8) below.

The vector B in equation (8) above contains the projection values onto the first eigenvector and the second eigenvector. The projection of the triangle points shown in FIG. 8A onto the first eigenvector lies inside the ellipse, whereas the projection onto the second eigenvector lies outside the ellipse. As described above, the second eigenvector encodes a change in intensity, so that the values projected onto the second eigenvector lie outside the ellipse, suggesting that there is a deformation in the intensity of the profile. Therefore, the points marked with triangles are projected onto the points marked with white circles on the ellipse in FIG. 8A. The eigenvectors thus projected are used in the reconstruction. That is, by using the projected eigenvectors in the above equation (7), a reconstructed profile as shown in FIG. 8C can be obtained.

As described above, the projection value of the second eigenvector is modified based on the second eigenvalue. This shifts the triangular points outside the ellipse in FIG. 8A to the points indicated by the white circles on the ellipse. The new projection values of the points marked with open circles for the second eigenvector are then combined into vector B _new and reconstructed the edge profile by applying equation (9) below, which is similar to equation (7) above. used for

Here, Ed _{reconstructed} represents the reconstructed edge profile.

All eigenvectors are projected into the eigenspace, and eigenvectors outside a predetermined range are projected into the predetermined range. This allows non-standard profiles to be reconfigured into standard profiles.

The eigenvectors are fixed, but the edge profile projection values are modified or changed according to the boundaries of the ellipse. Here the ellipses in the two-dimensional plane represent the eigenvalues of the first eigenvector and the second eigenvector.

In lateral position estimation apparatus 100 according to the present embodiment, 340 pieces of image data with 192×192 pixels for a 64 m×64 m region are used as training data for determining a reconstruction algorithm. Also, as described above, the LIDAR of the acquisition unit 10 acquires image data of 192×192 pixels, so the edge profile is a 1×192 vector. On the other hand, the map edge profile of the map information held by the storage unit 50 is a 1×256 vector. Matching these vectors gives the best match lateral position.

Also, the dominant eigenvectors may represent the most important properties and relationships between profile peaks obtained from the training data. Therefore, it was found that only the 14 main eigenvectors are sufficient to reconstruct the edge profile obtained by LIDAR. The number of eigenvectors required for reconstruction will be described with reference to FIG. FIG. 9 is a graph showing the relationship between the eigenvectors used in the reconstruction according to the present embodiment and the magnitude of change in profile caused by encoding using the eigenvectors. As shown in FIG. 9, the first 14 eigenvectors alone can produce about 75% of the total change.

[1-3. Lateral position estimation method]
Next, the lateral position estimation method used in lateral position estimation apparatus 100 according to the present embodiment will be described with reference to an operation example. FIG. 10 is a flow chart showing a lateral position estimation method according to this embodiment. FIG. 11 is a diagram showing an operation example of lateral position estimation apparatus 100 according to the present embodiment. FIG. 11 shows an operation example on the snow-covered road described above with reference to FIGS. Image (a) in FIG. 11 is an image showing map information. Graph (b) in FIG. 11 is a map edge profile generated from the map information using the above equations (1) and (2). Graph (c) in FIG. 11 shows the offset obtained by inputting the map edge profile shown in graph (b) and the edge profile before reconstruction shown in graph (f) into the above equation (3). FIG. 10 is a graph in which the pre-reconstruction edge profile shown in graph (f) is overlaid on the map edge profile shown in graph (b) based on the quantity; In graph (c), the map edge profile and edge profile are indicated by solid and dotted lines, respectively. Image (d) in FIG. 11 is a diagram in which image (a) and image (e) showing map information are superimposed based on the collation result shown in graph (c). An image (e) in FIG. 11 is an image showing image data acquired by the acquisition unit 10 . Graph (f) of FIG. 11 shows an edge profile generated from the image data shown in image (e) using equations (1) and (2) above. Graph (g) of FIG. 11 is the reconstructed edge profile obtained by equation (9) above. Graph (h) in FIG. 11 is a graph obtained by superimposing the edge profile before reconstruction shown in graph (f) and the reconstructed edge profile shown in graph (g). Graph (i) in FIG. 11 shows the offset obtained by inputting the map edge profile shown in graph (b) and the reconstructed edge profile shown in graph (g) into the above equation (3). FIG. 10 is a graph in which the pre-reconstruction edge profile shown in graph (f) is overlaid on the map edge profile shown in graph (b) based on the quantity; Here, graphs (c) and (i) in FIG. 11 are graphs obtained by superimposing graph (b) and graph (f), but the edge profiles input to the above equation (3) are different. Therefore, the offset amount is different. Graph (f) is superimposed at a more correct position in graph (i) than in graph (c).

Image (j) in FIG. 11 is a diagram obtained by superimposing image (a) and image (e) showing map information based on the matching result shown in graph (i). Since the data image cannot be restored based on the graph (g) showing the edge profile after reconstruction, the image (e), which is the original data image, is used as it is for the image (j). As with graph (i), the graph (g) after reconstruction is used as the input used in equation (3) above, not the graph (f) before reconstruction, so the position of image (e) is is slightly different from the position in image (d). In the image (j), the image (e) is overlaid in a more correct position than the image (d).

As shown in FIG. 10, the acquisition unit 10 of the lateral position estimation device 100 acquires image data of the road on which the vehicle V10 is traveling, captured by the vehicle V10 (S10). For example, an image such as that shown in image (e) of FIG. 11 is obtained from the LIDAR observation values. Note that the image (e) in FIG. 11 is the same image data as the image (a) in FIG.

Subsequently, the reconstruction unit 20 acquires an edge profile indicating the relationship between the lateral position and the edge strength obtained from the image data acquired by the acquisition unit 10 (S20). The reconstruction unit 20 generates an edge profile as shown in the graph (f) of FIG. 11 from the image data as shown in the image (e) of FIG.

Subsequently, the edge profile is reconstructed by emphasizing the edge intensity in the edge profile (S30). Here, the edge profile as shown in graph (g) of FIG. 11 is obtained by reconstructing the edge profile by the reconstruction method described above. Graph (h) in FIG. 11 is a graph in which the edge profile before reconstruction (dotted line) and the edge profile after reconstruction (solid line) are superimposed. As shown in graph (h) of FIG. 11, the reconstruction emphasizes the strength of the edge profile, such as edges at lateral locations corresponding to the carriageway boundaries of the road. At the same time, edges not found in the map information due to snow or the like are reduced.

Subsequently, the matching unit 30 acquires a map edge profile indicating the relationship between the lateral position and the edge strength obtained from the road map information corresponding to the image data acquired by the acquiring unit 10 (S40). In this embodiment, the matching unit 30 acquires map information as shown in the image (a) of FIG. 11 from the storage unit 50 and generates a map edge profile as shown in the graph (b) of FIG. do.

Subsequently, the collating unit 30 collates the reconstructed edge profile and the map edge profile (S50). Specifically, the collating unit 30 collates the map edge profile as shown in graph (b) of FIG. 11 and the reconstructed edge profile as shown in graph (g). The above formula (3) is used in the matching according to the present embodiment. As a result, the positional relationship between the reconstructed edge profile and the map edge profile is shown in graph (i) of FIG.

Subsequently, the collation unit 30 calculates the lateral position of the vehicle V10 based on the collation result (S60). Image (j) in FIG. 11 is an image showing the result of matching. Image (j) in FIG. 11 shows the position of image (e) in image (a) showing map information. The position within the white line frame of image (j) in FIG. 11 corresponds to image (e). As shown in image (j) in FIG. 11, the position of image (e) is obtained with high accuracy.

On the other hand, when matching is performed using the edge profile before reconstruction as it is, the edge profile and the map edge profile are matched as shown in the graph (c) of FIG. Image (d) in FIG. 11 is an image showing the result of matching. Image (d) in FIG. 11 shows the position of image (e) in image (a) showing map information, like image (j). As shown in image (d) of FIG. 11, when reconstruction is not performed, the position of the image data is shifted downward compared to image (j), indicating that matching is not correct.

As described above, according to the lateral position estimation device and the lateral position estimation method according to the present embodiment, by reconstructing the edge profile, the lateral position can be determined more accurately than when reconstruction is not performed. can be estimated.

Next, another operation example will be described with reference to FIGS. 12 to 14. FIG. 12 to 14 are diagrams showing other operation examples of lateral position estimation apparatus 100 according to the present embodiment. FIGS. 12 to 14 show operation examples similar to FIG. Images (a) in FIGS. 12 to 14 are images showing map information. Graphs (b) in FIGS. 12 to 14 are map edge profiles generated from the map information using the above equations (1) and (2). Graphs (c) in FIGS. 12 to 14 are graphs showing the results of comparing the map edge profile and the edge profile before reconstruction using the above equation (3). That is, each graph (c) is a graph obtained by superimposing the map edge profile of each graph (b) and the edge profile shown in graph (f). Image (d) in FIGS. 12 to 14 is a diagram in which image (a) showing map information and image (e) are superimposed based on the matching result shown in graph (c). Images (e) in FIGS. 12 to 14 are images showing image data acquired by the acquisition unit 10. FIG. Graphs (f) of FIGS. 12-14 show edge profiles generated from the image data shown in image (e) using equations (1) and (2) above. Graph (g) in FIGS. 12-14 is the reconstructed edge profile obtained by equation (9) above. Graph (h) in FIGS. 12 to 14 is a graph obtained by superimposing the edge profile before reconstruction shown in graph (f) and the reconstructed edge profile shown in graph (g). . Graphs (i) in FIGS. 12 to 14 show the map edge profile shown in graph (b) and the pre-reconstruction profile shown in graph (f) based on the matching result using the above equation (3). 4 is a graph superimposed with an edge profile. Image (j) in FIGS. 12 to 14 is a diagram in which image (a) and image (e) showing map information are superimposed based on the matching result using the above formula (3).

The operation example shown in FIG. 12 is an example showing the operation on a road during paving work. As shown in the image (e) of FIG. 12, the boundary line of the roadway is hardly visible because the road is under construction. Even in such road conditions, the lateral position can be accurately estimated by reconstructing the edge profile, as shown in image (j). On the other hand, when the edge profile is not reconstructed, as shown in the image (d) of FIG. 12, the position of the image data is shifted upward compared to the image (j), indicating that the matching is not correct. Recognize.

The example of operation shown in FIG. 13 is an example of operation on a road where the boundary line of the roadway is unclear and the outside of the roadway is covered with snow. In this case, the reflectance of the laser at the roadway boundary line is reduced, and the visibility of the roadway boundary line is reduced as shown in image (e) of FIG. 13 . On the other hand, due to the relatively high reflectivity of snow to the laser, edges formed by snow can dominate in matching. Even in such road conditions, the lateral position can be accurately estimated by reconstructing the edge profile, as shown in image (j). On the other hand, when the edge profile is not reconstructed, as shown in image (d) of FIG. It can be seen that they are not collated correctly.

The operation example shown in FIG. 14 is an example showing the operation on a wet road. In this case, the reflectance of the laser at the boundary line of the roadway is lowered, and as shown in the image (e) of FIG. 14, the position information of the boundary line of the roadway is hardly obtained. Even in such road conditions, the lateral position can be accurately estimated by reconstructing the edge profile, as shown in image (j). On the other hand, when the edge profile is not reconstructed, as shown in the image (d) of FIG. 14, the position of the image data is shifted from the information in the image (j), indicating that the matching is not correct. Recognize.

As described above, according to the lateral position estimation method and the lateral position estimation device 100 according to the present embodiment, edge profiles can be reproduced even if at least part of the information of signs such as road boundary lines is lost. By constructing it, the lateral position can be estimated with high accuracy. Further, according to the present embodiment, even when snow or the like may be misidentified as a road sign or the like, it is possible to accurately estimate the lateral position by reconstructing the edge profile. In other words, in this embodiment, it is possible to realize a lateral position estimation device and a lateral position estimation method that are robust against dynamic changes in road conditions. In addition to the above examples, the edge profile can also change depending on the state of plants on and near the road, for example. The lateral position estimation device and lateral position estimation method according to the embodiments are robust.

According to the results of experiments conducted by the inventors on a large number of roads with different conditions, when reconstruction was not performed, only about 31% could be accurately estimated, whereas reconstruction was performed. In some cases, accurate estimation was possible in about 83% of the experiments.

In order to realize automatic driving of the vehicle V10, it is necessary to ensure safety even when there are dynamic changes in road conditions due to weather or the like. Therefore, the lateral position estimating device and lateral position estimating method according to the present embodiment are particularly effective for the vehicle V10 that is automatically driven. Further, the lateral position estimation method according to the present embodiment can be realized by simple calculations to the extent that the lateral position can be estimated in real time during automatic driving of the vehicle V10.

[1-4. summary]
As described above, the lateral position estimation apparatus 100 according to the present embodiment is a lateral position estimation apparatus that estimates the lateral position of a moving body with respect to the traveling direction, and An acquisition unit 10 that acquires image data captured from the body, and edge strength is emphasized in an edge profile that indicates the relationship between the lateral position in the image data and the edge strength that indicates the rate of change in luminance in the image data with respect to the lateral position. a reconstruction unit 20 for reconstructing an edge profile by performing the reconstruction, the reconstructed edge profile, and a map edge profile showing the relationship between the lateral position and the edge strength obtained from the map information of the road corresponding to the image data. and a collating unit 30 for computing the lateral position of the moving body by collating the above.

As a result, even if the strength of the edge profile obtained from the image data is insufficient for accurate lateral position estimation, the reconstruction unit can emphasize the edge strength in the edge profile obtained from the image data. , the lateral position can be estimated with high accuracy. In other words, in this embodiment, it is possible to realize a lateral position estimation device that is robust against dynamic changes in road conditions. Therefore, by applying the lateral position estimating device according to the present embodiment to a vehicle that automatically drives, for example, it is possible to improve safety in automatic driving. In addition, in order to realize automatic driving of the vehicle V10, it is necessary to ensure safety even when there are dynamic changes in road conditions due to weather or the like. Therefore, the lateral position estimating device and lateral position estimating method according to the present embodiment are particularly effective for the vehicle V10 that is automatically driven. Further, the lateral position estimation method according to the present embodiment can be realized by simple calculations to the extent that the lateral position can be estimated in real time during automatic driving of the vehicle V10.

Further, the lateral position estimation method according to the present embodiment is a lateral position estimation method for estimating the lateral position of a moving body with respect to the running direction, and is a method of estimating a lateral position of a moving body in which a road on which the moving body is running is captured from the moving body. In an acquisition step of acquiring image data and an edge profile showing the relationship between lateral position in the image data and edge strength indicating the rate of change in brightness in the image data with respect to lateral position, the edge profile is obtained by enhancing the edge strength. and a reconstructed edge profile and a map edge profile indicating the relationship between lateral position and edge strength obtained from road map information corresponding to the image data. and a collating step of calculating the lateral position of the moving object.

As a result, it is possible to obtain the same effect as the lateral position estimation device described above.

Further, in the lateral position estimation device 100, the reconstruction unit 20 may determine a reconstruction algorithm in advance by machine learning.

As a result, it is possible to easily improve the accuracy of lateral position estimation by sufficiently preparing training data used in machine learning.

In the lateral position estimation device 100, the reconstruction unit 20 may use PCA in machine learning.

This makes it possible to model the data structure of the map information.

(Embodiment 2)
A lateral position estimation device and a lateral position estimation method according to Embodiment 2 will be described. In the lateral position estimation device and lateral position estimation method according to Embodiment 1, PCA is used in determining the edge profile reconstruction algorithm, but in the present embodiment, a neural network is used. A reconstruction algorithm determination method according to the present embodiment will be described below.

As described above, in this embodiment, a reconstruction algorithm is determined by machine learning using a neural network.

In the reconstruction unit according to the present embodiment, an edge profile obtained from image data that is a plurality of samples acquired by the acquisition unit and a map edge profile obtained from map information corresponding to the image data are input in advance. be done. Note that the image data here may be image data similar to the image data acquired by the acquisition unit, and does not necessarily have to be acquired by the acquisition unit.

When the edge profile is input to the reconstructor, the reconstructor also receives information about the position to which the edge profile corresponds in the map edge profile. Using this information, the reconstructor learns an algorithm that reconstructs the edge profile to a profile closer to the map edge profile. By inputting a sufficiently large number of edge profiles and map edge profiles to the reconstruction unit for learning, an algorithm for reconstructing edge profiles with high accuracy can be determined. That is, according to this embodiment, the lateral position can be estimated with high accuracy. Furthermore, according to the reconstruction unit and the reconstruction method according to the present embodiment, the accuracy of lateral position estimation can be further improved as the amount of machine learning is increased. Therefore, according to the present embodiment, the lateral position can be estimated with higher accuracy than the first embodiment.

(Modified example, etc.)
Although the lateral position estimation device and the lateral position estimation method according to one aspect of the present invention have been described above based on each embodiment, the present invention is not limited to these embodiments. As long as they do not deviate from the spirit of the present invention, various modifications conceived by those skilled in the art may be included in the scope of the present invention.

For example, in each of the embodiments described above, an example in which the acquisition unit 10 acquires image data by LIDAR has been described, but the acquisition unit 10 may acquire image data using a detector other than LIDAR. For example, image data may be acquired using another imaging device such as a camera.

In addition, in the lateral position estimation device and lateral position estimation method according to the above-described embodiments, two-dimensional image data at a height of 30 cm from the road surface can be used as map information and image data, as described above. Such two-dimensional image data can represent common road landmarks such as road boundaries, road markings, road edges, and curbs as road information about 30 cm or less from the road surface. . These landmarks are very common in cities. The edge profile in each of the embodiments described above represents the pattern of these landmarks by peaks of roadway boundaries, road markings, road edges, curbs, and the like. Since the lateral position estimating device and lateral position estimating method according to each of the above embodiments use only road surface information for machine learning and movement, it is possible to learn patterns with different peak distributions and amplitudes. Therefore, it can be used in a city or country different from the city or country in which the study was conducted.

Also, in each of the above embodiments, the edge profile was generated by the reconstruction unit 20, but the edge profile may be generated by a component other than the reconstruction unit 20. FIG. For example, the edge profile may be generated by the acquisition unit 10 .

Further, in each of the above-described embodiments, an example of using the vehicle V10 as an example of a moving object has been described, but the moving object is not limited to the vehicle V10. Any object that can move on the road may be used.

In the above embodiments, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor. Here, the software that implements the lateral position estimation apparatus 100 and the like of the above embodiment is a program that causes a computer to execute each step included in the flowchart shown in FIG.

INDUSTRIAL APPLICABILITY A lateral position estimation device and a lateral position estimation method according to an aspect of the present invention have the effect of accurately estimating the lateral position of a moving body such as a vehicle. can do.

10 acquisition unit 20 reconstruction unit 30 collation unit 40 position detection unit 50 storage unit 100 lateral position estimation device V10 vehicle

Claims (2)

A lateral position estimating device for estimating a lateral position of a moving body with respect to a traveling direction,
an acquisition unit that acquires image data obtained by imaging a road on which the mobile body is traveling, from the mobile body;
In the entire edge profile showing the relationship between the horizontal position in the image data and the edge strength indicating the luminance change rate in the image data with respect to the horizontal position, the edge strength is emphasized to improve the edge profile. a reconstruction unit that performs reconstruction;
By collating the reconstructed edge profile with a map edge profile indicating the relationship between the lateral position and the edge strength obtained from road map information corresponding to the image data, and a collation unit that calculates the lateral position ,
The reconstruction unit predetermines the reconstruction algorithm by machine learning,
The reconstruction unit
Using PCA (Principal Component Analysis) in the machine learning,
Reproducing the edge profile using a first eigenvector encoding a change in placement of peaks in the edge profile and a second eigenvector encoding a change in intensity proportional to a road boundary peak. compose and
In the graph showing the arrangement of the first eigenvector and the second eigenvector, among the eigenvectors representing the edge profile, the eigenvector positioned outside the ellipse representing the eigenvalues of the first eigenvector and the second eigenvector , reconstruct the edge profile by shifting on the ellipse
Lateral position estimator. A lateral position estimation method for estimating a lateral position of a moving body with respect to a running direction, comprising:
an acquisition step of acquiring image data of a road on which the mobile body is traveling, taken from the mobile body;
In the entire edge profile showing the relationship between the horizontal position in the image data and the edge strength indicating the luminance change rate in the image data with respect to the horizontal position, the edge strength is emphasized to improve the edge profile. a reconstruction step for reconstructing;
By collating the reconstructed edge profile with a map edge profile indicating the relationship between the lateral position and the edge strength obtained from road map information corresponding to the image data, a matching step of calculating the lateral position ;
In the reconstruction step, predetermining the reconstruction algorithm by machine learning,
In the machine learning,
Using PCA (Principal Component Analysis),
In the reconstruction step,
Reproducing the edge profile using a first eigenvector encoding a change in placement of peaks in the edge profile and a second eigenvector encoding a change in intensity proportional to a road boundary peak. compose and
In the graph showing the arrangement of the first eigenvector and the second eigenvector, among the eigenvectors representing the edge profile, the eigenvector positioned outside the ellipse representing the eigenvalues of the first eigenvector and the second eigenvector , reconstruct the edge profile by shifting on the ellipse
Lateral position estimation method.

Images