JP7028206B2 - Map generator and program - Google Patents

Description

The present disclosure relates to a map generator and a program.

Conventionally, there has been known a technique of generating a map using a plurality of captured images captured by an imaging device such as a camera mounted on a vehicle. For example, in the techniques described in Patent Documents 1 and 2, line segments such as road lanes (section lines) are detected from a plurality of captured images captured by a camera and used for creating a map.

Japanese Unexamined Patent Publication No. 2017-182564 Japanese Unexamined Patent Publication No. 2017-10393

However, in the above-mentioned conventional technique, a positioning error may occur at the position of a line segment such as a lane, or a line segment that has been erroneously detected may be included. Further, in the above-mentioned conventional technique, an undetected lane may occur. In such a case, the above-mentioned conventional technique cannot detect an error or the like, and since the undetected lane is not interpolated, the line segment at the actual position may not be detected.

The present disclosure has been made in consideration of the above techniques, and an object of the present disclosure is to provide a map generator and a program capable of generating a highly accurate line map.

In order to achieve the above object, the map generator of the first aspect of the present disclosure is a plurality of line segments from each of a plurality of captured images in which the same traveling section is captured by an imaging unit mounted on a vehicle. For each of the line segment detection unit that detects the line segment and the plurality of line segments detected by the line segment detection unit, each line segment is based on the position information of the vehicle associated with each of the plurality of captured images. The projection unit that converts the coordinates representing the positions of both end points into absolute coordinates in the bird's-eye view, and the plurality of line segments detected by the line segment detection unit are made into one series for each of the plurality of captured images, and the end points are described. Based on the position based on the absolute coordinates, the position difference amount between the end points of the line segment of the own series and the end points of the line segment of the other series was calculated from each series of the plurality of captured images . Based on the position difference amount, the line segment corresponding portion that associates the line segments so that the position difference amount becomes smaller between the two series of the own series and the other series, and the line segment correspondence. An end point is provided at a position in the middle of the corresponding end points of the associated line segments based on the correspondence of the line segments between the self-series and the other series associated with each other by the attachment part. A series consisting of an averaging unit that generates a new line segment and a series consisting of the new line segment, a line segment correspondence by the line segment corresponding portion, and the new line segment by the averaging unit. By repeating a series of processes for reconstructing a set of a plurality of series consisting of each of the plurality of new line segments generated by performing the generation of the plurality of series. It is provided with an integration unit that integrates each series of each captured image into one series .

The map generator according to the second aspect of the present disclosure is the map generator according to the first aspect, wherein the integrated unit converges the position of the associated line segment to the center position to form the two sequences. Integrate.

The map generator according to the third aspect of the present disclosure is the map generator according to the first aspect or the second aspect , wherein the line segment detection unit uses the core line of the dividing line as a line segment from each of the plurality of captured images. To detect.

The map generator according to the fourth aspect of the present disclosure is the map generator according to any one of the first to third aspects, wherein the averaging unit has a large position error in the set of the series. When averaging a series of n (n is an integer of 1 or more) and a series other than the n sets, a line segment is generated that is close to the position of the line segment of the series other than the n sets.

The map generator according to the fifth aspect of the present disclosure is the map generator according to any one of the first to fourth aspects, wherein the integrated unit is a plurality of new devices generated by the averaging unit. From a plurality of series consisting of each of the line segments, the higher-order series having high matching of the new line segment is selected, and the set of the selected series is reconstructed.

Further, in order to achieve the above object, the program of the eighth aspect of the present disclosure uses a computer as a plurality of captured images obtained by capturing the same traveling section by an imaging unit mounted on a vehicle. For each of the line segment detection unit that detects the line segment and the plurality of line segments detected by the line segment detection unit, each line is based on the position information of the vehicle associated with each of the plurality of captured images. A projection unit that converts the coordinates representing the positions of both end points of a minute into absolute coordinates in a bird's-eye view, and a plurality of line segments detected by the line segment detection unit are made into one series for each of the plurality of captured images, and the end points are described. Based on the position based on the absolute coordinates, the position difference amount between the end points of the line segment of the own series and the end points of the line segment of the other series was calculated from each series of the plurality of captured images . With a line segment correspondence part that associates line segments so that the position difference amount becomes smaller between the two series of the own series and the other series based on the position difference amount, with the line segment correspondence. An end point is provided at a position in the middle of the corresponding end points of the associated line segments based on the correspondence of the line segments between the self-series and the other series associated with each other by the unit. A averaging unit that generates a new line segment and generates a series consisting of the new line segments , a line segment mapping by the line segment corresponding portion, and a series consisting of the new line segment by the averaging unit. By repeatedly performing a series of processes for reconstructing a set of a plurality of series consisting of each of the plurality of new line segments generated by performing the generation on all the series, the plurality of said. The purpose is to function as an integrated unit that integrates each series of captured images into one series .

According to the present disclosure, it is possible to obtain an effect that a highly accurate line map can be generated.

It is a block diagram which shows an example of the whole structure of the map generation system of an embodiment. It is a figure which shows the outline of an example of the technique of this disclosure. It is a figure which shows an example of the hardware composition of the map generation apparatus of embodiment. It is a flowchart which shows an example of the procedure of the line map generation processing executed by the map generation apparatus. It is a figure which shows an example of the figure (EN coordinate diagram) in which the line segment detected by the line segment detection part from the captured image is plotted in the EN coordinate by the projection part. It is a figure for demonstrating an example which determines a set of feature points that overlap by translation as an inlier. It is a figure for demonstrating the correspondence of the line segment by the line segment correspondence part. It is a figure for demonstrating the interpolation of the translation amount by the line segment correspondence part. It is a figure for demonstrating the averaging process of the line segment by the averaging part. It is a figure for demonstrating the integration of a plurality of series by an integration part. It is a figure which shows an example of the state in which two series of line segments are associated with each other by the line segment correspondence part. It is a figure which shows an example of the integration result of the line segment of the series shown in FIG. 11A by the integration part. It is a figure which shows an example of the integration result when the whole line segment of each series shown in FIG. 11A is translated in parallel. It is a projection drawing which shows an example of the line segment of the series P. It is a projection drawing which shows an example of the line segment of the series Q. It is a figure which shows an example of the state in which the two series of line segments of FIG. 12A and FIG. 12B are associated with each other by the line segment correspondence part. It is a figure which shows an example of the integration result of the line segment of the series shown in FIG. 12C by the integration part.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, the overall configuration of the map generation system of the present embodiment will be described.
FIG. 1 is a block diagram showing an example of the overall configuration of the map generation system 1 of the present embodiment. The map generation system 1 of the present embodiment is a map generation system that generates a map from an image captured by an image pickup unit 12 mounted on a vehicle 2.

As shown in FIG. 1, the map generation system 1 includes a map generation device 10, an image pickup unit 12, a positioning unit 14, an image pickup image database 15, and a map database 18.

The image pickup unit 12 and the positioning unit 14 are mounted on the vehicle 2. Specifically, the image pickup unit 12 is a camera or the like, and takes a moving image or a still image at a predetermined timing with the road in front of the vehicle 2 as a subject. The positioning unit 14 has a function of measuring the current position of the vehicle 2, and examples thereof include a GPS (Global Positioning System) device.

The vehicle 2 outputs to the captured image database 15 in a state where the captured image captured by the imaging unit 12 is associated with the position of the vehicle 2, which is the positioning result of the positioning unit 14 when the captured image is captured. Will be done.

In the map generation system 1 of the present embodiment, a plurality of captured images captured by the imaging unit 12 mounted on the vehicle 2 traveling in the traveling section for the same traveling section are obtained by the positioning unit 14 of the vehicle 2. It is stored in the captured image database 15 in a state associated with the position of. In other words, the captured image database 15 stores a plurality of captured images captured in the same traveling section in a state associated with the captured position.

Although only one vehicle 2 is shown in FIG. 1, one vehicle 2 travels in the traveling section a plurality of times, and each time the vehicle 2 travels, an image captured by the image pickup unit 12 is acquired. It may be in the form of acquiring an image captured by the image pickup unit 12 when a plurality of vehicles 2 travel in the same traveling section. In the outline of an example of the technology of the present disclosure shown in FIG. ₂ , each of the _four vehicles of the vehicles 21 to 24 is equipped with the image pickup unit 12 and the positioning unit 14, and when the vehicle travels on the road R, the image pickup unit 12 and the positioning unit 14 are mounted. , The form in which the road R is imaged is shown. In this case, the captured image database 15 stores the captured image of the road R in a state in which the positions of the vehicles 21 to ₂₄ corresponding to the road R are associated with _each other as the shooting position.

The map generation device 10 includes a line segment detection unit 20, a projection unit 22, a line segment corresponding attachment unit 24, an averaging unit 26, and an integration unit 28.

FIG. 3 is a diagram showing an example of the hardware configuration of the map generator 10. As shown in FIG. 3, the map generator 10 is configured as a computer that performs various controls and various operations. That is, the map generator 10 includes a CPU (Central Processing Unit) 30A, a ROM (Read Only Memory) 30B, a RAM (Random Access Memory) 30C, a non-volatile storage unit 30D, an input / output interface (I / O) 30E, and communication. The unit 31 is provided. Each of the CPU 30A, ROM 30B, RAM 30C, non-volatile storage unit 30D, and I / O 30E is connected via the bus 30F.

The communication unit 31 is connected to the I / O 30E. The communication unit 31 is an interface for communicating with an external device such as a captured image database 15 or a map database 18 via a communication line.

In the map generation device 10 of the present embodiment, the ROM 30B stores a program for executing the "line map generation process" described later. The CPU 30A reads out the program stored in the ROM 30B and executes the program using the RAM 30C as a work area. When the CPU 30A executes the program stored in the ROM 30B, the CPU 30A functions as each of the line segment detection unit 20, the projection unit 22, the line segment corresponding unit 24, the averaging unit 26, and the integration unit 28.

The line segment detection unit 20 detects a set of feature points as a line segment from each of the plurality of captured images acquired from the captured image database 15. The line segment detection unit 20 detects a plurality of line segments including a section line segment of the road R from the captured image. The line segment detection unit 20 outputs the line segment detection result and the captured image associated with the imaging position to the projection unit 22.

The projection unit 22 converts the coordinates representing the position of each line segment into absolute coordinates for each of the plurality of line segments detected by the line segment detection unit 20 based on the imaging position of each captured image. Hereinafter, for convenience of explanation, an image obtained by converting a plurality of line segments detected from each captured image into absolute coordinates is referred to as a “projected image”. In the example shown in FIG. 2, the projected image _{501 including the line segment 91 1 representing} the lane marking ₉₀ is obtained from the captured image captured by the vehicle 211. Similarly, a projected image ₅₀₂ including a line segment 912 representing a line segment 90 is obtained from the captured image captured by the vehicle ₂₁₁ , and a _lane marking 90 is obtained from the captured image captured by the vehicle ₂₁₃ . A projected image ₅₀₃ including the line segment 913 representing the line segment ₉₁₃ is obtained _, and a projected image ₅₀₄ including the line segment 914 representing the line segment 90 is obtained from the captured image captured by the vehicle ₂₁₄ .

In the following, a plurality of line segments detected for each projected image (captured image) will be treated as one series.

In the line segment corresponding portion 24, for each series of line segments, the difference in position between the end points of the line segments of the other series and the end points of the line segments of the own series is the smallest in all of the plurality of line segments in each series. The line segments are associated with each other as described above. The line segment correspondence unit 24 outputs the correspondence result for each line segment to the averaging unit 26.

The averaging unit 26 generates a new line segment at a position in the middle of the line segment for each of the line segments of the two series associated with the line segment corresponding unit 24. In the example shown in _FIG . ₂ , the line segment sequence in the projected image 501 and the line segment sequence in the projected image 501 are associated with the line segment by the line segment corresponding unit 24, and the averaging unit 26 is used. The line segment 92 ₁₂ is generated from the line segment 91 ₁ included in the line segment series in the projected image 501 and _the line segment 91 ₂ included in the line segment series in the projected image ₅₀₁ . .. Further, the line segment sequence in the projected image 503 and the line segment sequence in the projected image _{504 are} associated with the line segment by the line segment corresponding portion 24 _, and the projected image 503 is performed by the averaging unit 26. The state in which the line segment 92 ₃₄ is generated from the line segment ₉₁₃ included in the line segment series in the above and the line segment 914 included in the line segment series in _the projected image ₅₀₄ is shown.

The integration unit 28 integrates a plurality of line segments associated with each other among the plurality of series based on the line segments generated by the averaging unit 26. Specifically, the integration unit 28 repeatedly performs each process of the line segment corresponding unit 24 and the averaging unit 26, and converges the position of each line segment. In the example shown in FIG. 2, the integrated unit 28 repeats each process of the line segment corresponding unit 24 and the averaging unit 26 for the series including the line segment 92 ₁₂ and the series including the line segment 92 ₃₄ . , The line segment 92 ₁₂ and the line segment 92 ₃₄ are converged to the position of the line segment 94 ₁₂₃₄ . In addition, in the integrated unit 28 in the present embodiment, when each process of the line segment corresponding unit 24 and the averaging unit 26 is repeated for a plurality of series, the averaging unit 26 produces a series having high line segment sound matching. By sorting, a set consisting of multiple series is reconstructed (details will be described later).

The information representing the position of the line segment converged by the integration unit 28 is output to the map database 18. The map generation device 10 repeats the processes of the line segment detection unit 20 to the integration unit 28 for each of the plurality of traveling sections, and outputs information indicating the position of each line segment to the map database 18. In the map generation system 1 of the present embodiment, a line map can be obtained by connecting information indicating the position of each line segment for each traveling section according to the position of the traveling section.

Next, the operation of the map generation device 10 of the present embodiment will be described in detail. When the CPU 30A executes the program stored in the ROM 30B, the line map generation process shown in FIG. 4 is executed. FIG. 4 shows a flowchart showing an example of the flow of the line map generation process executed by the map generation device 10 of the present embodiment. As an example, in the map generation device 10 of the present embodiment, FIG. 4 is taken at each predetermined timing and at each timing when a predetermined number of captured images are stored in the captured image database 15 for each traveling section. The line map generation process shown in 1 is executed.

In step S100, the line segment detection unit 20 detects a line segment from the captured image acquired from the captured image database 15. The method by which the line segment detection unit 20 detects a line segment from the captured image is not particularly limited, but as an example, the line segment detection unit 20 of the present embodiment captures two line segments representing the core wire of the dividing line from the captured image. Detect by digitizing, distance transforming, and Hough transforming. The line segment detection unit 20 projects, with respect to each row of the captured image obtained from the distance conversion, a peak portion in which the distance value is larger than the distance value in the left and right pixels into the huff space. The upper and lower end points of the line segment to be detected are obtained from the upper limit and the lower limit where the distance value is not 0 (zero).

The line segment detection unit 20 of the present embodiment performs this processing and the following processing on a plurality of captured images captured in the same traveling section.

In the next step S102, the projection unit 22 projects the line segment detected in the step S100 to obtain the absolute coordinates in the bird's-eye view. As an example, the projection unit 22 of the present embodiment converts the detected line segment into EN (East-North) coordinates such that the right side is east and the top side is north. The coordinates of the end points of each line segment are converted by the conversion formulas (1) to (4) below.

Specifically, when it is assumed that the image pickup surface of the image pickup image is substantially perpendicular to the road surface of the road R, the conversion between the coordinates of the image pickup image and the center coordinates of the image pickup unit 12 is performed in the equation (1) and (1). 2) Eq.
f: (vv ₀ ) = z: y ... (1)
f: (u-u ₀ ) = z: x ... (2)

In the above equations (1) and (2), (u, v) are the coordinates [pix] in the captured image, and (x, z) are the center coordinates of the imaging unit 12 and x [in the right-hand direction. m] and the traveling direction z [m]. Further, f is the focal length [pix] of the image pickup unit 12, y is the height [m] of the position where the image pickup unit 12 is installed in the vehicle 2, and (u ₀ , v ₀ ) is. The coordinates [pix] of the vanishing point in the captured image.

Further, the following equations (3) and (4) are used for the conversion from the center coordinates of the imaging unit 12 to the EN coordinates.
e = e ₀ + (x + Δx) cosθ + (z + Δz) sinθ ・ ・ ・ (3)
n = n ₀ + (x + Δx) sinθ + (z + Δz) cosθ ... (4)

In the above equations (3) and (4), (e, n) is the EN coordinate [m], and (e ₀ , n ₀ ) is the reference position of the vehicle 2 obtained by the positioning unit 14. EN coordinate [m]. Further, (Δx, Δz) is the relative position [m] of the image pickup unit 12 with respect to the reference position of the vehicle 2, and θ is the azimuth angle of the vehicle 2 (north is 0 and clockwise is positive) [rad). ].

FIG. 5 shows an example of a diagram (EN coordinate diagram) in which a line segment detected by the line segment detection unit 20 from an image captured by the image pickup unit 12 of the vehicle 2 is plotted on EN coordinates by the projection unit 22. .. As illustrated in FIG. 5, line segments and the like of unnecessary objects other than section lines are also detected afterwards.

In the next step S104, the line segment corresponding unit 24 applies the spectral matching of the following reference 1 to the end points of each line segment converted into EN coordinates in the above step S102, and applies one body 1 for a plurality of series of line segments. Derivation of the mapping of.
[Reference 1] M. Leordeanu and M. Hebert, “A spectral technique for correspondence problems using pairwise constraints,” Proc. 10th IEEE Int. Conf. On Computer Vision, vol. 2, pp. 1482-1489, Beijing, China , Oct. 2005.

Here, as a reference, consider the problem of associating a plurality of feature points included in the series P with a plurality of feature points included in the series Q. Hereinafter, the number of feature points included in the series P is | P |, and the number of feature points included in the series Q is | Q |. Two feature points are extracted from each of the series P and the series Q, and are expressed as p _i1 , p _j1 ∈ P, and q _i2 , q _j2 ∈ Q, respectively. When the correspondence between the points p _i1 and the point q _i2 also holds between the points p _j1 and the point q _j2 , (i1, i2) and (j1, j2) are the set constituting the inlier. I reckon. FIG. 6 shows an example of determining a set of feature points that overlap due to translation as an inlier.

While (i1, i2) and (j1, j2) are overlapping sets with the same translation amount, the set of (j1, k2) including another point q _k2 ∈ Q does not satisfy this. In order to express this event, a matrix M of the number of elements | P || Q | × | P || Q | is defined, and the elements M ((i1, i2), (j1, j2)) corresponding to the above are defined. Set "1" and "0" for other pairs. After setting all the elements, the following processing is used to obtain the optimum and consistent inter-element mapping result as a whole.

First, the eigenvector x corresponding to the first principal component of the matrix M is obtained by the power method. Then, x of the number of elements | P | | Q | × 1 is reorganized into the matrix X of the number of elements | P | × | Q |. Furthermore, the Hungarian method is applied to obtain a one-to-one correspondence. For example, in the example shown in FIG. 6, the set of inliers (i1, i2); (j1, j2) can be identified by the elements of the matrix X reorganized from the matrix M.

Next, consider the problem of associating the line segments in the sequence P in one projected image and the sequence Q in another projected image obtained in step S102. The difference from the method in Reference 1 is that the unit of collation is replaced with a line segment from a point, and the value stored in the matrix M is replaced with the end point position difference of the line segment. The line segment is represented by two points at both ends whose elements are EN coordinates. As shown in the example shown in FIG. 7, two line segments are selected from each run, and p _i1 , _p'i1 , p _j1 , _p'j1 ∈ P, q _i2 , _q'i2 , q _j2 , _q'j2 ∈ Q. It is expressed as. The position difference amount v _{i1 and i2} corresponding to the set (i1 and i2) are derived by the following equation (5).

Subsequently, the values representing the inliar-likeness of the sets (i1, i2) and (j1, j2) are calculated based on the endpoint position difference after applying the position difference amount obtained by the above equation (5), and the matrix M Store in. The matrix M is represented by the following equation (6).

This value becomes "1" when the end points of both line segments completely match after applying the position difference amount, and becomes closer to "0" as the distance increases. Λ in the above equation (6) is. It is a weight for suppressing a large amount of parallel movement, and is defined by the following equation (7) in consideration of positioning accuracy.

The line segments included in the series P or the series Q are selected as follows. Among the plurality of line segments in each of the projected images obtained in steps S100 and S102, the certainty that the line segments in which the positions of both end points match (for example, the distance is less than 0.5 m) exists in the plurality of frames. It is set to 1 and is used as a collation target. The other line segments have a certainty of 0 and are not used in this processing (with line segment correspondence). The line segment not used in this process is also used in the subsequent process (process of S108).

That is, a set in which the difference between the positions at both ends of the line segment after applying the amount of position difference is small is an inlier. As a result, the points on which the line segments are aligned are extracted by associating the points on the body surface.

In the next step S106, the line segment corresponding unit 24 interpolates the movement amount of the line segment. First, it is assumed that the position difference amount in the above equation (5) is calculated for the set of line segments determined to be inliers. If the series P and the series Q match due to global translation, the position difference amount of the line segment set corresponding to the maximum element of the eigenvector x can be regarded as the position difference amount between the series P and Q. However, in reality, local distortion occurs due to the influence of the trajectory error of the positioning unit 14. Therefore, from the one-to-one correspondence line segment sets obtained by the line segment correspondence attachment unit 24, the upper n sets having a large element of the eigenvector x are used. By this process, an appropriate value of n is obtained. In the subsequent processing, the series can be superposed by linearly interpolating the position difference amount of the line segments existing in the middle of the control points of the upper n sets of line segments.

FIG. 5 shows an example of interpolation by the line segment corresponding portion 24. The pairs (i1, i2) and (k1, k2) are inliers corresponding to the upper n pairs. The position difference amount v _{j1 of the line segment p j1 p'j1 ∈} P located in the middle of them along the traveling direction is the position difference amount of the front and rear line segments p _{i1 p'i1} and p _{k1 p'k1} ∈ P. Is derived from the following equations (8) and (9).

If the control point exists on only one side, the nearest neighbor position difference amount is applied as it is without extrapolation. For example, the upper limit of the parameter n is set to 10, and the parameter n is determined so that the position difference of the entire line segment between the series P and Q is the smallest in that range. The position difference of the entire line segment is derived by the following equation (10).

In the next step S108, the averaging unit 26 averages each line segment. In the processing of this step, all line segments are targeted for processing regardless of the above-mentioned certainty value. When the parameter n is determined and the set of correspondences used for integration is determined, the translation process is applied to all the line segments of the series P and the series Q based on the position difference amount derived by the line segment corresponding attachment unit 24.

The interpolation formulas of the above equations (8) and (9) are used to calculate the position difference amount. The translation amount t _j1 of the line segment j1 of the series P is derived by the following equation (11), and the translation amount t _j2 of the line segment j2 of the series Q is derived by the following equation (12). As shown in the following equations (11) and (12), the position difference amount v ^* of the set that gives the maximum element of the eigenvector x derived by the line segment corresponding unit 24 is used for deriving the translation amount. .. Unless otherwise specified, the coefficient k is 1/2 (k = 1/2). This is because the position translated by v ^* / 2 can be regarded as the most reliable intermediate position between the series P and Q. The details will be described later, but if any of the series is unreliable (the position is out of alignment), k = 1/2 is not set, and the value of the coefficient k is adjusted to a value that matches the reliable series.

The averaging unit 26 averages the input series P and Q based on the result of the line segment corresponding unit 24, and newly generates the averaging series R. As an example, in FIG. 9, the line segment p _{j1 p'j1} of the sequence P and the line segment q _{j2 q'j2} of the sequence R are averaged to generate the line segments r _{j1, j2 r'j1, j2 of the} sequence R. An example is shown.

When the line segment of the series R is expressed as r _{j1, j2 r'j1, j2} and the certainty is expressed as c (r _{j1, j2} ), the positions r _{j1 and j2} of the end points of the line segment of the series R are as follows (13). ), The distance d between both ends is expressed by the following equation (14), and the certainty c (r _{j1, j2} ) is expressed by the following equation (15). In the following equations (13) to (15), the certainty of the line segment p _{j1 p'j1} is c (p _j1 ), and the certainty of the line segment q _{j2 q'j2} is c (q _j2 ).

When multiple line segments of other series are targeted for one line segment, the line segment of the output series R r _{j1, j2 r'j1} for the set of line segments having the maximum certainty c (r _{j1 , j2} ). _{, J2} is generated.

For the line segments with a certainty of 1 in the series P and Q that did not generate the line segment of the series R with a certainty of 1, the line segment after the translation application is registered in the series R with the certainty of 1. do. For example, in the example shown in FIG. 9, for the line segment p _{l1 p'l1 with} a certainty degree of 1 in the series P, there is no line segment of the associated series Q and the line segment of the series R has a certainty degree of 1. Is not generated, so the line segments r _{l1, * r'l1, * after} translation application are registered in the series R with a certainty of 1.

In the next step S110, the integration unit 28 determines whether or not the processing of all the sets in the group has been completed. If the processing of all the sets in the group has not been completed yet, the determination in step S110 becomes a negative determination, and the process proceeds to step S112. In step S112, the integration unit 28 reconstructs the sequence based on the normalization certainty, then returns to step S104, and repeats each process of steps S104 to S108 based on the reconstructed sequence. On the other hand, when the processing of all the sets in the group is completed, the determination in step S110 becomes an affirmative determination, and the main line map generation processing ends.

In the present embodiment, when integrating a plurality of line segments, for example, a set of series {A1, A2, A3, A4 ...} For each lane of the road R, and {B1, b2, b3, b4 ... •} Is defined, and all combinations in each set are averaged by the averaging unit 26 in step S108.

For line segments whose conviction is "undecided" by the processing of step S108, if there is a case in which even one set can be associated with another series, the conviction is set to 1 and there is no such case. If so, the degree of certainty is set to 0.

The output number series (for example, 8 series) is selected in descending order of normalization certainty, and the set for the next step is reconstructed. The normalization conviction is expressed by, for example, the following equation (16).
Normalization certainty = (value obtained by summing exp (-d) of the above equation (15) in the output series) / (detection line fraction x mileage) ... (16)

When the combination of the same series is duplicated, for example, in the example shown in FIG. 10, in the case of "A123", the series having the maximum normalization certainty is adopted.

In order to exclude a large measurement error in the positioning unit 14, a series that is in the top percentage (for example, 40%) in descending order of the average value of the norm || v ^* || of the movement amount v ^* in the set is selected. , It is regarded as a series that is out of position, and the position of the line segment (end point) is adjusted. When the associated series is the series P and Q, one of the following modes is taken.

When the series P corresponds to the above, the coefficient k is set to 1 (k = 1) in the above equations (11) and (12). When the series Q corresponds to the above, the coefficient k is set to 0 (k = 0) in the above equations (11) and (12). If both the series P and Q correspond to the above, the movement amount obtained by averaging the movement amount v ^* from the series P and Q to other than the series P and Q is added.

As an example, the number of integrations in the set is set to 2 and the number of integrations between the sets is set to 1, and the integration unit 28 finally outputs the series having the maximum normalization certainty to the map database 18 as the integration result.

For example, in the example shown in FIG. 10, for a certain lane, the lines A1, A2, A3, and A4 detected from different sequences are used, and the lines A1 and A2 are separated by the processing of S104 to S108. When A12 is generated, line A13 is generated from line A1 and line A3, line A23 is generated from line A2 and line A3, and line A34 is generated from line A3 and line A4. Is shown. Further, the processes of steps S104 to S108 are repeated to generate line segment A123 from line segment A12 and line segment A13, line segment A123 from line segment A13 and line segment A23, and line segment A12 and line segment A34. Generate line segment A1234. In this case, since the line segments A123 overlap, a line segment A123 having a high degree of certainty of normalization, and in the example shown in FIG. 10, a combination of the line segments A12 and the line segment A13 is adopted.

Further, using the line segments B1, B2, B3, and B4 detected from different series, the line segment B1 and the line segment B2 are generated from the line segment B1 and the line segment B2 by the processing of the above S104 to S108, and the line segment B1 and the line segment are generated. The case where the line segment B13 is generated from B3, the line segment B24 is generated from the line segment B2 and the line segment B4, and the line segment B34 is generated from the line segment B3 and the line segment B4 is shown. Further, the process of steps S104 to S108 is repeated to generate a line segment B1234 from the line segment B13 and the line segment B24. In this case, the line segment B12 and the line segment B34 are not used for integration because they are a combination of line segments having a low degree of certainty of normalization.

Then, the integration unit 28 integrates the line segment A1234 of the set A and the line segment B1234 of the set B, derives the integration result A1234B1234, and outputs the integration result A1234B1234 to the map database 18.

11A and 11B show an application example of the map generator 10 of the present embodiment. In the example shown in FIG. 11A, a plurality of line segments each of the series P and the series Q are represented on the same plane, and the line segment of the series P and the line segment of the series Q are associated with each other by the line segment corresponding portion 24. It shows the state that was done. In the example shown in FIG. 11A, the associated line segments are indicated by arrows. Further, in the example shown in FIG. 11B, an example of the integration result obtained by integrating the sequence P and the sequence Q by the integration unit 28 is shown. In FIG. 11B, the line segment of the integration result (inlier) is shown by a solid line, and the line segment of the outlier is shown by a dotted line. Further, as a comparative example, FIG. 11C shows an example of the integration result when each line segment of the sequence P in FIG. 11A and the entire line segment of the sequence Q are translated in parallel. In FIG. 11C, similarly to FIG. 11B, the line segment of the inlier is shown by a solid line, and the line segment of the outlier is shown by a dotted line. As can be seen by comparing FIGS. 11B and 11C, according to the map generator 10 of the present embodiment, the line segments can be appropriately integrated between the series as compared with the case where the line segments of the series are simply translated. You can see that it is done.

Further, FIGS. 12A to 12D show an application example of the map generation device 10 of the present embodiment. In the example shown in FIG. 12A, a projection drawing showing a plurality of line segments of the series P detected by the line segment detection unit 20 and whose coordinates are converted by the projection unit 22 is shown. Further, in the example shown in FIG. 12B, a projection drawing showing a plurality of line segments of the series Q detected by the line segment detection unit 20 and whose coordinates are converted by the projection unit 22 is shown. Each of the series P and the series Q is a projection image obtained from the captured images obtained by capturing the left and right sides of the road R in the same traveling section. FIG. 12C shows a state in which the line segment of the sequence P shown in FIG. 12A and the line segment of the sequence Q shown in FIG. 12B are associated with each other by the line segment corresponding portion 24. In FIG. 12C, the line segments associated with each other are indicated by arrows in the same manner as in FIG. 11A. Further, FIG. 12D shows an example of the integration result obtained by integrating the sequence P and the sequence Q by the integration unit 28. In FIG. 12D, as in FIG. 11B, the line segment of the integration result (inlier) is shown by a solid line, and the line segment of the outlier is shown by a dotted line. As shown in FIG. 12D, according to the map generator 10 of the present embodiment, when the captured image obtained by capturing the left and right sides of the road R is used, the line segment representing the lane marking near the center of the road R is appropriate due to the integration. It can be seen that it has been detected in.

As described above, the map generation device 10 of each of the above embodiments includes a line segment detection unit 20, a line segment corresponding unit 24, an averaging unit 26, and an integration unit 28. The line segment detection unit 20 detects a plurality of line segments from each of the plurality of captured images captured in the same traveling section by the image pickup unit 12 mounted on the vehicle 2. The line segment corresponding unit 24 calculates the position difference amount between the end points of the line segment detected from the captured image and the end points of the line segment detected from the other captured images for each of the plurality of captured images. Based on the calculated position difference amount, the position difference amount is reduced between the two series of the series consisting of the line segments detected from the captured image and the series consisting of the line segments detected from the other captured images. Associate line segments. The averaging unit 26 generates a new line segment at a position between the associated line segments based on the mapping of the line segments between the two series associated with the line segment corresponding unit 24, and newly generates a new line segment. Generate a series of line segments. The integration unit 28 causes the averaging unit 26 to associate the two series of the series consisting of the new line segments with each other by the line segment corresponding unit 24 and generate the new line by the averaging unit 26. Integrate two series of line segments.

As described above, according to the map generation device 10 of the present embodiment, the line segment correspondence unit 24 is optimally and duplicated as a whole between the series by the mapping using the conviction degree indicating the degree of line segment matching as an index. The line segment without a line segment is associated. Further, the averaging unit 26 generates a new line segment by integrating both line segments at an intermediate position between the two corresponding line segments based on the matching result. Further, the line segment correspondence unit 24 performs interpolation of the movement amount based on the correspondence relationship of the surrounding line segments for the line segments that are not associated with each other, and updates the position. By these processes, it is possible to supplement the non-linear distortion of the line segment distribution due to the locus error of the positioning unit 14. Further, in the averaging unit 26 and the integration unit 28, when the averaging and integration are repeated, the series that are out of position, for example, the line segment detected afterwards can be excluded and integrated. Further, by increasing the number of series, in other words, the number of captured images, the line segment after integration tends to converge to the actual position (true position). In addition, the distortion due to the trajectory error also tends to converge. As described above, according to the map generation device 10 of the present embodiment, it is possible to absorb distortions that do not overlap by simply translating the entire line segment of each series.

Therefore, according to the map generation device 10 of the present embodiment, it is possible to generate a highly accurate line map.

The line map obtained by the map generation device 10 of the present embodiment and stored in the map database 18 can be used, for example, for estimating the self-position of the vehicle 2 and for automatic driving.

Various drives may be connected to the map generator 10. Various drives are devices that read data from a computer-readable and portable recording medium such as a flexible disk, magneto-optical disk, or CD-ROM, or write data to the recording medium. When various drives are provided, the control program may be recorded on a portable recording medium, and the control program may be read and executed by the corresponding drive.

1 Map generation system 10 Map generation device 12 Imaging unit 14 Positioning unit 20 Line segment detection unit 22 Projection unit 24 Line segment correspondence unit 26 Average unit 28 Integration unit

Claims (6)

A line segment detection unit that detects a plurality of line segments from each of a plurality of captured images captured in the same traveling section by an image pickup unit mounted on a vehicle.
For each of the plurality of line segments detected by the line segment detection unit, coordinates representing the positions of both end points of each line segment are obtained based on the position information of the vehicle associated with each of the plurality of captured images. A projection unit that converts to absolute coordinates in a bird's-eye view,
The plurality of line segments detected by the line segment detection unit are set as one series for each of the plurality of captured images.
Based on the position of the end point based on the absolute coordinates, the position difference amount between the end point of the line segment of the own series and the end point of the line segment of the other series is calculated from each series of the plurality of captured images. Then, based on the calculated position difference amount, the line segment corresponding portion that associates the line segments so that the position difference amount becomes small between the two series of the own series and the other series. ,
Based on the correspondence between the two series of the own series associated with the line segment corresponding portion and the other series, the position is in the middle of the corresponding end points of the associated line segments. An averaging unit that generates a new line segment provided with an end point and generates a series consisting of the new line segment.
A plurality of the new line segments generated by performing the mapping of the line segments by the line segment corresponding portion and the generation of the series consisting of the new line segments by the averaging section for all the series. An integration unit that integrates each series of each of the plurality of captured images into one series by repeatedly performing a series of processes for reconstructing a set of a plurality of series consisting of each of the above .
A map generator equipped with. The integration unit integrates the two series by converging the position of the associated line segment to the center position.
The map generator according to claim 1. The line segment detection unit detects the core wire of the dividing line as a line segment from each of the plurality of captured images.
The map generator according to claim 1 or 2 . When averaging a series of upper n (n is an integer of 1 or more) sets having a large position error in the set of the series and a series other than the n sets, the averaging unit is a series other than the n sets. Generate a line segment that is closer to the position of the line segment,
The map generator according to any one of claims 1 to 3 . The integration unit selects and selects a higher -order series having high consistency of the new line segment from a plurality of series consisting of each of the plurality of new line segments generated by the averaging unit. Reconstructing a set of sequences,
The map generator according to any one of claims 1 to 4 . Computer,
A line segment detection unit that detects a plurality of line segments from each of a plurality of captured images captured in the same traveling section by an image pickup unit mounted on a vehicle.
For each of the plurality of line segments detected by the line segment detection unit, coordinates representing the positions of both end points of each line segment are obtained based on the position information of the vehicle associated with each of the plurality of captured images. Projector that converts to absolute coordinates in a bird's-eye view,
The plurality of line segments detected by the line segment detection unit are set as one series for each of the plurality of captured images.
Based on the position of the end point based on the absolute coordinates, the position difference amount between the end point of the line segment of the own series and the end point of the line segment of the other series is calculated from each series of the plurality of captured images. Then, based on the calculated position difference amount, the line segment corresponding portion that associates the line segments between the two series of the own series and the other series so that the position difference amount becomes small.
Based on the correspondence between the two series of the own series associated with the line segment corresponding portion and the other series, the position is in the middle of the corresponding end points of the associated line segments. An averaging unit that generates a new line segment provided with an end point and generates a series consisting of the new line segment.
A plurality of the new line segments generated by performing the mapping of the line segments by the line segment corresponding portion and the generation of the series consisting of the new line segments by the averaging section for all the series. An integrated unit that integrates each series of each of the plurality of captured images into one series by repeatedly performing a series of processes for reconstructing a set of a plurality of series consisting of each of the above .
A program to function as.

Images