JP7001985B2 - Vehicle position estimation device, program, recording medium, and vehicle position estimation method - Google Patents

Description

The present disclosure relates to a vehicle position estimation device, a program, a recording medium, and a vehicle position estimation method.

Conventionally, many self-vehicle position estimation devices for estimating the position of a vehicle on a map have been proposed.

In the above-mentioned own vehicle position estimation device, lane boundaries and landmarks (road markings, road markings, etc.) are extracted from the image taken by the in-vehicle camera, and the lane level is combined with the detailed map information of the road. Some use a combination of these two pieces of information for consistency checking.

Japanese Unexamined Patent Publication No. 2005-265494

The above-mentioned own vehicle position estimation device is required to further improve the estimation accuracy of the vehicle position.

An object of the present disclosure is to provide a vehicle position estimation device, a program, a recording medium, and a vehicle position estimation method capable of further improving the vehicle position estimation accuracy.

The first form of this disclosure is
Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , An input unit that receives the road surface image captured by the image pickup device used in the vehicle, and
A first point cloud, which is a point cloud along the first direction, is extracted from the road markings included in the road surface image.
A second point cloud, which is a point cloud along the second direction, is extracted from the road markings included in the road surface image.
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison of the first line segment and the first point cloud, and the second line segment and the second point. It is equipped with a processing unit that calculates the vehicle position by performing comparison processing only for comparison of groups.
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
It is aimed at the vehicle position estimation device.

The second form of the disclosure is
On the computer
Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , The step of receiving the road surface image captured by the image pickup device used in the vehicle, and
A step of extracting a first point cloud, which is a point cloud along the first direction, from the road markings included in the road surface image.
A step of extracting a second point cloud, which is a point cloud along the second direction, from the road markings included in the road surface image, and
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison of the first line segment and the first point cloud, and the second line segment and the second point. By performing the comparison process limited to the point cloud comparison, the step of calculating the vehicle position and the step of calculating the vehicle position are executed.
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
Directed to the program.

The third form of the present disclosure is
On the computer
Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , The step of receiving the road surface image captured by the image pickup device used in the vehicle, and
A step of extracting a first point cloud, which is a point cloud along the first direction, from the road markings included in the road surface image.
A step of extracting a second point cloud, which is a point cloud along the second direction, from the road markings included in the road surface image, and
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison of the first line segment and the first point cloud, and the second line segment and the second point. By performing the comparison process limited to the point cloud comparison, the step of calculating the vehicle position and the step of calculating the vehicle position are executed.
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
Aimed at the recording medium on which the program was recorded.

The fourth form of the present disclosure is
Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , The step of receiving the road surface image captured by the image pickup device used in the vehicle, and
A step of extracting a first point cloud, which is a point cloud along the first direction, from the road markings included in the road surface image.
A step of extracting a second point cloud, which is a point cloud along the second direction, from the road markings included in the road surface image, and
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison of the first line segment and the first point cloud, and the second line segment and the second point. It is provided with a step of calculating the vehicle position by performing comparison processing limited to the comparison of the group.
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
It is directed to the vehicle position estimation method.

According to the present disclosure, the estimation accuracy of the vehicle position can be further improved.

Schematic diagram showing a vehicle equipped with the own vehicle position estimation device according to one embodiment of the present disclosure. Diagram illustrating the lane marking Diagram illustrating a stop line Diagram illustrating a pedestrian crossing Illustration exemplifying an arrow Block diagram showing the configuration of the vehicle position estimation device The flow diagram which shows the 1st part of the processing procedure of the own vehicle position estimation apparatus of FIG. The flow chart which shows the 2nd part of the processing procedure of the own vehicle position estimation apparatus of FIG. The flow chart which shows the 3rd part of the processing procedure of the own vehicle position estimation apparatus of FIG. A diagram showing a front bird's-eye view image, a rear bird's-eye view image, a left bird's-eye view image, a right bird's-eye view image, and a first point cloud detected from them. A diagram showing a front bird's-eye view image, a rear bird's-eye view image, a left bird's-eye view image, a right bird's-eye view image, and a second point cloud detected from them. A diagram showing a front bird's-eye view image, a rear bird's-eye view image, a left bird's-eye view image, a right bird's-eye view image, and edges detected from them.

Hereinafter, the vehicle position estimation device according to one embodiment of the present disclosure will be described in detail with reference to the drawings.

<1. Definition>
Table 1 below shows the meanings of acronyms and abbreviations used in the following explanations.

Further, in the present disclosure, the first direction a indicates the extension direction of the road, and the second direction b indicates the direction substantially perpendicular to the extension direction of the road.

<2. Embodiment>
<2-1. Configuration of own vehicle position estimation device>
In FIG. 1, the vehicle V includes an ADAS map storage unit 1, at least one image pickup device 2, a positioning device 3 by radio navigation, a self-contained navigation sensor group 4, a vehicle position estimation device 5, and a vehicle control unit. 6 and are installed.

The ADAS map storage unit 1 stores so-called ADAS map data. The ADAS map data includes not only the extension direction and shape of each road constituting the road network, but also information that can identify the road markings drawn on each road.

Road markings are markings that indicate regulations or instructions regarding road traffic and are strip-shaped lines or symbols drawn on the road surface. Examples of road markings include pavement lines, stop lines, pedestrian crossings and arrow marks.

As illustrated in FIG. 2A, the lane marking is a solid line or a broken line for partitioning a lane from the shoulder or another lane, and is a white line or a yellow line along the first direction a.

As illustrated in FIG. 2B, the stop line indicates a position on the road where the vehicle should stop, and consists of a white line along the second direction b.

The pedestrian crossing, as illustrated in FIG. 2C, is a plurality of white lines along the first direction a and includes a plurality of white lines arranged in the second direction b at intervals. In addition, although not shown, some pedestrian crossings are ladder-shaped (that is, a plurality of white lines along the first direction a are interposed between two white lines extending in the second direction b). be.

The arrow marks indicate that the target lane is for a right turn, a straight line, or a left turn, as shown in FIG. 2D.

In the ADAS map data, the outer shape of each road marking and pedestrian crossing is defined by a point cloud (a collection of coordinate values) of the edge.

Further, in the ADAS map data, the division line of FIG. 2A is a line segment parallel to the first direction a (hereinafter, may be referred to as a first line segment) with two points included in each white line portion or the like at both ends. Defined. More specifically, these two points are the center points of the two sides facing each other in the first direction a in the white line portion and the like.

On the other hand, in the ADAS map data, the white line portion extending in the second direction b of the stop line in FIG. 2B has two points included in each white line portion at both ends and is a line segment parallel to the second direction b (hereinafter referred to as the first line segment). It is sometimes called a two-line segment).

Although the description is simplified because it is not a main part of the present disclosure, the ADAS map data may include three-dimensional point cloud data indicating the shape of a feature or the like. Such three-dimensional point cloud data may be created by using an external sensor (for example, an infrared laser scanner) or a locator (for example, GPS) mounted on a dedicated surveying vehicle.

At least one image pickup device 2 is, in the present disclosure, image pickup devices 2F, 2B, 2L and 2R that image the surroundings of the vehicle V (preferably, front, rear, left and right sides). The image pickup devices 2F and 2B are mounted on the bumper of the vehicle V, for example, so as to image the road surface in front of and behind the vehicle V, and the image pickup devices 2L and 2R image the road surface in the left and right directions of the vehicle V. As described above, it is mounted on each door mirror of the vehicle V.

In the present disclosure, as described above, the image pickup devices 2F, 2B, 2L, and 2R can image a wide range around the vehicle V by taking images in different directions. That is, each image pickup device 2F, 2B, 2L, 2R can image not only the front of the lane in which the vehicle V is traveling, but also the roadside, the adjacent lane, the oncoming lane, and the like. As a result, the image pickup devices 2F, 2B, 2L, and 2R can reliably capture the stop line and the lane marking, and as a result, the own vehicle position estimation device 5 can extract the road markings.

The image pickup devices 2F, 2B, 2L, and 2R preferably synchronize with each other and take images at a predetermined frame rate, and represent the road surface conditions in the field of view of the respective image pickup devices 2F, 2B, 2L, and 2R. The image is transmitted to the own vehicle position estimation device 5. Hereinafter, the road surface images of the image pickup devices 2F, 2B, 2L, and 2R may be referred to as a front road surface image, a rear road surface image, a left road surface image, and a right road surface image. These road surface images are taken at the same time if the image pickup devices 2F, 2B, 2L, and 2R are synchronized with each other. Hereinafter, the road surface image is referred to as "captured at the same time" even when the vehicle position estimation device 5 handles each other's images as being captured in the same time zone even if the captured times are not exactly the same time. It is written as "what was done".

The positioning device 3 is, for example, a GPS receiver, and outputs information indicating the current absolute position of the vehicle V from received signals from a plurality of artificial satellites provided in the positioning system (GPS in the present disclosure).

The self-contained navigation sensor group 4 includes, for example, a directional sensor and a speed sensor. The directional sensor outputs a signal indicating the traveling direction of the vehicle V. The speed sensor outputs a signal indicating the moving speed of the vehicle V.
The sensor group 4 may include an acceleration sensor or an external world sensor (typically a stereo camera) instead of the speed sensor, or may include an angular acceleration sensor instead of the orientation sensor.

As shown in FIG. 3, for example, the own vehicle position estimation device 5 is an ECU including an input unit 61, a processing unit 62 for performing arithmetic processing, and an output unit 63 mounted on a substrate. The input unit 61 is connected to the ADAS map storage unit 1, the image pickup device 2, the positioning device 3 by radio navigation, and the self-contained navigation sensor group 4, and image data, sensor data, etc. It receives various data of. The processing unit 62 executes a program stored in advance and derives the current position of the vehicle V with high accuracy from the input information and the input signal of the input unit 61. The output unit 63 outputs the own vehicle position (current position) of the vehicle V derived by the processing unit 62 to the vehicle control unit 6.

In the present disclosure, the vehicle control unit 6 controls a vehicle control ECU for controlling the accelerator opening of the drive source of the vehicle V, the steering amount of the steering device, the deceleration of the braking device, and the like in order to automatically drive the vehicle V. Is.

<2-2. Processing of own vehicle position estimation device>
Next, the processing procedure of the own vehicle position estimation device 5 will be described with reference to FIGS. 1 to 6C.
In FIG. 4, the processing unit 62 functions as a bird's-eye view map data generation unit, and receives ADAS map data in a predetermined range including the current position of the vehicle V from the ADAS map storage unit 1 via the input unit 61 (step). S110). The ADAS map data received in this step includes a first line segment that defines a lane marking (see FIG. 2A) and a second line segment that defines a stop line and the like (see FIG. 2B and the like). Further, the current position of the own vehicle is obtained in steps S11 or S12 of FIG. 5, which will be described later. As the initial value of the current vehicle position, information from GNSS (Global Navigation Satellite System) or the like is used as the vehicle position. Further, what range of ADAS map data is received from the ADAS map storage unit 1 from the current position of the vehicle V is arbitrary, and a desired range can be set.

The processing unit 62 selects and extracts all the first line segments that define the lane markings (see FIG. 2A) from the ADAS map data acquired in step S110 (step S120), and determines the extracted first line segments. Generate the first bird's-eye view map data looking down from the viewpoint (step S130).

The processing unit 62 selects and extracts all the second line segments that define the stop line (see FIG. 2B) from the ADAS map data acquired in step S110 (step S140), and determines the extracted second line segment. The second bird's-eye view map data (stop line) looking down from the viewpoint is generated (step S150).

The processing unit 62 selects and extracts all the point groups that define the outer shape of the road markings (excluding the pedestrian crossing; see FIGS. 2A, 2B, and 2D) from the ADAS map data acquired in step S110. (Step S160), the third bird's-eye view map data (excluding the pedestrian crossing) looking down on the extracted point group from a predetermined viewpoint is generated (step S170).

From the ADAS map data acquired in step S110, the processing unit 62 defines a point cloud that defines the outer shape of the pedestrian crossing (see FIG. 2C) and a second line that defines a white line portion along the second direction b in the pedestrian crossing. Select and extract all the minutes (step S180).

Following step S180, the processing unit 62 generates third bird's-eye view map data (pedestrian crossing) looking down on the point group acquired in the same step from a predetermined viewpoint (step S190), and the second line segment acquired in the same step. Generates second bird's-eye view map data (pedestrian crossing) looking down from a predetermined viewpoint (step S200).

Next, the processing unit 62 functions as a bird's-eye view road surface image generation unit, and receives each bird's-eye view map data generated by the bird's-eye view map data generation unit (step S1 in FIG. 5A).

Next, the processing unit 62 reads the road surface image (that is, the front road surface image, the rear road surface image, the left road surface image, and the right road surface image) obtained at the same time by the image pickup apparatus 2F, 2B, 2L, 2R (that is, the front road surface image, the rear road surface image, and the right road surface image). Step S2).

Next, the processing unit 62 looks down on each road surface image read in step S2 from a predetermined viewpoint (that is, a front bird's-eye view image, a rear bird's-eye view image, a left bird's-eye view image, and a right bird's-eye view image). ) (Step S3). Here, the bird-view road surface image may be a composite image of the front bird-view road surface image, the rear bird-view road surface image, the left bird-view road surface image, and the right bird-view road surface image, or the front bird-view road surface image, the rear bird-view road surface image, and the left. It may be a group of images in which a square bird's-eye view road surface image and a right-side bird's-eye view road surface image are put together. FIG. 6A exemplifies a composite image of a front bird's-eye view road surface image, a rear bird's-eye view road surface image, a left bird's-eye view road surface image, and a right bird's-eye view road surface image. 6B and 6C are similar composite images.

Next, the processing unit 62 determines whether or not the first line segment has been extracted in step S120 (see FIG. 4) (step S21), and if YES, executes steps S4 and S5 described later. If NO, skip these steps. Further, the processing unit 62 determines whether or not the second line segment has been extracted in step S140 (see FIG. 4) (step S22), and if YES, executes steps S6 and S7 described later, and NO. If, skip these steps. Further, the processing unit 62 determines whether or not the point cloud that defines the outer shape of the road marking has been extracted in step S160 (see FIG. 4) (step S23), and if YES, steps S8 and S9 described later. And if NO, skip these steps.

That is, the first line segment that defines the lane marking (see FIG. 2A) is not always extracted in step S120. Similarly, in step S140, the second line segment defining the stop line (see FIG. 2B) is not always extracted, and in step S160, the outer shape of the road marking (see FIGS. 2A to 2D) is determined. The point cloud to be defined is not always extracted. Against this background, in the present disclosure, the processing load of the processing unit 62 is further reduced by performing the determination of step S21 and the like as described above and executing the processing of steps S4 and S5 and the like only when necessary. I am trying.

When it is determined that the first line segment has been extracted (step S21, YES), the processing unit 62 functions as the first extraction unit and performs edge detection processing or the like on each bird's-eye view road surface image generated in step S3. From the edges of each road sign obtained by this, a point cloud along the first direction (hereinafter referred to as a first point cloud) is extracted (step S4). FIG. 6A shows an example of the first point cloud with the reference numeral P1.

Next, the processing unit 62 performs pattern matching processing (comparison) using the first bird's-eye view map data obtained in step S130 and the first point cloud group P1 obtained in step S4, and performs the first bird's-eye view map data. Search for a part that matches the first point cloud P1 in (step S5).

Further, when it is determined that the second line segment has been extracted (step S22, YES), the processing unit 62 functions as the second extraction unit, and is used, for example, in parallel with step S4 in step S4 in parallel. Edge detection processing or the like is performed on each bird's-eye view road surface image that is the same as the one, and a point cloud along the second direction (hereinafter referred to as a second point cloud) is extracted from the edges of each road marking obtained by this (step S6). .. FIG. 6B shows an example of the second point cloud with the reference numeral P2.

Next, the processing unit 62 obtains the second bird's-eye view map data obtained by synthesizing the second bird's-eye view map data (stop line) and the second bird's-eye view map data (crosswalk) obtained in steps S150 and S200, and the second bird's-eye view map data obtained in step S6. A pattern matching process is performed using the second point group P2, and a portion matching the second point group P2 in the second bird's-eye view map data is searched for (step S7).

Further, when it is determined that the point group defining the outer shape of the road marking has been extracted (step S23, YES), the processing unit 62 functions as an edge extraction unit, for example, in parallel with steps S4 and S6, in steps. Edge detection processing or the like is performed on each bird's-eye view road surface image generated in S3 to extract the edges of each road marking (step S8). FIG. 6C shows an example of an edge with reference numeral P3.

Next, the processing unit 62 performs pattern matching processing using the third bird's-eye view map data obtained in steps S170 and 190 and the edge obtained in step S8, and matches the edge P3 in the third bird's-eye view map data. Search for the part to be used (step S9).

Note that FIG. 5 shows an example in which steps S4 and S5, steps S6 and S7, and steps S8 and S9 are executed in parallel. However, the present invention is not limited to this, and steps S4, S5, steps S6, S7, and steps S8, S9 may be executed in series in any order.

Next, the processing unit 62 determines whether or not there is a comparison target (step S10). Specifically, the processing unit 62 determines whether or not a matching portion has been found in all of steps S5, S7, and S9.

If NO is determined in step S10, the processing unit 62 is obtained from the current vehicle position obtained by odometry (for example, the current position obtained by the positioning device 3 or the output signal of the self-contained navigation sensor group 4). The current position) is the current position used in the next step S110 (see FIG. 4) (step S11).

On the other hand, if YES is determined in step S10, the processing unit 62 functions as a calculation unit, and the position where the total of the distances (1) to (3) below is equal to or less than the predetermined reference value (that is, the error becomes small). Such a position) is searched for, and the current position of the vehicle V to be used in the next step S110 is used (step S12).
(1) Total distance from each point of the first point group found in step S5 to the matching part (first line segment) in the first bird's-eye view map data (2) Each point of the second point group found in step S7 Total distance from each point of the edge found in step S9 to the matching part (outer shape) in the third bird's-eye view map data.

In addition, the processing unit 62 may search for a position where the total of the distances (1) to (3) is the minimum value in step S12.

The processing unit 62 passes the current position obtained in either one of steps S11 and S12 to the bird's-eye view map data generation unit for use in the next step S110 (step S13). Further, the processing unit 62 passes the obtained current position to the vehicle control unit 6 via the output unit 63. In addition, the processing unit 62 may output the obtained current position to, for example, a navigation system via the output unit 63.

The above processes of FIGS. 4 and 5 are repeatedly executed, for example, while the accessory power supply of the vehicle V is turned on.

As another example, in step S10, the processing unit 62 may determine whether or not a matching portion is found at least in both steps S5 and S7 as a criterion for determining whether or not there is a comparison target.

Further, as another example, it is possible to set not to perform the processing of steps S23, S8, and S9. In this case, in step S10, the processing unit 62 determines whether or not a matching portion is found in both steps S5 and S7 as a criterion for determining whether or not there is a comparison target. The process of step S12 (calculation unit) when YES is determined in step S10 is the same as described above except that the above (3) is not used.

<2-3. Action / effect of this vehicle position estimation device>
By the way, as in the past, simply extracting lane boundaries and landmarks (road markings, road markings, etc.) from images taken with an in-vehicle camera will cause the landmarks to deteriorate over time (specifically, paint peeling, etc.). Therefore, the shape of the lane boundary line or the landmark extracted from the image may not keep the original shape in the extreme. As a result, it is required to further improve the accuracy of the consistency check between the landmarks and the map information at the lane level, and by extension, the accuracy of the estimated current position.

As described above, in the present disclosure, the road markings are classified into a white line portion and the like along the first direction a and a white line portion and the like along the second direction b. Then, in the ADAS map data, the white line segment or the like along the first direction a (or the second direction b) is a first line segment (or a second line segment) parallel to the first direction a (or the second direction b). ) Is specified by two points at both ends.

The processing unit 62 functions as a first extraction unit (or a second extraction unit), and is a first point cloud that constitutes each road sign from the road surface image around the vehicle V obtained at the same time by the image pickup device 2. The first point cloud P1 (or the second point cloud P2) along the first direction a (or the second direction b), which is P1 (or the second point cloud P2), is extracted.

After that, the processing unit 62 functions as a calculation unit, and determines the position of the vehicle V based on the comparison result of the first line segment and the first point group P1 and the comparison result of the second line segment and the second point group P2. calculate.

As described above, the processing unit 62 is limited to the first direction a, searches for a portion where the first point group P1 extracted from the road surface image matches the first line segment extracted from the ADAS map data, and at the same time, the first By limiting to two directions b, the second point group P2 extracted from the road surface image is searched for a portion matching the second line segment extracted from the ADAS map data. As described above, in the present disclosure, the processing unit 62 limits or specializes in the first direction a and the second direction b, and estimates the position of the vehicle V in each direction. Even if it collapses, the estimated vehicle position (own vehicle position) becomes highly accurate as long as the first point cloud P1 and the second point cloud P2 along the first direction a and the second direction b are obtained.

Furthermore, by limiting to the first direction a and the second direction b and performing the matching process or the like by the processing unit 62, it is possible to reduce the processing load of the processing unit 62 as compared with the conventional case.

Further, as described above, in the present disclosure, the image pickup devices 2F, 2B, 2L, 2R can image not only the front of the lane in which the vehicle V is traveling, but also the roadside, the adjacent lane, the oncoming lane, and the like. .. As a result, it is highly possible that any one of the front road surface image, the rear road surface image, the left road surface image, and the right road surface image includes a stop line and a lane marking line, and as a result, the own vehicle position estimation device 5 is the first. The possibility of extracting the point cloud and the second point cloud increases.

<2-4. Addendum>
In the above, it has been described that the ADAS map storage unit 1 is mounted on the vehicle V. However, the present invention is not limited to this, and the ADAS map storage unit 1 may be located on a remote server. In this case, the processing unit 62 will read the ADAS map data from the remote server.

Further, in the above description, the first direction a is the extension direction of the road, and the second direction b is the direction perpendicular to the first direction a. However, not limited to this, the first direction a may have a deviation of about a tolerance with respect to the extension direction of the road, and the second direction b may have a deviation of about a tolerance with respect to the direction perpendicular to the first direction a. There may be.

The extension direction of the road is not always straight. In this case, it may be divided into sections of appropriate length on the road, and the first line segment may be defined for each section.

Further, in the above, a single ECU (that is, the own vehicle position estimation device 5) has executed the processes of FIGS. 4 and 5. However, the own vehicle position estimation device 5 may at least execute steps S4 to S13 of FIG. 5, and other processes may be executed by, for example, a vehicle control unit 6 or a navigation ECU other than the own vehicle position estimation device 5. good.

Further, the program executed by the processing unit 62 may be stored and provided in a non-transient recording medium such as a DVD, or may be stored in a server device on the network so as to be downloadable via the network. Is also good.

Further, in the above, it has been described that both the ADAS map data and the road surface image are converted into a bird's-eye view. However, the present invention is not limited to this, and for example, the ADAS map data and the road surface image may be converted into another coordinate system.

In addition, the above embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its gist or its main features.

The own vehicle position estimation device and the like according to the present disclosure can improve the estimation accuracy of the own vehicle position and are suitable for in-vehicle use.

1 ADAS map storage unit 2 Imaging device 5 Own vehicle position estimation device 61 Input unit 62 Processing unit 63 Output unit

Claims (11)

Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , An input unit that receives the road surface image captured by the image pickup device used in the vehicle, and
A first point cloud, which is a point cloud along the first direction, is extracted from the road markings included in the road surface image.
A second point cloud, which is a point cloud along the second direction, is extracted from the road markings included in the road surface image.
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison between the first line segment and the first point cloud, and the second line segment and the second point cloud. It is equipped with a processing unit that calculates the vehicle position by performing comparison processing limited to the comparison of
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
Own vehicle position estimation device. The first aspect of the present invention, wherein the processing unit extracts the first point cloud and the second point cloud from the edge information indicating the outer shape of the road marking included in the road surface image obtained at the same time. Vehicle position estimation device. When the first line segment is not included in the map data, the processing unit does not perform the extraction process of the first point cloud, and the second line segment is not included in the map data. The own vehicle position estimation device according to claim 1, which does not perform the extraction process of the second point cloud. The processing unit searches for a portion where the first point group extracted from the road surface image matches the first line segment by limiting in the first direction, and limits in the second direction to the road surface image. The own vehicle position estimation device according to claim 1, wherein the second point cloud extracted from is searched for a portion that matches the second line segment. Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the processing unit compares the first line segment and the first point cloud, and the second line segment and the second line segment. The own vehicle position estimation device according to claim 1, which calculates the vehicle position by performing a comparison process only based on the comparison of the second point cloud. The vehicle position estimation device according to claim 1, wherein the processing unit extracts the first point cloud and the second point cloud from the bird's-eye view road surface image converted so as to look down on the road surface image from a predetermined viewpoint. The processing unit
The distance from each point of the first point group to the matching portion with the first point group in the first bird's-eye view map data generated so as to look down on the first line segment from the predetermined viewpoint.
The total of the distances from each point of the second point cloud to the matching portion with the second point cloud in the second bird's-eye view map data generated so as to look down on the second line segment from the predetermined viewpoint is the minimum. The own vehicle position estimation device according to claim 6 , wherein the position of the vehicle is calculated as the current vehicle position. The processing unit
The distance from each point of the first point group to the matching portion with the first point group in the first bird's-eye view map data generated so as to look down on the first line segment from the predetermined viewpoint.
The total of the distance from each point of the second point cloud to the matching portion with the second point group in the second bird's-eye view map data generated so as to look down on the second line segment from the predetermined viewpoint is predetermined. The own vehicle position estimation device according to claim 6 , which calculates the position of the vehicle that is equal to or less than the reference value as the current vehicle position. On the computer
Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , The step of receiving the road surface image captured by the image pickup device used in the vehicle, and
A step of extracting a first point cloud, which is a point cloud along the first direction, from the road markings included in the road surface image.
A step of extracting a second point cloud, which is a point cloud along the second direction, from the road markings included in the road surface image, and
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison between the first line segment and the first point cloud, and the second line segment and the second point cloud. By performing the comparison process limited to the comparison of the above, the step of calculating the vehicle position and the step of calculating the vehicle position are executed.
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
program. On the computer
Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , The step of receiving the road surface image captured by the image pickup device used in the vehicle, and
A step of extracting a first point cloud, which is a point cloud along the first direction, from the road markings included in the road surface image.
A step of extracting a second point cloud, which is a point cloud along the second direction, from the road markings included in the road surface image, and
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison between the first line segment and the first point cloud, and the second line segment and the second point cloud. By performing the comparison process limited to the comparison of the above, the step of calculating the vehicle position and the step of calculating the vehicle position are executed.
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
A recording medium on which the program is recorded. Map data including a first line segment along the first direction that defines the road marking and a second line segment along the second direction different from the first direction that defines the road marking in a predetermined range including the position of the vehicle. , The step of receiving the road surface image captured by the image pickup device used in the vehicle, and
A step of extracting a first point cloud, which is a point cloud along the first direction, from the road markings included in the road surface image.
A step of extracting a second point cloud, which is a point cloud along the second direction, from the road markings included in the road surface image, and
Regarding the comparison between the road markings included in the map data and the road markings included in the road surface image, the comparison between the first line segment and the first point cloud, and the second line segment and the second point cloud. The step of calculating the vehicle position by performing the comparison processing limited to the comparison of the above is provided.
The first point cloud and the second point cloud are a set of coordinate values, and are
The first direction indicates an extension direction of the road, and the second direction is a direction substantially perpendicular to the extension direction on the road surface.
The road surface image captured by the image pickup device used in the vehicle is taken by a plurality of image pickup devices mounted on the vehicle.
Own vehicle position estimation method.

Images