KR20200120402A - Apparatus and method for obtaining estimated location of vehicle - Google Patents

Abstract

According to one embodiment, provided is a method for obtaining an estimated location of a vehicle, which comprises the steps of: estimating a location of a vehicle based on at least one of behavior information of the vehicle and GPS information of the vehicle; obtaining intersection information based on the landmark in a surrounding image of the vehicle; and correcting the estimated location of the vehicle by matching the obtained intersection information on a map.

Description

Apparatus and method for acquiring the estimated position of a vehicle {APPARATUS AND METHOD FOR OBTAINING ESTIMATED LOCATION OF VEHICLE}

The present invention relates to an apparatus and method for estimating the position of the vehicle and correcting the estimated position of the vehicle.

In general, a vehicle refers to a transportation device that travels on a road or track using fossil fuel, electricity, or the like as a power source.

Vehicles have been developed to provide various functions to drivers according to the development of technology. In particular, according to the trend of electrification of vehicles, vehicles having an active safety system (ASS) that operate to prevent accidents just before or at the moment of accidents have appeared.

Furthermore, in recent years, an advanced driver assistance system (ADAS: Advanced Driver Assist System) that actively provides information on the driving environment such as vehicle status, driver status, and surrounding environment in order to reduce the burden on the driver and improve convenience. Research on this equipped vehicle is actively underway.

Since the advanced driver assistance system operates according to the driving environment determined by the position of the own vehicle, it is necessary to accurately estimate the position of the vehicle. Conventionally, a method of estimating the location of a vehicle using a satellite signal has been widely used, but recently, research on a method of considering various information together to increase accuracy is actively underway.

Korean Patent Laid-Open Publication No. 10-2017-0097435 (published on August 28, 2017)

An object to be solved by the present invention is to provide an apparatus and method for obtaining an estimated position of a vehicle for correcting an estimated position of a vehicle by matching information on an intersection on a driving road on a map.

However, the problem to be solved by the present invention is not limited to those mentioned above, and another problem to be solved that is not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.

The method for obtaining an estimated location of a vehicle according to an embodiment of the present invention includes estimating the location of the vehicle based on at least one of vehicle behavior information and GPS information of the vehicle: a landmark in the surrounding image of the vehicle Acquiring intersection information based on; And correcting the estimated location of the vehicle by matching the obtained intersection information on a map.

In addition, correcting the estimated position of the vehicle may include: checking an intersection node corresponding to the intersection information in the map; Matching the intersection information with the identified intersection node; And correcting the estimated position of the vehicle based on the matching result.

In addition, correcting the estimated position of the vehicle based on the matching result may include first correcting the estimated position of the vehicle based on a moving distance for matching the intersection information; And second correcting the first corrected position on a link connected to the intersection node based on the traveling direction of the vehicle and the distance from the first corrected position.

In addition, the step of obtaining the intersection information based on the vehicle movement trajectory according to the estimated position of the vehicle may be further included.

In addition, it may further include the step of obtaining the intersection information by inputting the surrounding image of the vehicle to the intersection information estimator generated by learning the learning surrounding image.

In addition, the obtaining of the intersection information may include: extracting a first landmark closer to the intersection and a second landmark farther than the intersection in the surrounding image of the vehicle; Estimating positions of the first landmark and the second landmark; And acquiring the intersection information from the estimated location of the first landmark and the estimated location of the second landmark.

In addition, the first landmark may include a stop line closer than the intersection in the image surrounding the vehicle, and the second landmark may include a traffic light farther than the intersection in the surrounding image of the vehicle.

In addition, estimating the position of the first landmark may include obtaining 3D coordinates of the stop line from each of a plurality of surrounding images acquired while the vehicle is driving; Estimating the stop line candidate positions of each of the plurality of surrounding images from the 3D coordinates of each of the stop lines; And estimating the location of the stop line from the stop line candidate locations of each of the plurality of surrounding images.

In addition, estimating the position of the second landmark may include checking a pixel corresponding to the traffic light in each of a plurality of surrounding images acquired while the vehicle is driving; And estimating an intersection point of a vector passing through the identified pixel from an acquisition location of each of the plurality of surrounding images as a location of the traffic light.

In addition, the intersection information includes a center coordinate of the intersection, and obtaining the intersection information from the estimated location of the first landmark and the estimated location of the second landmark may include: The center coordinates of the intersection may be obtained based on the average of the location of the landmark and the estimated location of the second landmark.

In addition, estimating the position of the vehicle may include: estimating a first trajectory of the vehicle based on the GPS information of the vehicle; Estimating a second trajectory of the vehicle based on the behavior information of the vehicle; And estimating the position of the vehicle by comparing the first trajectory and the second trajectory.

In addition, estimating the first trajectory of the vehicle may include: checking the number of effectively received satellite signals among the GPS information; And estimating a first trajectory of the vehicle by using the received satellite signal when the number of the effectively received satellite signals is greater than or equal to the threshold number.

In addition, estimating the position of the vehicle by comparing the first trajectory and the second trajectory may include obtaining a difference between the first trajectory and the second trajectory; And estimating the position of the vehicle based on an average of the first and second trajectories when the obtained difference is less than or equal to the threshold difference.

An apparatus for obtaining an estimated location of a vehicle according to an embodiment of the present invention includes: a location estimating unit that estimates a location of the vehicle based on at least one of vehicle behavior information and GPS information of the vehicle; An intersection information acquisition unit that obtains intersection information based on a landmark in the surrounding image of the vehicle; And an estimated position correction unit that corrects the estimated position of the vehicle by matching the obtained intersection information on a map.

According to an embodiment of the present invention, it is possible to more accurately estimate the position of the vehicle using not only GPS information including satellite signals, but also vehicle behavior information and surrounding images of the vehicle. In particular, it is possible to increase the accuracy of estimating the location of a vehicle even in an area with a weak satellite signal such as an urban area or a tunnel.

In addition, by applying this to an ADAS or an autonomous vehicle, precise vehicle control is possible, and through this, driving stability and driving efficiency of the vehicle can be improved.

1 and 2 are control block diagrams of a vehicle position estimation system according to various embodiments.
3 is a flowchart of a method of obtaining an estimated position of a vehicle according to an embodiment of the present invention.
4 is a flowchart of a method of estimating a location of a vehicle according to an embodiment of the present invention.
5 is a diagram illustrating a method of estimating a position of a vehicle based on behavior information according to an embodiment of the present invention.
6 is a diagram illustrating a method of matching a first trajectory and a second trajectory according to an embodiment of the present invention.
7 is a diagram for explaining a method of correcting an estimated vehicle position using road information in a map according to an embodiment of the present invention.
8 is a flowchart of a method of obtaining intersection information according to an embodiment of the present invention.
9 is a diagram illustrating a first landmark in a surrounding image according to an embodiment of the present invention.
10 is a diagram illustrating a second landmark in a surrounding image according to an embodiment of the present invention.
11 is a diagram illustrating a method of estimating a location of a first landmark by an intersection information acquisition unit according to an embodiment of the present invention.
12 is a view for explaining a method of estimating a position of a second landmark by using two surrounding images by an intersection information acquisition unit according to an embodiment of the present invention.
13 is a diagram for explaining a method of estimating a position of a second landmark by using a plurality of surrounding images by an intersection information acquisition unit according to an embodiment of the present invention.
14 is a diagram illustrating a method of obtaining intersection information according to an embodiment of the present invention.
15 is a diagram for describing a method of obtaining intersection information according to another embodiment of the present invention.
16 is a diagram illustrating a method of correcting an estimated vehicle position according to an exemplary embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in a variety of different forms, and only these embodiments make the disclosure of the present invention complete, and are common knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to those who have, and the invention is only defined by the scope of the claims.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in an embodiment of the present invention, which may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout this specification.

1 and 2 are control block diagrams of a vehicle position estimation system according to various embodiments.

Referring to FIG. 1, a system 1 for obtaining an estimated position of a vehicle according to an exemplary embodiment may include a vehicle V and an apparatus 100 for obtaining an estimated position of the vehicle.

The vehicle V may refer to a transportation means capable of moving humans, objects, animals, etc. from one location to another while driving along a road or track. The vehicle V according to an exemplary embodiment may include a three-wheeled or four-wheeled vehicle, a two-wheeled vehicle such as a motorcycle, a construction machine, a prime mover bicycle, a bicycle, and a train running on a track.

The vehicle V of FIG. 1 includes a GPS module and may receive a satellite signal including navigation data from at least one Global Position System (GPS) satellite. The vehicle V may acquire a current location of the GPS-based vehicle V and a traveling direction of the vehicle V based on a satellite signal. In this case, the GPS module may receive a plurality of satellite signals from each of the plurality of GPS satellites.

In addition, the vehicle V of FIG. 1 may store a general map and/or a precision map in advance. Here, the general map includes road information and may be implemented as a navigation map, for example. The navigation map may mean a map including road information represented by links and nodes. Links and nodes as road information will be described later.

A precision map is a map that includes more information than a general map, and has high accuracy for safe and precise vehicle (V) control, and refers to a map that includes information on altitude, slope, curvature, etc. as well as the plane position of the road. can do. Also, the precision map may mean a map further including information on road facilities such as lanes, signs, traffic lights, and guard rails.

In addition, the vehicle V of FIG. 1 may be equipped with an Advanced Driver Assistance System (ADAS). Here, the advanced driver assistance system is a system that provides driving environment information such as vehicle (V) status, driver status, and surrounding environment information or actively controls the vehicle (V) in order to reduce the burden on the driver and improve convenience. Can mean

The advanced driver assistance system mounted on the vehicle V may include a sensing means for detecting a driving environment of the vehicle V. The sensing means according to an embodiment is a radar that detects a driving environment by irradiating a pulse around the vehicle V and receiving an echo pulse reflected from an object located in a corresponding direction, and a laser around the vehicle V. And/or a LiDAR that receives a laser reflected from an object located in a corresponding direction, and/or an ultrasonic wave is irradiated around the vehicle (V), and an echo ultrasound reflected from an object located in the corresponding direction is received. It may include an ultrasonic sensor and the like.

In addition, advanced driver assistance systems may include cameras as sensing means. The camera is provided to face the front, side, and/or rear of the vehicle V, and the surrounding images in the corresponding direction may be photographed. The captured surrounding image may be a basis for acquiring information such as lanes or signs as well as objects around the vehicle V through an image processing process.

Meanwhile, the vehicle V may include a behavior sensor capable of detecting behavior information according to driving. Here, the behavior information may include all information related to the movement of the vehicle V. The behavior sensor according to an embodiment may include a wheel speed sensor that detects a rotation speed of a front wheel and/or a rear wheel of a vehicle, and a yaw rate sensor that detects attitude information of the vehicle.

The apparatus 100 for obtaining the estimated position of the vehicle may estimate and correct the position of the vehicle V by using information received from the vehicle V. Here, the position of the vehicle may include an attitude angle as well as position coordinates.

For position estimation, the apparatus 100 for obtaining an estimated position of a vehicle may exchange information by communicating with the vehicle V through various known communication methods. The apparatus 100 for obtaining an estimated location of a vehicle according to an embodiment uses a known communication method such as CDMA, GSM, W-CDMA, TD-SCDMA, WiBro, LTE, and EPC to communicate with the vehicle V through a base station. can do. In contrast, the device 100 for obtaining an estimated location of a vehicle according to another embodiment includes a wireless LAN, Wi-Fi, Bluetooth, Zigbee, Wi-Fi Direct (WFD), By adopting a communication method such as UWB (Ultra wideband), infrared communication (IrDA; Infrared Data Association), BLE (Bluetooth Low Energy), NFC (Near Field Communication), it is also possible to communicate with the vehicle (V) within a predetermined distance. However, the method of communicating with the vehicle V by the apparatus for obtaining the estimated position of the vehicle is not limited to the above-described embodiment.

Meanwhile, the vehicle may be equipped with a navigation system that provides the driver with map information and route information to a destination. The navigation system not only matches and provides route information with map information, but also provides additional information such as the location of a speeding camera in the map information to help the driver's safe driving.

The navigation system may estimate a vehicle position using a satellite signal received from a GPS satellite. The navigation map used by the navigation system consists of links and nodes.The navigation system checks a link on the navigation map similar to the direction of the vehicle based on the received satellite signal, and is based on the received satellite signal among the identified links. The location of the vehicle can be estimated by matching the location of the vehicle with the closest link.

In this case, if the distance between the location of the vehicle based on the satellite signal and the adjacent link is less than or equal to the threshold value, the navigation system determines that the finally estimated vehicle location is valid, but if the distance exceeds the threshold value, it may determine that the route is deviated. However, in places where the strength of the satellite signal is weak, such as an urban center or a tunnel, the navigation system misrecognizes the location of a satellite signal-based vehicle or incorrectly identifies adjacent links, thereby significantly lowering the vehicle's location accuracy or repetitively. It can be judged as a departure.

In order to solve this, the apparatus 100 for obtaining an estimated location of a vehicle may correct the estimated location of the vehicle V by matching information of an intersection on a driving road on a map, in particular, a node of a navigation map.

To this end, the apparatus 100 for obtaining an estimated position of a vehicle includes a position estimating unit 110 for estimating the position of the vehicle based on the vehicle behavior information and GPS information of the vehicle; An intersection information acquisition unit 120 that obtains intersection information based on a landmark in the surrounding image of the vehicle; And an estimated position correction unit 130 for correcting the estimated position of the vehicle by matching the obtained intersection information on a map.

Meanwhile, in FIG. 1, a case in which the vehicle estimated position obtaining device 100 is provided separately from the vehicle V to form the vehicle estimated position obtaining system 1 is illustrated, but unlike this, the vehicle estimated position obtaining device It may be possible for 100 to be provided inside the vehicle V.

Referring to FIG. 2, a system 1 for obtaining an estimated position of a vehicle according to another exemplary embodiment may be composed of a vehicle V including an apparatus 100 for obtaining an estimated position of a vehicle. However, except for the method in which the apparatus 100 for obtaining the estimated position of the vehicle is provided, the operation method of the system 1 for obtaining the estimated position of the vehicle of FIG. 2 and the system for obtaining the estimated position of the vehicle 1 of FIG. 2 is the same. Do.

Hereinafter, a method of obtaining an estimated position of a vehicle according to an exemplary embodiment of the present invention will be described with reference to FIG. 3, focusing on the operation of each component of the above-described apparatus 100 for obtaining an estimated position of a vehicle.

3 is a flowchart of a method of obtaining an estimated position of a vehicle according to an embodiment of the present invention.

First, the apparatus 100 for obtaining an estimated position of the vehicle may estimate the position of the vehicle V based on the behavior information of the vehicle V and the GPS information of the vehicle V (S100). Specifically, the position estimation unit 110 of the apparatus 100 for obtaining the estimated position of the vehicle V calculates the movement trajectory of the vehicle V based on the behavior information of the vehicle V and the GPS information of the vehicle V. It is obtained, and the position of the vehicle V may be estimated based on this.

Hereinafter, a method of estimating the position of the vehicle V will be described with reference to FIG. 4.

4 is a flowchart of a method of estimating a location of a vehicle according to an embodiment of the present invention.

First, the location estimating unit 110 may estimate the first trajectory of the vehicle V based on GPS information of the vehicle V (S110). To this end, the location estimating unit 110 may use GPS information obtained from a plurality of viewpoints by the vehicle V being driven. After estimating the position of the vehicle V for each of the plurality of GPS information, the position estimating unit 110 may estimate a first trajectory of the vehicle V from the position of the vehicle V.

Here, the GPS information may include a satellite signal received by a GPS module mounted on the vehicle V from a GPS satellite, and the satellite signal may include a GPS module, that is, the position coordinates of the vehicle V. At this time, the location estimating unit 110 may check the number of satellite signals effectively received by the GPS module. If the number of effectively received satellite signals is greater than or equal to the threshold number, the position estimation unit 110 may determine that the satellite signal is valid and use it to estimate the first trajectory.

Then, the location estimating unit 110 may estimate the second trajectory of the vehicle V based on the behavior information of the vehicle V (S120). To this end, the position estimating unit 110 may use the behavior information obtained by the vehicle V being driven at a plurality of viewpoints. In particular, each acquisition time point of the plurality of behavior information may be the same as each of the plurality of GPS information acquisition time points, or a plurality of motion information acquisition time points may belong within a time interval formed by the plurality of GPS information acquisition time points.

After estimating the position of the vehicle V for each of the plurality of motion information, the position estimating unit 110 may estimate a second trajectory of the vehicle V from the position of the vehicle V. Hereinafter, a method of estimating the position of the vehicle V based on the behavior information will be described with reference to FIG. 5.

5 is a diagram illustrating a method of estimating a position of a vehicle based on behavior information according to an embodiment of the present invention.

Referring to FIG. 5, when a vehicle (v) with a width b located at the origin O at the k time curve travels and is located at A at the k+1 time point, the k+1 time point based on the detected value of the wheel speed sensor. The location of the vehicle (v) can be obtained.

If the moving distance S _l of the left wheel of the vehicle v and the moving distance S _r of the right wheel are provided by the wheel speed sensor, the position of the vehicle v at point k+1 (x', y') Can be estimated according to Equation 1.

Here, the matrix [x(k), y(k), θ(k)] means the position and direction of the vehicle v at the point k, and S _r is the speed of the right wheel of the vehicle v The moving distance according to S _l may mean a moving distance according to the speed of the left wheel of the vehicle v.

Further, Δθ, which is the amount of change in θ between the k time point and the k+1 time point, may be obtained by the yaw rate value.

Referring back to FIG. 4, the position estimating unit 110 may estimate the position of the vehicle V based on the first trajectory and the second trajectory (S130). To this end, the position estimation unit 110 may compare the first trajectory and the second trajectory. Hereinafter, a method of estimating the position of the vehicle V by comparing the first trajectory and the second trajectory will be described with reference to FIG. 6.

6 is a diagram illustrating a method of matching a first trajectory and a second trajectory according to an embodiment of the present invention.

Unlike GPS information, the position of the vehicle V, which is estimated based on the behavior information of the vehicle V, may not be able to estimate the absolute position. However, for a short moving distance of the vehicle V of several hundred m or less, the position of the vehicle V can be relatively accurately estimated. Therefore, the position estimation unit 110 compares the first trajectory estimated based on the GPS information with the second trajectory estimated based on the behavior information, thereby verifying the validity of the position of the vehicle V estimated from the GPS information. I can.

Specifically, the position estimation unit 110 may obtain a difference by comparing the first trajectory and the second trajectory. Referring to FIG. 6, the position estimation unit 110 may match a first trajectory estimated based on GPS information and a second trajectory estimated based on the behavior information, and obtain an error according to the matching result.

Then, when the obtained difference is equal to or less than the threshold difference, the position estimating unit 110 may estimate the position of the vehicle V using the first trajectory and the second trajectory. Here, the critical difference may mean a maximum difference between two trajectories when the first trajectory is determined to be valid. That is, the position estimating unit 110 may estimate the position of the vehicle V using the two trajectories only when the first trajectory is valid.

When the difference is less than or equal to the threshold difference, the position estimating unit 110 may estimate the position of the vehicle V based on an average of the first and second trajectories. Specifically, the position estimating unit 110 may estimate the position [x _k+1 , y _k+1 , θ _k+1 ] of the vehicle V at a time point k+1 according to Equation 2.

Here, x _{k+1 (gps)} and y _{k+1 (gps)} refer to the x-axis and y-axis position coordinates of the vehicle V estimated based on GPS information at the time point k+1, and θ _{k+1 (gps)} means the driving direction of the vehicle (V) estimated based on GPS information at the time k+1, and x _k+1(dr) and y _k+1(dr) are the behavior information at the time k+1 Refers to the x-axis and y-axis position coordinates of the vehicle V estimated based on, and θ _k+1(dr) means the traveling direction of the vehicle V estimated based on the behavior information at the time k+1. can do. In addition, w is a weight, and may be determined as 0.5 if the difference between the two trajectories is less than or equal to the threshold difference, and 0 if not.

That is, when the first trajectory is valid, the position estimating unit 110 may estimate the position of the vehicle V based on the average of the first trajectory and the second trajectory. On the other hand, when the first trajectory is not valid, the position estimating unit 110 may estimate the position of the vehicle V using only the second trajectory, that is, behavior information.

In addition, the location estimation unit 110 may correct the estimated location of the vehicle V using a map. Hereinafter, a method of correcting the estimated position of the vehicle V using road information in the map will be described with reference to FIG. 7.

7 is a diagram for explaining a method of correcting an estimated vehicle position using road information in a map according to an embodiment of the present invention.

As described above, the vehicle V may store a map in advance, and the stored map may include a general map and/or a precision map. In addition, a general map according to an embodiment may be implemented as a navigation map, and the navigation map may include road information represented by links and nodes.

Referring to FIG. 7, a node N denotes a location where a road is connected to a road, and may include an intersection, a junction, and the like. In addition, the links L _a and L _b connected through the node N represent a shape of a road and may include information such as a traveling direction of the road and an allowable speed.

The position estimating unit 110 may obtain a movement trajectory of the vehicle V based on the continuously estimated position of the vehicle V. Since the moving trajectory obtained as described above includes information on the moving direction of the vehicle V, the location estimating unit 110 may search for a link similar to the moving direction information of the moving trajectory in the map.

Then, the position estimating unit 110 may correct the position of the vehicle V from the estimated position of the vehicle V to the nearest link among the searched links. In FIG. 7, when the position estimating unit 110 searches for _a link L _a as a link adjacent to the position of the vehicle V, the estimated position P _G1 of the vehicle V is corrected to the position P _G2 on the link L _a . Illustrate that.

At this time, when the distance from the estimated location P _G1 of the vehicle V to the location P _G2 to be corrected exceeds the threshold distance, the location estimating unit 110 may determine the location of the estimated vehicle V. It can be determined that the position P _G1 of V) is not valid. In this case, the position estimating unit 110 may reestimate the position of the vehicle V based on the route deviation.

Referring back to FIG. 3, after estimating the position of the vehicle V according to the above-described method, the apparatus 100 for obtaining the estimated position of the vehicle V is an intersection based on the landmark in the surrounding image of the vehicle V. Information can be obtained (S200). Specifically, the intersection information acquisition unit 120 of the apparatus 100 for obtaining the estimated position of the vehicle V extracts a landmark in the surrounding image obtained by the camera of the vehicle V, and obtains intersection information from the landmark. I can. Hereinafter, a method of obtaining intersection information will be described with reference to FIG. 8.

8 is a flowchart of a method of obtaining intersection information according to an embodiment of the present invention.

First, the intersection information acquisition unit 120 may extract the first landmark and the second landmark from the surrounding image of the vehicle V (S210). Hereinafter, definitions of the first landmark and the second landmark will be described with reference to FIGS. 9 and 10.

9 is a diagram illustrating a first landmark in a surrounding image according to an embodiment of the present invention, and FIG. 10 is a diagram illustrating a second landmark in a surrounding image according to an embodiment of the present invention. In FIGS. 9 and 10, the surrounding image is a front image of the front of the vehicle V.

Referring to FIG. 9, an intersection may exist in front of the vehicle V. In this case, the first landmark may mean a landmark located on the intersection entry path of the vehicle V facing the intersection. The first landmark according to an embodiment may include a stop line L _M1 on a road.

In addition, referring to FIG. 10, the second landmark may mean a landmark located on an exit path of the vehicle V facing the intersection. The second landmark according to an embodiment may include a traffic light L _M2 .

Hereinafter, for convenience of description, it is assumed that the first landmark is a stop line and the second landmark is a traffic light.

The intersection information acquisition unit 120 may use a landmark learner generated in advance to extract the first landmark and the second landmark in the surrounding image. The landmark learner is generated by learning the learning surrounding image through a machine learning method such as deep learning, and when the surrounding image is input, the first landmark and the second landmark in the surrounding image may be extracted as outputs.

Then, the intersection information acquisition unit 120 may estimate the positions of the first landmark and the second landmark (S220). In this case, the intersection information acquisition unit 120 may estimate locations of the first landmark and the second landmark according to different methods.

Hereinafter, a method of estimating a position of a first landmark will be described with reference to FIG. 11, and then a method of estimating a position of a second landmark will be described with reference to FIGS. 12 and 13.

11 is a view for explaining a method of estimating a location of a first landmark by an intersection information acquisition unit according to an embodiment of the present invention, and FIG. 12 is a diagram illustrating a method of estimating the location of a first landmark by an intersection information acquisition unit according to an embodiment of the present invention. A diagram for explaining a method of estimating the location of a second landmark using an image, and FIG. 13 is a diagram for an intersection information acquisition unit according to an embodiment of the present invention to determine the location of a second landmark by using a plurality of surrounding images. It is a figure for demonstrating the method of estimation.

In order to obtain the position Pw of the first landmark with respect to the stop line, the intersection information acquisition unit 120 may assume that the ground is flat. Then, the pixel coordinate m corresponding to both end points of the stop line in the surrounding image may be converted to the coordinate m _new when the camera faces the front. Specifically, the intersection information acquisition unit 120 may obtain coordinates m _new of both end points of the stop line converted according to the image coordinate system when the camera C faces the front according to Equation 3.

Here, K denotes a calibration matrix related to internal parameters of the camera C assuming a pin-hole model, and R _ideal denotes the pixel coordinates in the surrounding image when the camera C faces the front. Denotes a rotation matrix transformed into an image coordinate system of, and m may mean pixel coordinates corresponding to both end points of the stop line in the surrounding image.

Then, the intersection information acquisition unit 120 may obtain the angle θ _p from the stop line to the ground to the camera C using Equation 4.

Here, (u _new ,v _new ) means the x,y values of the coordinates m _new of both end points of the stop line transformed by Equation 3, and (u ₀ ,v ₀ ) means the principal point of the camera can do.

After obtaining the angle θ _p , the intersection information obtaining unit 120 obtains a three-dimensional position [Xc, Yc, Zc] corresponding to both end points of the reference stop line of the camera coordinate system when the camera faces the front.

Here, h refers to the height of the camera (C) mounting from the ground, z refers to the distance to the stop line when the position of the camera (C) is projected onto the ground, and f _x and f _y refers to the focal length. I can.

Finally, the intersection information acquisition unit 120 uses the external parameters between the camera C and the vehicle V and the position and movement direction of the vehicle V estimated by the position estimation unit 110 to [X _c , From Y _c ,Z _c ], P _W [X _w ,Y _w ,Z _w ], which is the three-dimensional position of the stop line based on the absolute coordinate system, can be obtained.

In addition, when the vehicle V heading for the intersection continuously acquires surrounding images while driving on the stop line, the intersection information acquisition unit 120 may obtain 3D coordinates based on the absolute coordinate system of the stop line from each surrounding image. In this case, when the obtained 3D coordinates of the stop line are both end points of the stop line, the intersection information acquisition unit 120 may estimate the average of the 3D coordinates as a candidate position of the stop line. Then, the intersection information acquisition unit 120 may estimate the average of the obtained stop line candidate locations or the candidate location of the stop line estimated from the last obtained surrounding image as the location of the stop line.

In addition, in order to estimate the location of the second landmark, the intersection information acquisition unit 120 may use triangulation. Specifically, the intersection information acquisition unit 120 extracts the same second landmark from at least two surrounding images captured at different locations, and estimates the three-dimensional position of the second landmark by applying triangulation to the extraction result. can do.

Referring to FIG. 12, the camera captures a second landmark at a first position L ₁ to obtain a first surrounding image I ₁ , and captures a second landmark at a second position L ₂ to obtain a second surrounding image I _2. Can be obtained. At this time, the intersection information acquisition unit 120 obtains a vector V ₁ passing through the pixel X ₁ corresponding to the second landmark in the first surrounding image I ₁ from the first position, and the second surrounding image from the second position L ₂ . A vector V ₂ passing through the pixel X ₂ corresponding to the second landmark in I ₂ can be obtained. Then, the intersection information acquisition unit 120 may estimate a point where the vectors V ₁ and V ₂ intersect as the second landmark position P in the 3D space.

When estimating the position of the second landmark according to this method, the accuracy of the determined position of the second landmark may be affected by the number of surrounding images from which the second landmark is extracted. As described above, in order to estimate the position of the second landmark in the 3D space, at least two peripheral images are used, and the average of the plurality of 3D positions estimated from the plurality of surrounding images is calculated as the second in the 3D space. Measurement errors can be reduced by estimating the location of the landmark.

In addition, as the distance between the plurality of locations where a plurality of surrounding images are captured by the camera increases, the 3D location accuracy of the second landmark estimated according to the above method may increase. This is because, as the distance between the photographing locations increases, the pixel error of the second landmark extracted from the surrounding image decreases, and the 3D position error of the second landmark determined based on this decreases. For example, the position error of the second landmark in two surrounding images taken at two locations separated by 1m is 1 pixel, and the location of the second landmark in two surrounding images taken at two locations separated by 2m When the error is 1 pixel, the 3D position of the second landmark determined based on each is highly accurate in the latter case.

Referring to FIG. 13, a surrounding image may be captured at a plurality of locations 710, 720, and 730, respectively, and an error range of the position of the second landmark estimated by the surrounding image captured at the first location 710 Although 711 is relatively large, the error range 721 of the second landmark position estimated by considering the surrounding image at the second position 720 is smaller, and considers the surrounding image at the third position 730 Thus, it can be confirmed that the error range 731 of the estimated second landmark position becomes smaller.

Alternatively, the intersection information acquisition unit 120 may estimate the location of the second landmark and use only the error range for the estimated location of the second landmark, that is, the covariance of the location of the second landmark is less than or equal to a predetermined threshold.

The position of the 3D second landmark can be more accurately estimated using a Kalman filter when the covariance is given. In this process, Equation 6 below may be used.

Here, d denotes a three-dimensional direction vector from the lens of the camera to the position of the second landmark, λ is a constant for normalization purposes that makes d=(a, b, c) ^T as a unit vector, and R May mean a 3D rotation matrix representing the orientation of the camera. Also, K may mean a calibration matrix related to internal parameters of the camera assuming a pinhole model. In addition, P ^* represented by 3D coordinates can be obtained according to Equations 7 to 9 as follows.

Here, (x _i , y _i , z _i ) may mean the i-th position among a plurality of positions of the camera. The covariance of the 3D position P of the second landmark estimated based on the above equation is A ^-1 , which is the 3D position of the second landmark in the surrounding image captured at the first (i=1) shooting position. Indicates an error.

In addition, in the case of estimating the three-dimensional positions of a plurality of candidate second landmarks according to the above-described method, the intersection information acquisition unit 120 may be May be determined as the 3D position of the second landmark.

Referring to FIG. 8 again, after estimating the positions of the first landmark and the second landmark, the intersection information acquisition unit 120 retrieves intersection information from the estimated positions of the first landmark and the second landmark. It can be obtained (S230). Here, the intersection information means information related to the location of the intersection, and the intersection information according to an embodiment may include a center coordinate of the intersection.

The intersection information acquisition unit 120 according to an embodiment may obtain the center coordinates of the intersection based on an average of the estimated location of the first landmark and the estimated location of the second landmark. Hereinafter, a method of obtaining intersection information according to an embodiment will be described with reference to FIG. 14.

14 is a diagram illustrating a method of obtaining intersection information according to an embodiment of the present invention.

As described above, the first landmark may exist on the entry path of the intersection, and the second landmark may exist on the exit path of the intersection. Specifically, the representative position of the intersection may be determined by a relationship between the position of the first landmark and the position of the second landmark within the area determined between the first landmark and the second landmark. Here, the representative location may mean a location that can be expressed in one coordinate to map an intersection on a map. If the apparatus 100 for obtaining an estimated location of a vehicle according to an embodiment of the present invention uses a navigation map, the coordinates of the intersection center corresponding to the intersection node on the navigation map may be assumed as the representative location of the intersection.

If the representative location of the intersection is assumed to be the intersection center coordinate, the intersection information acquisition unit 120 is an intersection between the location of the first landmark and the location of the second landmark according to the type of the first landmark and the second landmark. The center coordinates can be determined. If the first landmark means a stop line and the second landmark means a traffic light, the intersection information acquisition unit 120 is an average of the estimated location of the first landmark and the estimated location of the second landmark. Can be determined as the coordinates of the center of the intersection.

14 illustrates a case in which an estimated position P _L of a stop line is obtained as a first landmark and an estimated position P _S of a traffic light is obtained as a second landmark. The intersection information acquisition unit 120 may obtain an intersection center coordinate, which is a representative location of the intersection, based on the estimated position P _L of the stop line and the estimated position P _s of the traffic light. When the stop line and the traffic light exist on the links in opposite directions connected to one node on the navigation map, the intersection information acquisition unit 120 includes a first link where the estimated stop line P _L exists so as to correspond to the navigation map. And a second link in which the estimated position P _s of a traffic light exists, and a position at which the first link and the second link are connected may be determined as the center coordinate P _c of the intersection. In particular, the intersection information acquisition unit 120 according to an embodiment may determine an average of the first link where the estimated position P _L of the stop line exists and the estimated position Ps of the traffic light as the center coordinate P _c of the intersection.

In contrast, the intersection information acquisition unit 120 may obtain intersection information without using the previously estimated positions of the first landmark and the second landmark. Hereinafter, a method of obtaining intersection information according to another embodiment will be described with reference to FIG. 15.

15 is a diagram for describing a method of obtaining intersection information according to another embodiment of the present invention.

The intersection information acquisition unit 120 according to another embodiment may obtain intersection information based on the estimated location and movement trajectory of the vehicle V. To this end, the intersection information acquisition unit 120 may first determine whether the vehicle V turns left or right. When matching the estimated location of the vehicle V with a link on the map, if the traveling direction of the vehicle V estimated from the matched link is different by more than a threshold value, the intersection information acquisition unit 120 may move the vehicle ( It can be determined that V) is rotating.

If the vehicle V turns left or right, the intersection information acquisition unit 120 fits the moving trajectory of the vehicle V into two straight lines, and determines the intersection of the fitted straight lines as the intersection center coordinates. I can. Referring to FIG. 15, the intersection information acquisition unit 120 is a vehicle V from a position P _C1 , P _C2 , P _C3 , P _C4 , P _C5 , and P _C6 sequentially estimated at a plurality of viewpoints. We can obtain the movement trajectory of and fit it into two straight lines L _d1 and L _d2 . Then, the intersection information acquisition unit 120 may determine the intersection PC of the two straight lines L _d1 and L _d2 as the center coordinates of the intersection.

In addition, the intersection information acquisition unit 120 according to another embodiment may acquire intersection information by inputting an image around the vehicle V into an intersection information estimator generated by learning a learning surrounding image. To this end, the intersection information estimator may be generated in advance and stored in the apparatus 100 for obtaining an estimated position of the vehicle V or an external device capable of communicating with the vehicle V.

In order to generate the intersection information estimator, a plurality of intersections and surrounding images other than the intersection may be used as the learning surrounding images. When one learning surrounding image is input, the intersection information estimator composed of an artificial neural network outputs a probability value to be recognized as an intersection, and the loss function representing the error between the output data and the correct answer data by the artificial neural network It is possible to calculate the output value and modify the parameters of the artificial neural network to minimize the output value of the loss function through Back Propagation theory. By repeatedly performing this process for all the learning surrounding images, intersection recognition may be learned so that the sum of output values of the loss function for all input images is minimized.

When an intersection is recognized based on this, deep learning learning for estimating a relative distance to the center of the intersection may be performed. Specifically, when one learning surrounding image including an intersection is input, the intersection information estimator composed of an artificial neural network estimates the relative distance from the vehicle V to the intersection center position, and outputs a probability value for the estimated distance. It is possible to calculate the output value for the loss function representing the error between the output data by the intersection information estimator and the correct answer data, and modify the parameters of the artificial neural network to minimize the output value of the loss function through Back Propagation theory. This process is repeatedly performed for all the learning surrounding images, so that the sum of the output values of the loss function for all input images can be minimized, and the estimation of the relative distance to the center of the intersection can be learned.

Then, after the intersection is recognized from the surrounding image and the relative distance from the intersection center position is estimated, the intersection information acquisition unit 120 may obtain an absolute coordinate for the intersection center position from this. Specifically, the intersection information acquisition unit 120 may estimate the absolute coordinate P _w1 with respect to the center of the intersection by using Equation 10.

Here, P _w1,x and P _w1,y are absolute coordinates for the center of the intersection, and x _k(vehicle) , x _k(vehicle) , θ _k(vehicle) are the estimated location and direction of the vehicle (V). It means, and d may mean a relative distance from the vehicle V to the center point of the intersection.

If there are a plurality of acquired surrounding images, when the probability of the intersection recognition of the intersection information estimator is greater than or equal to the threshold, the intersection information acquisition unit 120 uses the case where the probability of estimating the relative distance from the intersection center position is maximum. Absolute coordinates of the intersection center point can be obtained.

On the contrary, if the probability of the intersection recognition of the intersection information estimator is greater than or equal to the threshold, the intersection information acquisition unit 120 places the probability of estimating the relative distance from the intersection center position as a weight, and the average of the weights using Equation 11 The weighted average P _w1,avg can be calculated.

Here, w _sum is the _sum of all weights, n is the number of the intersection center position, and w(i) is the weight of the absolute position Pw1(i) with respect to the i-th intersection center.

Referring back to FIG. 3, finally, the apparatus 100 for obtaining the estimated location of the vehicle V may correct the estimated location of the vehicle V by matching the obtained intersection information on the map (S300). Specifically, the estimated position correction unit 130 of the apparatus 100 for obtaining the estimated position of the vehicle V matches the coordinates of the intersection center to the node on the map, and corrects the estimated position of the vehicle V based on the matching result. can do.

Hereinafter, a method of correcting the estimated position of the vehicle V will be described with reference to FIG. 16.

16 is a diagram illustrating a method of correcting an estimated vehicle position according to an exemplary embodiment of the present invention.

The estimated position correction unit 130 may check the intersection node corresponding to the intersection information in the map, match the intersection information with the confirmed intersection node, and then correct the estimated position of the vehicle V based on the matching result. have. Specifically, the estimated position correction unit 130 first corrects the position of the vehicle V estimated based on the moving distance for matching the intersection information, and then the moving direction and the first corrected position of the vehicle V The first corrected position on the link connected to the intersection node may be second corrected based on the distance from.

Referring to FIG. 16, the estimated position correction unit 130 may match the intersection center coordinate P _C with the intersection node P _{CM which} is the nearest node on the map. Then, the estimated position correction unit 130 may acquire coordinate movement amounts Δx and Δy for matching intersection information. That is, the estimated position correction unit 130 may obtain coordinate movement amounts Δx and Δy for matching intersection information from a difference between the intersection center coordinate P _C and the intersection node P _CM .

Based on this, the estimated position correction unit 130 may first correct the estimated vehicle V position P _P according to Δx and Δy. For example, the estimated position correction unit 130 moves the x coordinate among the estimated position coordinates of the vehicle V by Δx, and moves the y coordinate among the estimated position coordinates of the vehicle V by Δy. The position P _P of the vehicle V may be first corrected.

Finally, the estimated position correction unit 130 uses the vehicle V to the position P _P on the link closest to the position of the first corrected vehicle V while the driving direction of the vehicle V and the driving direction of the road are similar. The position of can be corrected for the second time. Since the vehicle traveling on the road is highly likely to be located on a link corresponding to the road on the navigation map, the estimated position correction unit 130 estimates the position of the first corrected vehicle V by second correcting the position of the adjacent link. Can increase the accuracy of If the location of the first corrected vehicle V exists on the link of the map, the second correction may be omitted.

The apparatus and method for obtaining an estimated position of a vehicle according to the above-described embodiment may more accurately estimate the position of a vehicle using not only GPS information including satellite signals, but also vehicle behavior information and surrounding images of the vehicle. In particular, it is possible to increase the accuracy of estimating the location of a vehicle even in an area with a weak satellite signal such as an urban area or a tunnel.

In addition, by applying this to an ADAS or an autonomous vehicle, precise vehicle control is possible, and thus driving stability and driving efficiency of the vehicle can be improved.

Meanwhile, each step included in the method for obtaining the estimated position of the vehicle according to the above-described exemplary embodiment may be implemented in a computer-readable recording medium that records a computer program programmed to perform this step.

Further, each step included in the method for obtaining the estimated position of the vehicle according to the above-described exemplary embodiment may be implemented as a computer program programmed to perform such a step.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential quality of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain the technical idea, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

According to an embodiment, the above-described vehicle location estimation apparatus, vehicle location estimation method, and a computer-readable recording medium storing a computer program programmed to perform such a method may be used in various fields such as indoors or industrial sites. Therefore, there is a possibility of industrial use.

1: Vehicle estimated position acquisition system
100: a device for obtaining an estimated position of a vehicle
V: vehicle

Claims (16)

Estimating the location of the vehicle based on at least one of the vehicle behavior information and the vehicle GPS information:
Acquiring intersection information based on the landmark in the surrounding image of the vehicle; And
Comprising the step of correcting the estimated location of the vehicle by matching the obtained intersection information on a map
How to obtain the estimated location of the vehicle. The method of claim 1,
Correcting the estimated position of the vehicle,
Identifying an intersection node corresponding to the intersection information in the map;
Matching the intersection information with the identified intersection node; And
Comprising the step of correcting the estimated position of the vehicle based on the matching result
How to obtain the estimated location of the vehicle. The method of claim 2,
Correcting the estimated position of the vehicle based on the matching result,
First correcting the estimated position of the vehicle based on a moving distance for matching the intersection information; And
And second correcting the first corrected position on a link connected to the intersection node based on the traveling direction of the vehicle and the distance from the first corrected position.
How to obtain the estimated location of the vehicle. The method of claim 1,
Further comprising the step of obtaining the intersection information based on the vehicle movement trajectory according to the estimated vehicle position
How to obtain the estimated location of the vehicle. The method of claim 1,
Further comprising the step of obtaining the intersection information by inputting the surrounding image of the vehicle into an intersection information estimator generated by learning the learning surrounding image
How to obtain the estimated location of the vehicle. The method of claim 1,
The step of obtaining the intersection information,
Extracting a first landmark closer to the intersection and a second landmark farther than the intersection in the surrounding image of the vehicle;
Estimating positions of the first landmark and the second landmark; And
Comprising the step of obtaining the intersection information from the estimated location of the first landmark and the estimated location of the second landmark
How to obtain the estimated location of the vehicle. The method of claim 6,
The first landmark,
Including a stop line closer than the intersection in the surrounding image of the vehicle,
The second landmark,
Including a traffic light farther than the intersection in the surrounding image of the vehicle
How to obtain the estimated location of the vehicle. The method of claim 7,
The step of estimating the location of the first landmark,
Acquiring 3D coordinates of the stop line from each of a plurality of surrounding images acquired while the vehicle is driving;
Estimating the stop line candidate positions of each of the plurality of surrounding images from the 3D coordinates of each of the stop lines; And
And estimating the position of the stop line from the stop line candidate positions of each of the plurality of surrounding images
How to obtain the estimated location of the vehicle. The method of claim 7,
The step of estimating the location of the second landmark,
Checking a pixel corresponding to the traffic light in each of a plurality of surrounding images acquired while the vehicle is driving; And
Comprising the step of estimating an intersection point of a vector passing through the identified pixel from an acquisition position of each of the plurality of surrounding images as a position of the traffic light
How to obtain the estimated location of the vehicle. The method of claim 7,
The intersection information,
Including the coordinates of the center of the intersection,
Obtaining the intersection information from the estimated location of the first landmark and the estimated location of the second landmark,
Obtaining the center coordinates of the intersection based on the average of the estimated location of the first landmark and the estimated location of the second landmark
How to obtain the estimated location of the vehicle. The method of claim 1,
The step of estimating the location of the vehicle,
Estimating a first trajectory of the vehicle based on the GPS information of the vehicle;
Estimating a second trajectory of the vehicle based on the behavior information of the vehicle; And
Comprising the step of estimating the position of the vehicle by comparing the first trajectory and the second trajectory
How to obtain the estimated location of the vehicle. The method of claim 11,
The step of estimating the first trajectory of the vehicle,
Checking the number of effectively received satellite signals among the GPS information; And
When the number of the effectively received satellite signals is greater than or equal to a threshold number, estimating a first trajectory of the vehicle using the received satellite signals.
How to obtain the estimated location of the vehicle. The method of claim 11,
The step of estimating the position of the vehicle by comparing the first trajectory and the second trajectory,
Acquiring a difference between the first trajectory and the second trajectory; And
If the obtained difference is less than or equal to a critical difference, estimating the position of the vehicle based on an average of the first and second trajectories.
How to obtain the estimated location of the vehicle. A position estimation unit estimating a position of the vehicle based on at least one of vehicle behavior information and GPS information of the vehicle;
An intersection information acquisition unit that obtains intersection information based on a landmark in the surrounding image of the vehicle; And
Comprising an estimated position correction unit for correcting the estimated position of the vehicle by matching the obtained intersection information on a map
A device for obtaining the estimated position of the vehicle. A computer-readable recording medium on which a program including instructions for performing each step according to the method according to any one of claims 1 to 13 is recorded. A program stored on a computer-readable recording medium programmed to perform each step according to the method according to any one of claims 1 to 13.

Images