KR102454599B1 - Method and apparatus for determining the position of an object on a lane map - Google Patents

Abstract

The present disclosure relates to a method and an apparatus for determining the position of an object on a lane map. In the method according to an embodiment of the present disclosure, image data captured by a camera mounted on a vehicle can be obtained. In addition, in the method, predetermined processing is performed on the image data to generate feature information about a plurality of features included in the image data. Furthermore, in the method, a plurality of features corresponding to the same object can be determined from a plurality of pieces of image data based on the correlation between the plurality of features included in the image data. In addition, in the method, the position of an object on a lane map can be determined based on the position information of a vehicle on the lane map and the feature position information on the plurality of pieces of image data.

Description

METHOD AND APPARATUS FOR DETERMINING THE POSITION OF AN OBJECT ON A LANE MAP

The present invention relates to a method and apparatus for determining the location of an object on a road map.

A vehicle traveling on a road may drive on any one of a plurality of lanes included in the road. A vehicle may frequently change lanes while driving, and a situation in which the number of lanes on a road is changed frequently also occurs.

Due to the convergence of information and communication technology and the vehicle industry, the smartization of vehicles is rapidly progressing. Due to smartization, vehicles are evolving from simple mechanical devices to smart cars, and autonomous driving is attracting attention as a core technology of smart cars. Autonomous driving is a technology that allows the vehicle to reach its destination on its own without the driver operating the steering wheel, accelerator pedal, or brake.

Various additional functions related to autonomous driving are continuously being developed, and research on how to provide a safe autonomous driving experience to passengers by recognizing and judging the driving environment using various data and controlling the vehicle is required.

Recently, research is needed to more accurately display various objects (for example, traffic lights, road signs, bumps, etc.) located on or around the road on which the vehicle travels on the map with less computation. .

The above-mentioned background art is technical information possessed by the inventor for the derivation of the present invention or acquired in the process of derivation of the present invention, and cannot necessarily be said to be a known technique disclosed to the general public prior to the filing of the present invention.

SUMMARY OF THE INVENTION The present invention is to provide a method and apparatus for determining the location of an object on a road map. The problem to be solved by the present invention is not limited to the above-mentioned problems, and other problems and advantages of the present invention not mentioned can be understood by the following description, and more clearly understood by the embodiments of the present invention will be In addition, it will be appreciated that the problems and advantages to be solved by the present invention can be realized by means and combinations thereof indicated in the claims.

As a technical means for achieving the above-described technical problem, a first aspect of the present disclosure is a method for determining a position of an object on a road map, the method comprising: acquiring image data captured by a camera mounted on a vehicle; generating feature information for a plurality of features included in the image data by performing predetermined processing on the image data; determining a plurality of features corresponding to the same object in a plurality of image data based on a degree of correlation between the plurality of features included in the image data; and determining a location of an object on the road map based on location information of the vehicle on the road map and feature location information on the plurality of image data.

A second aspect of the present disclosure provides an apparatus for determining a location of an object on a road map, comprising: a memory in which at least one program is stored; and a processor that performs an operation by executing the at least one program, wherein the processor acquires image data captured by a camera mounted on a vehicle, and performs predetermined processing on the image data to obtain the image data Generates feature information for a plurality of features included in , and based on a degree of correlation between the plurality of features included in the image data, a plurality of features corresponding to the same object in the plurality of image data and determining the location of the object on the road map based on the location information on the lane map of the vehicle and the feature location information on the plurality of image data.

A third aspect of the present disclosure may provide a computer-readable recording medium recording a program for executing the method according to the first aspect on a computer.

In addition to this, another method for implementing the present invention, another system, and a computer-readable recording medium storing a computer program for executing the method may be further provided.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to the problem solving means of the present disclosure described above, it is possible to reduce data weight by determining which object is the same in a plurality of image data by using feature information corresponding to metadata about the object without using the entire image data. do.

1 to 3 are diagrams for explaining an autonomous driving method according to an exemplary embodiment.
4 is a diagram for explaining an example of a plurality of objects included in image data according to an exemplary embodiment.
5A to 5B are diagrams for explaining feature information included in image data according to an exemplary embodiment.
6A to 6B are exemplary diagrams for describing feature merging in image data according to an embodiment.
7 is an exemplary diagram for describing a method of determining a location of an object on a road map according to an exemplary embodiment.
8 is an exemplary view for explaining a method of determining a position of an object based on position information of a vehicle and a position of a feature on an image according to an embodiment.
9 is an exemplary diagram for describing a method of determining a location of an object on a road map using a plurality of driving trajectories, according to an exemplary embodiment.
10 is an exemplary diagram for describing a vehicle fitting method and an object fitting method according to an exemplary embodiment.
11 is a flowchart of a method of determining a location of an object on a road map according to an exemplary embodiment.
12 is a block diagram of an object positioning apparatus according to an embodiment.

Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the detailed description in conjunction with the accompanying drawings. However, it should be understood that the present invention is not limited to the embodiments presented below, but may be implemented in a variety of different forms, and includes all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. . The embodiments presented below are provided to complete the disclosure of the present invention, and to completely inform those of ordinary skill in the art to the scope of the present invention. In describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented in various numbers of hardware and/or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or by circuit configurations for a given function. Also, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as an algorithm running on one or more processors. Further, the present disclosure may employ prior art techniques for electronic configuration, signal processing, and/or data processing, etc. Terms such as "mechanism", "element", "means" and "configuration" will be used broadly. and are not limited to mechanical and physical configurations.

In addition, the connecting lines or connecting members between the components shown in the drawings only exemplify functional connections and/or physical or circuit connections. In an actual device, a connection between components may be represented by various functional connections, physical connections, or circuit connections that are replaceable or added.

Hereinafter, a 'vehicle' may mean any type of transportation means used to move people or things with an engine, such as a car, a bus, a motorcycle, a kickboard, or a truck.

Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

1 to 3 are diagrams for explaining an autonomous driving method according to an exemplary embodiment.

Referring to FIG. 1 , an autonomous driving device according to an embodiment of the present invention may be mounted on a vehicle to implement an autonomous driving vehicle 10 . The autonomous driving device mounted on the autonomous driving vehicle 10 may include various sensors for collecting surrounding context information. For example, the autonomous driving device may detect the movement of the preceding vehicle 20 driving in front through an image sensor and/or an event sensor mounted on the front of the autonomous driving vehicle 10 . The autonomous driving device may further include sensors for detecting the front of the autonomous driving vehicle 10 as well as other driving vehicles 30 running in the adjacent lane, and pedestrians in the vicinity of the autonomous driving vehicle 10 .

At least one of the sensors for collecting situation information around the autonomous vehicle may have a predetermined angle of view (FoV) as shown in FIG. 1 . For example, when the sensor mounted on the front of the autonomous vehicle 10 has an angle of view (FoV) as shown in FIG. 1 , information detected from the center of the sensor may have a relatively high importance. This may be because information corresponding to the movement of the preceding vehicle 20 is mostly included in the information detected from the center of the sensor.

The autonomous driving device controls the movement of the autonomous driving vehicle 10 by processing information collected by the sensors of the autonomous driving vehicle 10 in real time, while at least some of the information collected by the sensors may be stored in a memory device. .

Referring to FIG. 2 , the autonomous driving device 40 may include a sensor unit 41 , a processor 46 , a memory system 47 , and a vehicle body control module 48 . The sensor unit 41 includes a plurality of sensors 42-45, and the plurality of sensors 42-45 may include an image sensor, an event sensor, an illuminance sensor, a GPS device, an acceleration sensor, and the like.

Data collected by the sensors 42 - 45 may be transmitted to the processor 46 . The processor 46 stores the data collected by the sensors 42-45 in the memory system 47, and controls the vehicle body control module 48 based on the data collected by the sensors 42-45 to control the vehicle. can determine the movement of The memory system 47 may include two or more memory devices and a system controller for controlling the memory devices. Each of the memory devices may be provided as one semiconductor chip.

In addition to the system controller of the memory system 47 , each of the memory devices included in the memory system 47 may include a memory controller, and the memory controller may include an artificial intelligence (AI) operation circuit such as a neural network. The memory controller may generate arithmetic data by giving a predetermined weight to data received from the sensors 42 - 45 or the processor 46 , and store the arithmetic data in the memory chip.

3 is a diagram illustrating an example of image data acquired by a sensor of an autonomous vehicle equipped with an autonomous driving device. Referring to FIG. 3 , image data 50 may be data acquired by a sensor mounted on the front of the autonomous vehicle. Therefore, the image data 50 may include the front part 51 of the autonomous driving vehicle, the preceding vehicle 52 in the same lane as the autonomous driving vehicle, and the driving vehicle 53 and the background 54 around the autonomous driving vehicle. .

In the image data 50 according to the embodiment shown in FIG. 3 , data of an area in which the front part 51 and the background 54 of the autonomous vehicle appear is data with little possibility of affecting the operation of the autonomous vehicle. can be In other words, the front part 51 and the background 54 of the autonomous vehicle may be regarded as data having relatively low importance.

On the other hand, the distance from the preceding vehicle 52 and the lane change movement of the driving vehicle 53 may be very important factors in the safe operation of the autonomous vehicle. Accordingly, data of a region including the preceding vehicle 52 and the driving vehicle 53 in the image data 50 may have relatively high importance in driving the autonomous vehicle.

The memory device of the autonomous driving device may store the image data 50 received from the sensor by assigning different weights to each region. For example, a high weight is given to data in an area including the preceding vehicle 52 and the driving vehicle 53 , and a low weight is given to data in an area in which the front part 51 and the background 54 of the autonomous vehicle appear. can be given

4 is a diagram for explaining an example of a plurality of objects included in image data according to an exemplary embodiment.

A plurality of objects may be included in image data acquired from a camera mounted on a vehicle at a specific point in time.

The information about the object includes object type information and object attribute information. Here, the object type information is index information indicating the type of object, and is composed of a group that is a large range and a class that is a detailed range. And, the object attribute information indicates attribute information on the current state of the object, and includes motion information, rotation information, traffic information, color information, and visibility information.

In an embodiment, the groups and classes included in the object type information may be as shown in Table 1 below, but is not limited thereto.

In addition, the information included in the object property information may include Action, Rotate, Traffic info, color, and Visibility information.

Action information expresses motion information of an object and may be defined as stopping, parking, moving, and the like. In the case of a vehicle, stop, parking, and movement may be determined as object property information, and in the case of an immovable object such as a traffic light, object property information may be determined as a default value of stop.

Rotate information represents rotation information of an object and may be defined as front, rear, horizontal, vertical, and side. In the case of a vehicle, object property information may be determined in the front, rear, and side directions, and for a traffic light in a horizontal or vertical direction, object property information may be determined horizontally or vertically, respectively.

Traffic info means traffic information of an object, and can be defined as an instruction, caution, regulation, auxiliary sign, etc. of a traffic sign. Color refers to the color information of an object, and it can express the color of an object, the color of a traffic light, and a traffic sign.

For example, object attribute information of a specific vehicle may be determined as stop, front, red, visibility level 3 .

Referring to FIG. 4 , the objects included in the image 400 obtained from the camera may be, but are not limited to, traffic lights, signs, ego-lanes, intersections, road markings, bumps, and the like.

5A to 5B are diagrams for explaining feature information included in image data according to an exemplary embodiment.

As the vehicle travels on the road, a plurality of image data may be acquired from a camera mounted on the vehicle.

An object 511 may be included in the image data 510 obtained from the camera mounted on the vehicle. Referring to FIG. 5A , the object 511 may be a traffic light.

The image data may have a predetermined size M x N (M and N are natural numbers). The plurality of image data may include the same object 511, but as the vehicle travels along the road, the relative positions of the vehicle and the object 511 continue to change. position will change.

When the entire image data is used to determine which object is the same in each image data, the amount of data transfer and calculation becomes considerably large. Accordingly, it is difficult to process through edge computing in a device mounted on a vehicle, and real-time analysis is also difficult.

Referring to FIG. 5B , a feature 521 included in image data 520 is shown. The feature 521 is metadata about the object 511, and the feature information includes object type information (group, class, etc.), location information of the object 511 on the image data 520, size information of the object 511, etc. this may be included. Referring to FIG. 5B , the feature information includes information that the corresponding object 511 corresponds to a traffic light class, information that has a rectangular shape, and that the lower left vertex of the object 511 is located at (m, n) on the image data. Information and information that the size of the object 511 is a x b may be included.

In the present disclosure, instead of using the entire image data, it is possible to determine which object is the same in a plurality of image data by using feature information corresponding to metadata about the object.

6A to 6B are exemplary diagrams for describing feature merging in image data according to an embodiment.

Referring to FIG. 6A , image data acquired from a camera mounted on a vehicle at a specific time is shown. The first image data 610 is image data obtained at a first time point, and the second image data 620 is image data obtained at a second time point after a predetermined time has elapsed from the first time point.

Meanwhile, the first image 601 and the second image 602 are provided to help explain, and show an actual road photographed by a camera.

The same objects are included in the first image data 610 and the second image data 620 .

An apparatus for determining a location of an object on a map by road (hereinafter, an 'object location determining apparatus') may determine a plurality of features representing the same object from a plurality of image data.

Referring to FIG. 6A , in the first image data 610 , a first feature group 611 and a second feature group 612 may be positioned adjacent to each other. That is, when the distance between the objects corresponding to the first feature group 611 and the second feature group 612 and the vehicle is far apart, corresponding to the first feature group 611 and the second feature group 612 . The actual distance difference between the objects may not be reflected, and the objects may be located adjacent to each other on the first image data 610 .

On the other hand, in the second image data 620 , the first feature group 621 and the second feature group 622 may be located at a certain distance from each other. That is, when the distance between the objects corresponding to the first feature group 611 and the second feature group 612 and the vehicle becomes closer compared to the location of the vehicle when the first image data 610 is captured, the first The difference in actual distance between the objects corresponding to the feature group 611 and the second feature group 612 is further reflected, so that the objects may be located at some distance from each other on the second image data 620 .

Meanwhile, in the first image data 610 and the second image data 620 , the first feature groups 611 and 621 represent the same objects, and the second feature groups 612 and 622 represent the same objects, but The size may also vary depending on the data.

As in the examples of the first image data 610 and the second image data 620 , a plurality of features representing the same object may be included in image data captured while the vehicle is driving, but in the process of driving the vehicle, As the position between the vehicle and the object changes every moment, the position, size, positional relationship, and arrangement relationship of a plurality of features are different from each other on each image data.

The object positioning apparatus may determine a plurality of features corresponding to the same object in the plurality of image data by aggregating the features in the plurality of image data.

The object positioning apparatus may determine a plurality of features corresponding to the same object in a plurality of image data based on a degree of correlation between a plurality of features included in the image data. The degree of correlation may be determined based on an arrangement relationship and a size relationship between the plurality of features, but the factor determining the degree of correlation is not limited thereto.

In an embodiment, the object positioning apparatus may determine a plurality of features representing the same object from a plurality of image data using a graph-matching method.

The object positioning apparatus may register a graph composed of nodes and links by designating a plurality of features including a key feature as nodes in the image data and setting a connection relationship between the plurality of features as links. Also, the object positioning apparatus may determine whether key features in each image data represent the same object through comparison between graphs registered from different image data.

Specifically, the object positioning apparatus selects the first feature 611a of the first image data and the second feature 621a of the second image data 620, and the first feature 611a and the second feature 621a ) to explore other nearby features. The object positioning apparatus determines that when a second neighboring feature 621b having properties similar to a first neighbor feature 611b around the first feature 611a exists around the second feature 621a, the first A graph using the first feature 611a and the second feature 621a as key features may be registered for each of the neighboring feature 611b and the second neighboring feature 621b. Here, the attribute may include at least one of a class, an aspect ratio of a feature, a distance, and an angle.

The object positioning apparatus may repeat the steps of searching and registering all the features around the first feature 611a as a target. After attempting to construct a graph for all the features around the first feature 611a, the object positioning device determines the number of features and the similarity of the registered key features (eg, size, size relationship with neighboring features, etc.) Based on the similarity between the first feature 611a and the second feature 621a may be calculated. The object positioning apparatus performs the steps of searching, registering as a graph, repeating, and calculating the similarity with respect to the remaining features except for the second feature 621a of the second image data 620. It may be determined that the first feature 611a of the first image data 610 and the second feature 621a of the second image data 620 having the highest similarity represent the same object.

In the present disclosure, it is possible to utilize feature information corresponding to metadata about an object without using the entire image data. Accordingly, in the present disclosure, as the position between the vehicle and the object changes every moment in the process of driving the vehicle, even if the position, size, positional relationship, and arrangement relationship of a plurality of features are different on each image data, within the plurality of image data can be determined which is the same object in .

Meanwhile, the object positioning apparatus may determine that features included in the graph have the same depth. Having the same depth may mean that the position in the lateral direction of the road (the driving direction of the vehicle) is the same or within a predetermined range (eg, 1 m).

When the object positioning apparatus performs post-processing on a predetermined feature, the same post-processing may be performed on the remaining features included in the graph having the same depth.

Referring to FIG. 6A , in the object positioning apparatus, the plurality of features included in the first feature groups 611 and 621 have the same depth, and the plurality of features included in the second feature groups 612 and 622 are mutually exclusive. It can be determined to have the same depth. In this case, when the object positioning apparatus corrects the position of a predetermined feature included in the first feature groups 611 and 621 , the same position is also applied to the remaining features included in the first feature groups 611 and 621 . can be corrected.

Referring to FIG. 6B , image data acquired from a camera mounted on a vehicle at a specific point in time is illustrated. The vehicle's longitudinal position on the road may change (eg change lanes) while the vehicle is running. The first image data 630 is image data photographed by a vehicle traveling on the first lane of the road, and the second image data 640 is image data photographed by a vehicle traveling on the fourth lane of the road.

Meanwhile, the first image 603 and the second image 604 are for illustrative purposes, and show the appearance of an actual road photographed by a camera.

The same objects are included in the first image data 630 and the second image data 640 .

The object positioning apparatus may determine a plurality of features representing the same object from a plurality of image data.

Referring to FIG. 6B , in the first image data 630 , a first feature group 631 and a second feature group 632 may be positioned adjacent to each other. Also, the first feature group 631 may be positioned on the right side of the second feature group 632 . That is, the arrangement between the first feature group 631 and the second feature group 632 according to the relative positional relationship between the vehicle and the plurality of objects on the first image data 630 captured while the vehicle is driving in the first lane. The relationship may be determined as described above.

On the other hand, in the second image data 640 , the first feature group 641 and the second feature group 642 may be located at a certain distance from each other. Also, the first feature group 641 may be positioned on the left side of the second feature group 642 . That is, the arrangement between the first feature group 641 and the second feature group 642 according to the relative positional relationship between the vehicle and the plurality of objects on the second image data 640 taken while the vehicle is driving in the four lanes. The relationship may be determined as described above.

Meanwhile, in the first image data 630 and the second image data 640 , the first feature groups 631 and 641 represent the same objects, and the second feature groups 632 and 642 represent the same objects, but images The size may also vary depending on the data.

As in the examples of the first image data 630 and the second image data 640 , a plurality of features representing the same object may be included in image data captured while the vehicle is driving, but the vehicle is driving in a certain lane. As the position between the vehicle and the object varies depending on the function, the position, size, positional relationship, and arrangement relationship of a plurality of features are different from each other on each image data.

The object positioning apparatus may determine a plurality of features corresponding to the same object in the plurality of image data by merging the features in the plurality of image data.

The object positioning apparatus may determine a plurality of features corresponding to the same object in a plurality of image data based on a degree of correlation between a plurality of features included in the image data.

Using the method described above in FIG. 6B , the object positioning apparatus may determine a plurality of features corresponding to the same object from a plurality of image data.

In the present disclosure, instead of using the entire image data, feature information corresponding to metadata about an object may be used. Accordingly, in the present disclosure, even if the location, size, positional relationship, and arrangement relationship of a plurality of features are different on each image data as the location between the vehicle and the object changes as the vehicle travels in different driving lanes, the plurality of images You can determine which objects are identical within your data.

7 is an exemplary diagram for describing a method of determining a location of an object on a road map according to an exemplary embodiment.

The object positioning apparatus may acquire location information of the vehicle using a GPS mounted on the vehicle. The location information of the vehicle may be location information on a road map.

Also, the object positioning apparatus may determine the location of the object on the road map by using location information in the image data of a plurality of features determined to correspond to the same object in the plurality of image data.

Referring to FIG. 7 , as described with reference to FIGS. 6A to 6B , in the first image data 630 and the second image data 640 , the first feature groups 631 and 641 represent the same objects, and the second feature group (632, 642) may represent the same objects.

In an embodiment, the object positioning device uses the location information on the road map obtained through the location checking device connected to the vehicle, and the first image data 630 and the second image data 640 of the object on the road map. location can be determined. Meanwhile, the object positioning apparatus may use one or three or more image data to determine the location of the object on the road map.

Referring to FIG. 7 , the apparatus for determining an object position includes a position of a first object group 710 corresponding to the first feature group 631 and 641 and a second object group corresponding to the second feature group 632 and 642 ( The location of 720 ) may be determined and displayed on the road map 700 .

8 is an exemplary view for explaining a method of determining a position of an object based on position information of a vehicle and a position of a feature on an image according to an embodiment.

Referring to FIG. 8 , the vehicle 800 is moving from a first position to a second position along a driving path.

The object positioning device may acquire location information of the vehicle 800 using a GPS mounted on the vehicle 800 or a location checking device connected to the vehicle. The object positioning apparatus may obtain location information on a lane map of the vehicle 800 by using location information of the vehicle 800 .

The object positioning apparatus may obtain peripheral (front, side, rear, etc.) image data 810a and 810b of the vehicle 800 from the camera 830 mounted on the vehicle 800 .

The first image data 810a is image data captured by the camera 830 when the vehicle 800 is located at the first position, and the second image data 810b is the image data when the vehicle 800 is located at the second position. This is image data captured by the camera 830 when the

The first image data 810a may include a first feature 811 , and the second image data 810b may include a second feature 812 . The first feature 811 and the second feature 812 represent the same object, the traffic light 820 .

Although the first feature 811 and the second feature 812 represent the same object, when the vehicle 800 moves from the first position to the second position, the relative distance between the vehicle 800 and the traffic light 820 is The position and size of the features 811 and 812 indicating the traffic light 820 in each image data 810a and 810b captured by the camera 830 of the vehicle 800 are changed accordingly.

The object positioning apparatus may determine the location of the object 820 on the road map based on location information of the vehicle 800 on the road map and location information of the features 811 and 812 on the image data 810a and 810b.

In an embodiment, the object positioning apparatus may determine the location of the object 820 on the road map by using a distance measurement method, but the method of determining the location of the object 820 on the road map is not limited thereto.

In an embodiment, the object positioning apparatus may acquire the posture value of the camera 830 . The posture value of the camera 830 may include factors related to an installation position, an installation direction, and an installation angle of the camera 830 .

Referring to FIG. 8 , the object positioning apparatus considers the attitude value of the camera 830 , and virtual rays passing through the features 811 and 812 on the image data 810a and 810b from the camera 830 . You can determine the location of the point where (rays) meet.

Specifically, the vehicle 800 may be traveling from the first position to the second position. A feature 811 representing the object 820 is displayed in the first image data 810a captured by the camera 830 of the vehicle 800 located at the first position, and the camera of the vehicle 800 located at the second position is displayed. A feature 812 representing the object 820 may be displayed in the second image data 810b captured in operation 830 . The object positioning device includes an imaginary first ray 841 that starts from the camera 830 of the vehicle 800 at a first position and passes through a feature 811 on the first image data 810a, and A location 850 of a point where a virtual second light ray 842 starting from the camera 830 of the vehicle 800 and passing through the feature 812 on the second image data 810b meets each other may be determined.

The object positioning apparatus may determine the location of the object 820 on the road map based on the moving distance of the vehicle 800 and the location of the point where the virtual rays meet.

Specifically, the object positioning apparatus uses a position 850 of a point where the first ray 841 and the second ray 842 meet, and the distance the vehicle 800 moves from the first position to the second position. , the location of the object 820 on the lane map may be determined.

In an embodiment, the object positioning apparatus may further acquire an attitude value of the vehicle 800 . The attitude value of the vehicle 800 may include a factor related to a slope of a road on which the vehicle 800 is traveling, a difference in air pressure between tires of the vehicle 800 , and a steering angle of the vehicle 800 .

The object positioning apparatus considers the attitude value of the camera 830 and the attitude value of the vehicle 800, and starts from the camera 830 and passes through the features 811 and 812 on the image data 810a, 810b. It is possible to determine the location of the point where the rays meet. 9 is an exemplary diagram for describing a method of determining a location of an object on a road map using a plurality of driving trajectories, according to an exemplary embodiment.

Referring to FIG. 9 , a first trajectory 910 and a second trajectory 920 of a vehicle traveling on a predetermined road are shown. In FIG. 9 , the position of the 1-1 object 911 and the position of the 1-2 th object 921 are determined based on a plurality of image data captured while the vehicle travels along the first trajectory 910 . a determined position. In addition, the position of the 2-1 object 912 and the position of the 2-2 object 922 are positions determined based on a plurality of image data captured while the vehicle travels along the second trajectory 920 . to be.

The 1-1 object 911 and the 1-2 object 921 are referred to as first group objects, and the 2-1 object 912 and the 2-2 object 922 are second group objects. to be referred to as

The object positioning device configures a first graph using the first group objects, configures a second graph using the second group objects, and compares the similarity between the first graph and the second graph to the first group object It is possible to determine the same object in the group objects and the second group objects.

Specifically, in order to calculate the degree of correlation between the objects, the object positioning apparatus targets a predetermined object generated for each vehicle or each time, and is located in the vicinity of the predetermined object (eg, a similar distance and direction from the road). When an object of a class exists, it can be registered in the graph, and the objects having the highest similarity can be determined to be the same object by calculating the similarity based on the location.

When a 1-21 object 921 of the same class exists around the 1-1 object 911, the object positioning device divides the object into the 1-1 object 911 and the 1-2 object 921. One graph may be configured, and a second graph may be configured with a 2-1 object 912 and a 2-2 object 922 . The object positioning apparatus may determine which object is the same in each graph by comparing the similarity by using each object as a key object in the graph.

In FIG. 9 , in the object positioning apparatus, a 1-1 object 911 and a 2-1 object 912 represent the same object, and a 1-2 th object 921 and a 2-2 object 922 . can be determined to represent the same object.

The object positioning apparatus may merge positions of objects determined to be the same object and display final positions for each object on the road map 900 .

In FIG. 9 , the apparatus for determining an object position may determine the position of the 1-3 th object 931 as the final position by merging the positions of the 1-1 object 911 and the 1-2 th object 921 . Also, the apparatus for determining an object position may determine the position of the 2-3 th object 932 as the final position by merging the positions of the 2-1 th object 912 and the 2-2 nd object 922 .

10 is an exemplary diagram for describing a vehicle fitting method and an object fitting method according to an exemplary embodiment.

The object positioning device may acquire the driving trajectory of the vehicle using a positioning device connected to the vehicle. The positioning device may be a GPS. Hereinafter, it is assumed that the GPS is used as the positioning device. Also, the object positioning apparatus may map the obtained driving trajectory to the road map 1000 and display the driving trajectory on the road map 1000 .

Referring to FIG. 10 , the object positioning apparatus may obtain a first driving trajectory 1011 using a GPS mounted on a vehicle and display it on the road map 1000 .

Meanwhile, due to various causes, a GPS signal received from a GPS mounted on a vehicle may have an error. For example, GPS signal reception may be temporarily unstable when a vehicle passes through an alley with dense buildings. Alternatively, due to radio waves reflected by the glass surface of the building, the vehicle may receive an erroneous GPS signal.

The object positioning apparatus may determine a current driving lane of the vehicle. The information on the current driving lane may be information included in the feature information.

The object positioning apparatus may determine that the current driving lane of the vehicle is a predetermined lane 1030 . In this case, the object positioning apparatus may determine that the first driving trajectory 1011 obtained from the GPS signal does not match the predetermined lane 1030 . Specifically, the object positioning device calculates the separation distance between the first driving trajectory 1011 obtained from the GPS signal and the predetermined lane 1030, and when the separation distance exceeds a threshold value, the first driving trajectory 1011 It may be determined that it does not match with the predetermined lane 1030 . This may be a case where the first driving trajectory 1011 and the predetermined lane 1030 do not match due to an error in the GPS signal even though the vehicle is currently driving in the predetermined lane 1030 .

The object positioning apparatus may perform lane fitting so that the driving trajectory of the vehicle is fitted to the lane on the lane map based on the current driving lane included in the feature information.

In detail, the object positioning apparatus may perform lane fitting in consideration of a distance and curvature between a driving trajectory of a vehicle and a lane marking line on a lane map. The object positioning device may perform lane fitting so that at least a portion of the driving trajectory of the vehicle is linearly moved and at least a portion of the curvature is adjusted so that the driving trajectory of the vehicle matches the lane marking line on the lane map corresponding to the current driving lane. have.

Referring to FIG. 10 , the object positioning device linearly moves at least a part of the first driving trajectory 1011 of the vehicle, adjusts at least a portion of the curvature to correct it as the second driving trajectory 1012 , thereby providing a predetermined value on the road map. The road fitting may be performed to fit the road 1030 .

In an embodiment, the object 1021 may be displayed on the road map 1000 . The object positioning apparatus may perform object fitting so that the location of the object on the road map 1000 is fitted in association with the road fitting result.

Referring to FIG. 10 , the object positioning device sets the position of the first object 1021 to the second object 1022 in the same way that the first driving trajectory 1011 of the vehicle is corrected to the second driving trajectory 1012 . Object fitting can be performed by calibrating to the position of .

Meanwhile, the lane fitting may be performed before and/or after the image merging described above with reference to FIGS. 6 to 7 is completed.

11 is a flowchart of a method of determining a location of an object on a road map according to an exemplary embodiment.

The method for determining the current lane of the vehicle shown in FIG. 11 is related to the embodiments described in the drawings described above, and therefore, even if omitted below, the contents described in the drawings are those of FIG. 11 . method can also be applied.

Referring to FIG. 11 , in operation 1110 , the processor may acquire image data captured by a camera mounted on a vehicle.

In operation 1120, the processor may generate feature information for a plurality of features included in the image data by performing a predetermined process on the image data.

The predetermined processing may be applied without limitation as long as it is a processing method capable of generating feature information from image data. Specifically, the predetermined processing may include a processing method capable of generating object type information (group, class, etc.), location information of an object on image data, information on size of an object, and the like from image data. For example, the predetermined processing is, Traffic lights detection (DLD), Traffic signs detection (TSD), Ego-lane recognition (ELR), Intersection recognition (ELR), Road markings detection (RMD), Bump recognition (BR), etc. It can be implemented by a module for

In operation 1130, the processor may determine a plurality of features corresponding to the same object in the plurality of image data based on the degree of correlation between the plurality of features included in the image data.

The processor may determine a plurality of features corresponding to the same object in the plurality of image data by aggregating the plurality of image data.

The processor may determine a plurality of features representing the same object in the plurality of image data by using a graph-matching method.

The processor may select the first feature of the first image data and the second feature of the second image data, and search for other features around the first feature and the second feature. When a second neighboring feature having similar properties to a first neighboring feature around the first feature exists around the second feature, the processor determines whether the first feature and the second neighboring feature are respectively 2 Features can be registered as a graph with key features. Here, the attribute may include at least one of a class, an aspect ratio of a feature, a distance, and an angle.

The processor may repeat the steps of searching and registering all the features around the first feature. After the processor attempts to construct a graph for all features around the first feature, based on the number of features and the similarity of registered key features (eg, size, size relationship with neighboring features, etc.), the first feature and a similarity between the second feature and the second feature may be calculated. The processor performs the steps of searching, registering as a graph, repeating, and calculating a degree of similarity with respect to the remaining features except for the second feature of the second image data, so that the degree of similarity to the first feature of the first image data is performed. It may be determined that the second feature of the second image data with the highest α represents the same object.

The processor may determine that features included in the graph have the same depth.

When the processor performs post-processing on a predetermined feature, the processor may perform the same post-processing on the remaining features included in the graph having the same depth.

In operation 1140, the processor may determine the location of the object on the road map based on the location information on the vehicle lane map and the feature location information on the plurality of image data.

The processor may acquire location information on a road map of the vehicle by using a location checking device connected to the vehicle.

The processor may determine the location of the object based on the location information on the lane map of the vehicle and the feature location information on the image data.

The processor may determine the location of the object by using location information of a plurality of features determined to correspond to the same object in the plurality of image data.

In an embodiment, the object may include at least one of a traffic light, a road sign, a road mark, a current driving lane, an intersection, and a bump.

In an embodiment, the processor may acquire the driving trajectory of the vehicle by using a positioning device connected to the vehicle. Also, the processor may perform lane fitting so that the driving trajectory of the vehicle is fitted to the lane on the lane map based on the current driving lane included in the feature information.

In an embodiment, the processor may perform object fitting so that the location of the object on the road map is fit in association with the road fitting result.

In an embodiment, the processor may display the first positions of the first group objects determined from the first trajectory of the vehicle traveling on the predetermined road on the road map. Also, the processor may display the second positions of the second group objects determined from the second trajectory of the vehicle traveling on the predetermined road on the road map. In addition, the processor may determine the same object from the first group objects and the second group objects, merge the positions of the objects determined as the same object, and display the final positions of the respective objects on the map by car.

In an embodiment, the processor configures a first graph using the first group objects, configures a second graph using the second group objects, and compares the similarity between the first graph and the second graph to obtain a first graph. The same object may be determined from the group objects and the second group objects.

12 is a block diagram of an object positioning apparatus according to an embodiment.

Referring to FIG. 12 , the object positioning apparatus 1200 may include a communication unit 1210 , a processor 1220 , and a DB 1230 . Only the components related to the embodiment are shown in the object positioning apparatus 1200 of FIG. 12 . Accordingly, it can be understood by those skilled in the art that other general-purpose components may be further included in addition to the components shown in FIG. 12 . Also, according to an embodiment, the apparatus 1200 for determining an object location may not include the communication unit 1210 .

The communication unit 1210 may include one or more components for performing wired/wireless communication with an external server or an external device. For example, the communication unit 1210 may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiving unit (not shown).

The DB 1230 is hardware for storing various types of data processed in the object positioning apparatus 1200 , and may store a program for processing and controlling the processor 1220 . The DB 1230 may store payment information, user information, and the like.

DB 1230 is a random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD- It may include ROM, Blu-ray or other optical disk storage, a hard disk drive (HDD), a solid state drive (SSD), or flash memory.

The processor 1220 controls the overall operation of the object positioning apparatus 1200 . For example, the processor 1220 may generally control the input unit (not shown), the display (not shown), the communication unit 1210 , the DB 1230 , and the like by executing programs stored in the DB 1230 . The processor 1220 may control the operation of the object positioning apparatus 1200 by executing programs stored in the DB 1230 .

The processor 1220 may control at least some of the operations of the object positioning apparatus described above with reference to FIGS. 1 to 11 .

The processor 1220 includes application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, and microcontrollers. It may be implemented using at least one of (micro-controllers), microprocessors, and other electrical units for performing functions.

As an embodiment, the object positioning apparatus 1200 may be an electronic device having mobility. For example, the traffic information providing apparatus 800 may be implemented as a smartphone, a tablet PC, a PC, a smart TV, a personal digital assistant (PDA), a laptop, a media player, a navigation device, a device equipped with a camera, and other mobile electronic devices. can In addition, the object positioning apparatus 1200 may be implemented as a wearable device such as a watch, glasses, a hair band, and a ring having a communication function and a data processing function.

In another embodiment, the object positioning device 1200 may be an electronic device embedded in a vehicle. For example, the object positioning device 1200 may be an electronic device inserted into a vehicle through tuning after a production process.

As another embodiment, the object positioning apparatus 1200 may be a server located outside the vehicle. The server may be implemented as a computer device or a plurality of computer devices that communicate through a network to provide commands, codes, files, contents, services, and the like. The server may receive data necessary to determine the moving path of the vehicle from devices mounted on the vehicle, and determine the moving path of the vehicle based on the received data.

As another embodiment, the process performed by the object positioning apparatus 1200 may be performed by at least a portion of an electronic device having mobility, an electronic device embedded in a vehicle, and a server located outside the vehicle.

Embodiments according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium includes a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and DVD, a magneto-optical medium such as a floppy disk, and a ROM. , RAM, flash memory, and the like, hardware devices specially configured to store and execute program instructions.

Meanwhile, the computer program may be specially designed and configured for the present invention, or may be known and used by those skilled in the computer software field. Examples of the computer program may include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store™) or between two user devices. It can be distributed directly or online (eg, downloaded or uploaded). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

The steps constituting the method according to the present invention may be performed in an appropriate order, unless the order is explicitly stated or there is no description to the contrary. The present invention is not necessarily limited to the order in which the steps are described. The use of all examples or exemplary terms (eg, etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is limited by the examples or exemplary terms unless defined by the claims. it's not going to be In addition, those skilled in the art will appreciate that various modifications, combinations, and changes may be made in accordance with design conditions and factors within the scope of the appended claims or their equivalents.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the spirit of the present invention is not limited to the scope of the scope of the present invention. will be said to belong to

Claims (13)

A method for determining the location of an object on a map by road, the method comprising:
acquiring image data captured by a camera mounted on a vehicle;
generating feature information for a plurality of features included in the image data by performing predetermined processing on the image data;
determining a plurality of features corresponding to the same object in a plurality of image data based on a degree of correlation between the plurality of features included in the image data; and
determining a position of an object on the lane map based on position information on the lane map of the vehicle and feature position information on the plurality of image data;
including,
The method is
displaying first positions of first group objects determined from a first trajectory of a vehicle traveling on a predetermined road on the road map;
displaying second positions of second group objects determined from a second trajectory of the vehicle traveling on the predetermined road on the road map; and
determining the same object from the first group objects and the second group objects, merging the locations of the objects determined as the same object, and displaying the final location of each object on the road map;
A method further comprising: The method of claim 1,
wherein the object comprises at least one of a traffic light, a road sign, a road mark, and a bump. The method of claim 1,
The step of determining a plurality of features corresponding to the same object,
determining a plurality of features corresponding to the same object in the plurality of image data by aggregating the features in the plurality of image data;
A method comprising The method of claim 1,
The step of determining a plurality of features corresponding to the same object,
determining a plurality of features representing the same object in a plurality of image data using a graph-matching method;
A method comprising 5. The method of claim 4,
selecting a first feature of the first image data and a second feature of the second image data;
searching for other features around the first feature and the second feature;
When a second neighbor feature having similar properties to a first neighbor feature around the first feature exists around the second feature, the first neighbor feature and the first neighbor feature registering a feature and the second feature as a key feature in a graph;
repeating the searching and registering for all features around the first feature;
after attempting to construct a graph for all features around the first feature, calculating a similarity between the first feature and the second feature based on the number of features and the similarity of registered key features; and
For the remaining features except for the second feature of the second image data, the searching step, the step of registering as the graph, the repeating step, and the steps of calculating the similarity of the first image data are performed. determining that the second feature of the second image data having the highest similarity to the first feature represents the same object;
A method comprising 6. The method of claim 5,
The method is
determining that the features included in the graph have the same depth;
How to include more. 7. The method of claim 6,
The method is
performing the same post-processing on the remaining features included in the graph having the same depth when the post-processing is performed on a predetermined feature;
A method comprising 6. The method of claim 5,
wherein the attribute comprises at least one of a class, an aspect ratio of a feature, a distance, and an angle. The method of claim 1,
The method is
acquiring location information on the lane map of the vehicle using a location checking device connected to the vehicle;
A method further comprising: delete The method of claim 1,
The step of displaying the final positions of the respective objects on the road map comprises:
A first graph is constructed using the first group objects, a second graph is constructed using the second group objects, and the first group objects are obtained by comparing the similarity between the first graph and the second graph. and determining the same object in the second group of objects;
A method comprising An apparatus for determining a location of an object on a road map, comprising:
a memory in which at least one program is stored; and
A processor for performing an operation by executing the at least one program,
The processor is
Acquire image data taken from a camera mounted on the vehicle,
generating feature information for a plurality of features included in the image data by performing a predetermined processing on the image data;
Determining a plurality of features corresponding to the same object in a plurality of image data based on a degree of correlation between the plurality of features included in the image data,
determining a location of an object on the road map based on location information on the vehicle lane map and feature location information on the plurality of image data;
The processor is
Displaying the first positions of the first group objects determined from the first trajectory of the vehicle traveling on the predetermined road on the road map,
displaying the second positions of the second group objects determined from the second trajectory of the vehicle traveling on the predetermined road on the road map,
Determining the same object from the first group objects and the second group objects, and merging the positions of the objects determined as the same object to display the final location for each object on the map by car, . A computer-readable recording medium in which a program for executing the method of claim 1 in a computer is recorded.

Images