KR102464586B1 - Traffic light location storage apparatus and method - Google Patents

Abstract

A traffic light location storage device and method are disclosed. Traffic light location storage device of the present invention is a mobile mapping system for detecting the MMS (Mobile Mapping System) data of the road and the surrounding while driving the actual road; a recognition unit that detects a traffic light using image data among the MMS data detected by the mobile mapping system and detects a traffic light using LAS data according to whether the current location of the vehicle is included in the set location according to the detection result; and a position coordinate processing unit for detecting and storing the position coordinates of the traffic light detected by the recognition unit.

Description

TRAFFIC LIGHT LOCATION STORAGE APPARATUS AND METHOD

The present invention relates to an apparatus and method for constructing traffic light data, and more particularly, to a traffic light location storage apparatus and method for storing location data of a traffic light in map information using MMS (Mobile Mapping System) data and LAS data.

Global positioning system (GPS) is a high-tech navigation system using artificial satellites. It is used together with a geographic information system (GIS) to provide the driver with road conditions, etc. It helps you reach your target point conveniently and quickly.

On the other hand, in complex times like the modern era, as roads are expanded or modified day by day, auxiliary facilities necessary for roads, such as intersections, traffic signal networks, and crosswalks, frequently disappear or are newly built.

However, due to the complicated and time-consuming process of making and calibrating the numerical map mounted on the GPS, it is possible to update only a few times a year.

As a result, the numerical map mounted on the GPS is often operated without being updated from time to time in a state where the auxiliary facilities that change due to changes or modifications in road conditions are changed, and this is the result of drivers who trust the GPS and drive. It is an obstacle to safe driving.

Background art of the present invention is disclosed in 'Traffic signal detection apparatus and method using photographed image' of Korean Patent Registration No. 10-1745557 (June 2, 2017).

Conventionally, in order to update the traffic lights of the road on the map, the image captured by the vehicle is analyzed to recognize the traffic lights of the road and update the map. However, since this method simply uses only the photographed image, there is a problem in that the recognition rate for the traffic light is lowered, and as a result, the location information of the traffic light is inaccurate.

The present invention has been devised to improve the above problems, and an object according to an aspect of the present invention is a traffic light location storage device for storing location data of traffic lights in map information using MMS (Mobile Mapping System) data and LAS data and to provide a method.

A traffic light location storage device according to an aspect of the present invention includes: a mobile mapping system for detecting MMS (Mobile Mapping System) data measured on a road and surroundings while driving on an actual road; a recognition unit that detects a traffic light using image data among the MMS data detected by the mobile mapping system and detects a traffic light using LAS data according to whether the current location of the vehicle is included in the set location according to the detection result; and a position coordinate processing unit for detecting and storing the position coordinates of the traffic light detected by the recognition unit.

The recognition unit of the present invention may include: a first recognition unit for detecting a traffic light by machine learning from image data among MMS data detected by the mobile mapping system; a zone determination unit for determining whether the current location is within a preset setting area according to the detection result of the first recognition unit; and a second recognition unit for recognizing a traffic light using the LAS data detected by the mobile mapping system when the current location detected by the area determination unit is within the set area.

The location coordinate processing unit of the present invention is characterized in that when a traffic light is detected by the first recognition unit, the location coordinates of the traffic light are detected using the distance to the traffic light and the current location of the vehicle.

The set zone of the present invention is characterized in that it includes at least one of an intersection zone, a crosswalk zone and a child protection zone.

The zone determination unit of the present invention is characterized in that when it is determined that the traffic light is not detected as a result of the detection of the first recognition unit, the current location is determined whether the current location is within the set area.

The second recognition unit of the present invention comprises: a pixel coordinate detection unit for signal processing the image data of the mobile mapping system as a color feature of a traffic light, and detecting pixel coordinates of a region of interest estimated as a traffic light based on the color value of the traffic light; and a LAS traffic light detector configured to detect a traffic light object corresponding to the pixel coordinates detected by the pixel coordinate detector by using the LAS data of the mobile mapping system.

The LAS traffic light detection unit of the present invention is characterized in that the mask processing the portion corresponding to the ground height in the LAS data.

The LAS traffic light detector of the present invention projects the LAS data in an omnidirectional direction to the region of interest detected by the pixel coordinate detector to detect a traffic light object pattern symbol corresponding to a traffic light through symbol pattern matching in the region of interest, characterized in that do.

The location coordinate processing unit of the present invention detects the distance to the traffic light detected by the LAS traffic light detection unit, and detects the location coordinates of the traffic light using the detected distance to the traffic light and the current position of the vehicle.

A traffic light location storage method according to an aspect of the present invention comprises the steps of: detecting, by a mobile mapping system, MMS (Mobile Mapping System) data measured on a road and surroundings while driving on an actual road; detecting, by a recognition unit, a traffic light using image data among MMS data detected by the mobile mapping system, and detecting a traffic light using LAS data according to whether the current location of the vehicle is included in the set location according to the detection result; and the position coordinate processing unit detecting and storing the position coordinates of the traffic light detected by the recognition unit.

The step of detecting the traffic light of the present invention may include: detecting, by a machine learning, a traffic light in image data among MMS data detected by the mobile mapping system by a first recognition unit; determining, by an area determination unit, whether the current location is within a preset area according to the detection result of the first recognition unit; and detecting a traffic light using the LAS data detected by the mobile mapping system when the current location detected by the area determination unit is within the set area by the second recognition unit.

The set zone of the present invention is characterized in that it includes at least one of an intersection zone, a crosswalk zone and a child protection zone.

In the step of detecting a traffic light using the LAS data of the present invention, the recognition unit processes the image data of the mobile mapping system as a traffic light color feature to determine the pixel coordinates of the region of interest estimated as a traffic light based on the traffic light color value. and detecting the traffic light object corresponding to the pixel coordinates detected by the pixel coordinate detection unit using the LAS data of the mobile mapping system.

In the step of detecting a traffic light using the LAS data of the present invention, the recognition unit projects the LAS data in an omnidirectional direction to the region of interest detected by the pixel coordinate detection unit to correspond to a traffic light through symbol pattern matching in the region of interest It is characterized in that it detects a traffic light object pattern symbol.

In the step of detecting a traffic light using the LAS data of the present invention, the recognition unit is characterized in that the masking process for a portion corresponding to the ground height in the LAS data.

A traffic light location storage apparatus and method according to an aspect of the present invention can store location data of a traffic light in map information using MMS data and LAS data, and update the location of a traffic light in real time.

1 is a block diagram of a traffic light location storage device according to an embodiment of the present invention.
2 is an exemplary diagram of a traffic light location storage according to an embodiment of the present invention.
3 is a diagram conceptually illustrating a traffic light location storage process according to an embodiment of the present invention.
4 is a diagram illustrating an example of a region of interest according to an embodiment of the present invention.
5 is a diagram exemplarily illustrating a process of finding a traffic light object through traffic light object pattern symbol matching according to an embodiment of the present invention.
6 is a flowchart of a method for storing a location of a traffic light according to an embodiment of the present invention.

Hereinafter, an apparatus and method for storing a location of a traffic light according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. In this process, the thickness of the lines or the size of the components shown in the drawings may be exaggerated for clarity and convenience of explanation. In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users and operators. Therefore, definitions of these terms should be made based on the content throughout this specification.

1 is a block diagram of an apparatus for storing a location of a traffic light according to an embodiment of the present invention, FIG. 2 is an exemplary diagram for storing a location of a traffic light according to an embodiment of the present invention, and FIG. It is a diagram conceptually showing a traffic light location storage process, FIG. 4 is a diagram showing an example of a region of interest according to an embodiment of the present invention, and FIG. 5 is a traffic light object pattern symbol matching according to an embodiment of the present invention It is a diagram exemplarily showing a process of finding a traffic light object.

Referring to FIG. 1 , a traffic light location storage device according to an embodiment of the present invention includes a mobile mapping system 10 , a recognition unit 20 , and a location coordinate processing unit 30 .

Mobile mapping system (Mobile Mapping System) 10 detects MMS (Mobile Mapping System) data that surveys the road and surroundings while driving the actual road, detects the LAS data using the MMS data, and then the detected image The data and LAS data are transmitted to the recognition unit 20 .

That is, the mobile mapping system 10 is equipped with a high-performance CCD (Charge Coupled Device) camera and positioning equipment (GPS (Global Positioning System)/INS (Inertial Navigation System), Odometer, etc.), and a laser scanner or digital camera, etc. Equipped with an image acquisition device of

The mobile mapping system 10 generates LAS data by using laser data and driving route data acquired through a laser scanner among the MMS data when the MMS data obtained by driving the actual road and measuring the road and surroundings is stored.

The LAS data is laser point data having three-dimensional precise coordinates, and a traffic light may be detected based on the LAS data. This will be described later.

The recognition unit 20 detects a traffic light using image data among the MMS data detected by the mobile mapping system 10, and, as a result of the detection, uses LAS data according to whether the current location of the vehicle is included in a preset location. detect traffic lights

That is, the recognition unit 20 detects a traffic light by analyzing the image data among the MMS data detected by the mobile mapping system 10 through machine learning. When accurately detected, the position coordinate processing unit 30 detects and stores the position coordinates of the corresponding traffic light.

On the other hand, the recognition unit 20 ends the process of detecting and storing the position coordinates of the traffic light when the detection result by machine learning is an incorrect answer, that is, when machine learning for image data has been performed correctly but no traffic light is detected. do.

On the other hand, the recognition unit 20 determines whether the current location of the vehicle is included within a preset setting area when the detection result by machine learning is non-detection, that is, when the machine learning result does not clearly determine the traffic light. After detecting a traffic light using LAS data according to the result, the location coordinates of the corresponding traffic light are detected and stored.

2 and 3 , the recognition unit 20 includes a first recognition unit 21 , a zone determination unit 22 , and a second recognition unit 23 .

The first recognition unit 21 detects a traffic light using machine learning from image data among MMS data detected by the mobile mapping system 10 . In this case, the first recognition unit 21 detects a traffic light using machine learning, for example, deep learning, in the mobile mapping system 10 .

The first recognition unit 21 transmits the detection result to the location coordinate processing unit 30 when the traffic light detection result using deep learning is accurately detected (correct answer). In this case, the location coordinate processing unit 30 detects the location coordinates of the traffic light by using the distance to the traffic light and the current location of the vehicle.

The area determination unit 22 determines whether the current location is within a preset area according to the detection result of the first recognition unit.

First, the zone determination unit 22 terminates the detection and storage of the location of the traffic light because the traffic light does not exist in the case where the machine learning for the image data is accurately performed but the traffic light is not detected (mistake).

As a result of the detection by the machine learning of the first recognition unit 21 , the zone determination unit 22 determines whether the current location is within a preset area when the traffic light is not accurately detected (not detected).

A set area is an area where traffic lights are likely to be located, and the set area may include an intersection area, a crosswalk area, and a child protection area.

That is, the zone determination unit 22 ends the process of detecting and storing the position coordinates of the traffic light when the detection result by the machine learning is an incorrect answer in which the machine learning for the image data has been accurately performed but the traffic light is not detected.

On the other hand, the zone determination unit 22 determines whether the current location of the vehicle is included in the above-described set zone when the detection result by machine learning indicates that the traffic light is not accurately detected.

The second recognition unit 23 recognizes a traffic light by using the LAS data detected by the mobile mapping system 10 when the current location of the vehicle is included within the set area as a result of the determination of the area determination unit 22 .

Referring to FIG. 2 , the second recognition unit 23 includes a pixel coordinate detection unit 231 and an LAS traffic light detection unit 232 .

The pixel coordinate detector 231 processes the image data of the mobile mapping system 10 as a color characteristic of a traffic light, and detects pixel coordinates of a region of interest (ROI) estimated as a traffic light based on the color value of the traffic light.

That is, as shown in FIG. 3 , the pixel coordinate detection unit 231 performs image pre-processing on the image data of the mobile mapping system 10 into traffic light colors, for example, traffic light color characteristics such as red, blue, and yellow.

When the color of the traffic light is detected through the image pre-processing process, the pixel coordinate detector 231 detects a region of interest based on the corresponding color value as shown in FIG. 4 . In this case, the pixel coordinate detection unit 231 sets a length equal to twice (2r) of the diameter as the height of the region of interest based on the diameter (r) of the corresponding color, and sets a length equal to four times (4r) of the diameter Set the width of the region of interest.

The LAS traffic light detection unit 232 detects a traffic light object corresponding to the pixel coordinates detected by the pixel coordinate detection unit 231 using the LAS data of the mobile mapping system 10 .

That is, the LAS traffic light detection unit 232 projects the LAS data in the omnidirectional direction to the region of interest detected by the pixel coordinate detection unit 231 as described above to detect the shape of the traffic light in the region of interest, that is, the traffic light object pattern symbol. do.

The LAS traffic light detection unit 232 detects a shape-based traffic light object pattern symbol as shown in FIG. 5 using the LAS data of the mobile mapping system 10, and matches the symbol pattern corresponding to the traffic light to the traffic light object pattern symbol can be detected.

In this case, the LAS traffic light detection unit 232 minimizes errors in symbol pattern matching by masking the portion corresponding to the ground height in the LAS data.

That is, if the detection result of the traffic light according to the machine learning of the first recognition unit 21 is not detected, there is a possibility that the traffic light is present.

Also, check whether the current location is included in the set area where traffic lights are likely to exist.

As such, if a traffic light is not detected as a result of machine learning and exists within the current set area, there is a possibility that a traffic light exists. By detecting , it is finally determined that the traffic light at the current location is located.

When a traffic light is accurately detected by the first recognizing unit 21 or a traffic light is detected by the second recognizing unit 23, the location coordinate processing unit 30 uses the distance to the corresponding traffic light and the current location of the vehicle. Detects location coordinates of , and stores the detected location coordinates in map information.

Here, as a method for the location coordinate processing unit 30 to detect the location coordinates of a traffic light using the MMS data, various methods may be used in addition to the method using the distance to the traffic light and the current location of the vehicle as described above. .

Hereinafter, a method for storing a location of a traffic light according to an embodiment of the present invention will be described with reference to FIG. 6 .

6 is a flowchart of a method for storing a location of a traffic light according to an embodiment of the present invention.

Referring to FIG. 6 , first, the mobile mapping system 10 detects MMS data obtained by measuring the road and surroundings while driving on the actual road ( S10 ).

In this case, the mobile mapping system 10 stores the MMS data measured as described above, and generates LAS data using the laser data and driving route data acquired through the laser scanner among these MMS data, and the MMS data and LAS data to the recognition unit 20 .

Next, the first recognition unit 21 detects a traffic light from the image data among the MMS data detected by the mobile mapping system 10 by machine learning (S20).

In this case, the first recognition unit 21 detects the traffic light using deep learning, and the detection result is a correct answer that correctly detected the traffic light, an incorrect answer in which detection is performed correctly but the traffic light is not detected, and an incorrect answer in which the detection is performed incorrectly. may appear as detection.

Accordingly, the zone determination unit 22 determines whether the detection result by machine learning of the first recognition unit 21 is the correct answer in which the traffic light is correctly detected (S20), and if the determination result is the correct answer, the location coordinate processing unit 30 detects the distance to the traffic light, and detects the position coordinates of the traffic light using the detected distance to the traffic light and the current position of the vehicle (S70).

On the other hand, as a result of the determination in step S20, if the detection result by machine learning of the first recognition unit 21 is not the correct answer of detecting the traffic light, it is determined whether the detection result is non-detection (S30).

If the detection result is not non-detection as a result of the determination in step S30, it is an incorrect answer that detection has been performed correctly but the traffic light is not detected. Since the wrong answer is a case where the traffic light does not clearly exist as a result of detection by machine learning, the process of detecting and storing the location of the traffic light is terminated.

On the other hand, if the detection result is not detected as a result of the determination in step S30, the area determination unit 22 determines whether the current location is within a preset area, that is, any one of an intersection area, a crosswalk area, and a child protection area. It is determined (S40).

As a result of the determination in step S40, if the current location of the vehicle is included within the set area, the pixel coordinate detection unit 231 signals the image data of the mobile mapping system 10 as a color feature of the traffic light, and signals the traffic light based on the color value of the traffic light The pixel coordinates of the region of interest, which are estimated as , are detected (S50).

Next, the LAS traffic light detection unit 232 detects a traffic light object corresponding to the pixel coordinates detected by the pixel coordinate detection unit 231 using the LAS data of the mobile mapping system 10 ( S60 ).

That is, the LAS traffic light detection unit 232 projects the LAS data in the omnidirectional direction to the region of interest detected by the pixel coordinate detection unit 231 as described above and detects the traffic light object pattern symbol in the region of interest through symbol pattern matching. .

In this case, the LAS traffic light detection unit 232 minimizes errors in symbol pattern matching by masking the portion corresponding to the ground height in the LAS data.

As such, if a traffic light is not detected as a result of machine learning and exists within the current set area, there is a possibility that a traffic light exists. By detecting , it is finally determined that the traffic light at the current location is located.

On the other hand, when a traffic light is detected by the second recognition unit 23 as described above, the location coordinate processing unit 30 detects the location coordinates of the traffic light using the distance to the corresponding traffic light and the current location of the vehicle, and the detected The location coordinates are stored in the map information (S70).

As described above, the apparatus and method for storing the location of a traffic light according to an embodiment of the present invention can store the location data of the traffic light in map information using MMS data and LAS data, and update the location of the traffic light in real time.

Although the present invention has been described with reference to the embodiment shown in the drawings, this is merely exemplary, and it is understood that various modifications and equivalent other embodiments are possible by those of ordinary skill in the art. will understand Accordingly, the true technical protection scope of the present invention should be defined by the following claims.

10: Mobile Mapping System
20: recognition unit
21: first recognition unit
22: zone judgment unit
23: second recognition unit
231: pixel coordinate detection unit
232: LAS traffic light detection unit
30: position coordinate processing unit

Claims (15)

a mobile mapping system that detects MMS (Mobile Mapping System) data obtained by measuring a road and its surroundings while driving on a real road;
a recognition unit that detects a traffic light using image data among the MMS data detected by the mobile mapping system and detects a traffic light using LAS data according to whether the current location of the vehicle is included in the set location according to the detection result; and
and a position coordinate processing unit for detecting and storing the position coordinates of the traffic light detected by the recognition unit,
The recognition unit detects a traffic light from image data among the MMS data detected in the mobile mapping system by machine learning, and if the detection result is correct, transmits the detection result to the location coordinate processing unit, and if the detection result is not detected, according to the current location Traffic light location storage device, characterized in that for detecting the traffic light using the LAS data detected by the mobile mapping system. The method of claim 1, wherein the recognition unit
a first recognition unit that detects a traffic light from image data among the MMS data detected by the mobile mapping system by machine learning and transmits the detection result to the location coordinate processing unit if the detection result is correct;
a zone determination unit terminating the process of detecting the position coordinates of a traffic light if the detection result of the first recognition unit is incorrect, and determining whether the current position is within a preset area if the detection result is not detected; and
and a second recognition unit for recognizing a traffic light using the LAS data detected by the mobile mapping system when the current location is within the set area as a result of the determination of the area determination unit. The apparatus of claim 2, wherein the location coordinate processing unit detects the location coordinates of the traffic light by using the distance to the traffic light and the current location of the vehicle when the traffic light is detected by the first recognition unit.
3. The device of claim 2, wherein the set zone includes at least one of an intersection zone, a crosswalk zone, and a child protection zone.
The method of claim 1, wherein the recognition unit
If the detection result is not detected, it is determined whether the current location is within a preset area, and if the determination result is that the current location is within the set area, the traffic light location, characterized in that the traffic light is recognized using the LAS data detected by the mobile mapping system storage device.
The method of claim 2, wherein the second recognition unit
a pixel coordinate detector for signal processing the image data of the mobile mapping system as a color feature of a traffic light to detect pixel coordinates of a region of interest estimated as a traffic light based on the color value of the traffic light; and
and a LAS traffic light detection unit for detecting a traffic light object corresponding to the pixel coordinates detected by the pixel coordinate detection unit by using the LAS data of the mobile mapping system.
The apparatus of claim 6, wherein the LAS traffic light detection unit masks a portion corresponding to the ground height in the LAS data.
The method of claim 6, wherein the LAS traffic light detector detects a traffic light object pattern symbol corresponding to a traffic light through symbol pattern matching in the region of interest by projecting LAS data to the region of interest detected by the pixel coordinate detection unit in an omnidirectional direction. Traffic light location storage device, characterized in that.
The method of claim 6, wherein the position coordinate processing unit detects the distance to the traffic light detected by the LAS traffic light detection unit, and detects the location coordinates of the traffic light using the detected distance to the traffic light and the current position of the vehicle. A traffic light location storage device.
detecting, by the mobile mapping system, MMS (Mobile Mapping System) data obtained by surveying the road and surroundings while driving on the actual road;
detecting, by a recognition unit, a traffic light using image data among MMS data detected by the mobile mapping system, and detecting a traffic light using LAS data according to whether the current location of the vehicle is included in the set location according to the detection result; and
Detecting and storing the position coordinate processing unit position coordinates of the traffic light detected by the recognition unit,
The recognition unit detects a traffic light from image data among the MMS data detected in the mobile mapping system by machine learning, and if the detection result is correct, transmits the detection result to the location coordinate processing unit, and if the detection result is not detected, according to the current location A traffic light location storage method, characterized in that the traffic light is detected by using the LAS data detected by the mobile mapping system.
11. The method of claim 10, wherein detecting the traffic light comprises:
detecting, by a first recognition unit, a traffic light from image data among the MMS data detected by the mobile mapping system by machine learning, and transmitting the detection result to the location coordinate processing unit if the detection result is correct;
terminating the process of detecting the position coordinates of the traffic light if the detection result of the first recognition unit is an incorrect answer by the area determination unit, and determining whether the current location is within a preset setting area if the detection result is not detected; and
and recognizing a traffic light by using the LAS data detected by the mobile mapping system if the second recognition unit determines that the current location is within the set area as a result of the determination of the area determination unit.
12. The method of claim 11, wherein the set zone includes at least one of an intersection zone, a crosswalk zone and a child sanctuary zone.
The method of claim 11, wherein in the step of detecting a traffic light using the LAS data,
The recognition unit processes the image data of the mobile mapping system as a color feature of a traffic light to detect pixel coordinates of a region of interest estimated as a traffic light based on a color value of a traffic light, and uses the LAS data of the mobile mapping system to determine the region of interest. A method of storing a traffic light location, comprising detecting a traffic light object corresponding to pixel coordinates.
The method of claim 13, wherein in the step of detecting a traffic light using the LAS data,
The recognition unit projects the LAS data on the region of interest in omnidirectional directions to detect a traffic light object pattern symbol corresponding to a traffic light through symbol pattern matching in the region of interest.
The method of claim 13, wherein in the step of detecting a traffic light using the LAS data,
The recognition unit is a method for storing the location of a traffic light, characterized in that the mask processing the portion corresponding to the ground level in the LAS data.

Images