WO2021115455A1

WO2021115455A1 - Traffic information identification and smart traveling method, device, apparatus, and storage medium

Info

Publication number: WO2021115455A1
Application number: PCT/CN2020/135926
Authority: WO
Inventors: 付万增; 王哲; 石建萍
Original assignee: 上海商汤临港智能科技有限公司
Priority date: 2019-12-13
Filing date: 2020-12-11
Publication date: 2021-06-17
Also published as: CN112991791B; JP2022540084A; CN112991791A; KR20220015488A

Abstract

A traffic information identification and smart traveling method, a device, an apparatus, and a storage medium, for improving accuracy of traffic information identification results. The method comprises: acquiring map data, positioning information of a mobile apparatus, and sensing data of the environment of the mobile apparatus collected by at least one sensor (S101), wherein the environment comprises at least one first traffic light; performing traffic light detection on the sensing data to obtain sensing information of the first traffic light (S102); extracting, on the basis of first positioning information, stored information of at least one second traffic light in the map data (S103); performing matching processing on the sensing information of the first traffic light and the stored information of the second traffic light to obtain matching information (S104); and outputting, at least according to the sensing information of the first traffic light and the matching information, an identification result of the first traffic light in the environment (S105).

Description

Traffic information recognition and intelligent driving method, device, equipment and storage medium

Cross-reference statement

This application claims the priority of the Chinese patent application with application number 201911285926.9 filed with the Chinese Patent Office on December 13, 2019, the entire content of which is incorporated into this application by reference.

Technical field

The embodiments of the present application relate to a method, device, equipment, and storage medium for traffic information identification and intelligent driving.

Background technique

Traffic light recognition plays a vital role in the realization of intelligent transportation. Taking automatic driving as an example, traffic light recognition guides the driving of automatic driving. For the detection of traffic lights, it is very susceptible to environmental factors such as lighting conditions, light intensity, weather, etc., and the shape of traffic lights varies greatly, resulting in inaccurate detection results of traffic lights.

Summary of the invention

The embodiments of the present application provide a method, device, equipment, and storage medium for traffic information identification and intelligent driving.

In a first aspect, an embodiment of the present application provides a method for identifying traffic information, including: acquiring map data, first positioning information of a movable device, and sensing data collected by at least one sensor for the environment in which the movable device is located; wherein , The environment includes at least one first traffic light; performing traffic light detection on the perception data to obtain the perception information of the first traffic light; and extracting at least one of the map data based on the first positioning information The stored information of the second traffic light; the sensing information of the first traffic light and the stored information of the second traffic light are matched to obtain matching information; at least according to the sensing information of the first traffic light and the Matching information, outputting the recognition result of the first traffic light in the environment.

Optionally, the method further includes: extracting a first road connection relationship indicated by the second traffic light in the map data based on the first positioning information; Information and the matching information, outputting the recognition result of the first traffic light in the environment, including: outputting according to the perception information of the first traffic light, the matching information and the first road connection relationship The recognition result of the first traffic light in the environment and the second road connection relationship indicated by it.

Optionally, the sensor includes at least one camera; the perception data is an environmental image of the environment where the movable device is collected through the camera; and the detection of traffic lights is performed on the perception data to obtain the first The perception information of a traffic light includes: performing traffic light detection on the environment image to obtain the perception information of the first traffic light.

Optionally, the sensor includes at least one lidar; the perception data is laser point cloud data of the environment where the movable device is collected by the lidar; the traffic light detection is performed on the perception data, Obtaining the perception information of the first traffic light includes: performing traffic light detection on the laser point cloud data to obtain the perception information of the first traffic light.

Optionally, the extracting storage information of at least one second traffic light in the map data based on the first positioning information includes: determining that the first traffic light is in the environment according to the first positioning information According to the second positioning information, search the corresponding storage information of the second traffic light in the map data.

Optionally, the performing matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain matching information includes: mapping the second traffic light to the first traffic light Under the coordinate system where the light is located, obtain a second mapped traffic light; under the coordinate system where the first traffic light is located, according to the distance between the first traffic light and the second mapped traffic light, and the The ordering principle, quantity, shape and/or size of the first traffic light and the second traffic light are matched to the first traffic light and the second traffic light.

Optionally, the performing matching processing on the sensing information of the first traffic light and the stored information of the second traffic light to obtain matching information includes: dividing the first traffic light into a first traffic light group , And divide the second traffic light into a second traffic light group; match the first traffic light group and the second traffic light group; match the matched first traffic light group and the Each of the first traffic lights and each of the second traffic lights in the second traffic light group are matched one by one.

Optionally, the dividing the first traffic light into a first traffic light group and dividing the second traffic light into a second traffic light group includes: separately dividing the first traffic light and the The second traffic lights are clustered to obtain the first traffic light group and the second traffic light group.

Optionally, the matching the first traffic light group with the second traffic light group includes: mapping the second traffic light group to the coordinate system where the first traffic light group is located, Obtain a second mapped traffic light group; in the coordinate system where the first traffic light group is located, according to the distance between the first traffic light group and the second mapped traffic light group, and the first traffic light group The ordering principle, quantity, shape and/or size of each of the first traffic lights and each of the second traffic lights in the light group and the second traffic light group, the first traffic light group and the second traffic light The two traffic light groups are matched.

Optionally, after clustering the first traffic light and the second traffic light to obtain the first traffic light group and the second traffic light group, the method further includes: determining The remaining first traffic lights after clustering the first traffic lights, and the remaining second traffic lights after clustering the second traffic lights; map the remaining second traffic lights to the The distance between the second remaining mapped traffic light and the remaining first traffic light in the coordinate system where the remaining first traffic light is located, and the remaining first traffic light and the remaining second traffic light The shape and size of the remaining first traffic light and the remaining second traffic light are matched.

Optionally, the method further includes: clustering a plurality of second traffic lights indicating the same first road connection relationship; and determining a connection with the plurality of second traffic lights according to the result of the clustering and the matching information. A plurality of first traffic lights matching the traffic lights; based on a preset principle, the plurality of first traffic lights are calibrated.

Optionally, the method further includes: determining whether the perception information of the first traffic light conforms to the change based on the temporal change rule of the perception information of the first traffic light and the perception information of the historical time period Rule; in the case that the perception information of the first traffic light meets the change rule, it executes at least according to the perception information of the first traffic light and the matching information, outputting an output to at least one of the first traffic lights in the environment The step of identifying the result of a traffic light; in the case that the perception information of the first traffic light does not conform to the change rule, search for the perception information of the historical time period that conforms to the change rule, and execute at least according to the The step of outputting the recognition result of the first traffic light in the environment by the perception information of the historical time period and the matching information.

Optionally, the performing matching processing on the perception information of the first traffic light and the stored information of the second traffic light includes: if the number of the first traffic lights is equal to the number of the second traffic lights , The first traffic light and the second traffic light are matched one by one; if the number of the first traffic light is greater than the number of the second traffic light, the map will be triggered under the following conditions Data update: the number of matching times when the number of the first traffic lights is greater than the number of the second traffic lights is greater than or equal to the preset number, or the preset number of movable devices pass through the area where the first traffic lights are located When the number of the first traffic lights is greater than the number of the second traffic lights; if the number of the first traffic lights is less than the number of the second traffic lights, according to the additional number of the second traffic lights The lights complement the first traffic light.

Optionally, the method further includes: determining the sensing data collected at the current moment according to the matching information and/or the match between the timing change rule of the sensing information of the first traffic light and a preset timing change rule Whether there is a misdetection or missed detection of the first traffic light; determine whether the cumulative number of false detections or missed detections of the first traffic light in the sensing data collected within a certain period of time exceeds the set threshold; if the cumulative number of times exceeds the set Threshold, then trigger an alarm.

In a second aspect, an embodiment of the present application provides an intelligent driving method, including: acquiring a traffic light image collected by an image acquisition unit; using the method described in the first aspect to determine a traffic light recognition result in the traffic light image; based on The traffic light recognition result controls the movable device to travel.

In a third aspect, an embodiment of the present application provides a traffic information identification device, which includes: a first acquisition module, configured to acquire map data, first positioning information of a movable device, and pass at least one device for the environment in which the movable device is located. Perception data collected by a sensor; wherein the environment includes at least one first traffic light; a detection module for detecting traffic lights on the perception data to obtain the perception information of the first traffic light; an extraction module, using Based on the first positioning information, extract the stored information of at least one second traffic light in the map data; a matching module is used to detect the perception information of the first traffic light and the stored information of the second traffic light Performing matching processing to obtain matching information; an output module configured to output a recognition result of the first traffic light in the environment at least according to the perception information of the first traffic light and the matching information.

In a fourth aspect, an embodiment of the present application provides an intelligent driving device, including: a second acquisition module, which acquires traffic light images collected by an image acquisition unit; and an identification module, which is used to identify the traffic using the method described in the first aspect The result of traffic light recognition in the light image; a control module for controlling the mobile device to drive based on the result of the traffic light recognition.

In a fifth aspect, an embodiment of the present application provides a traffic information identification device, including: a memory; and a processor; wherein the memory stores a computer program, and the computer program is configured to be executed by the processor to implement such as The method described in the first aspect.

In a sixth aspect, an embodiment of the present application provides a smart driving device, including: a memory; and a processor; wherein the memory stores a computer program, and the computer program is configured to be executed by the processor to achieve the following The method described in the two aspects.

In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the methods described in the first and second aspects.

In an eighth aspect, an embodiment of the present application provides a computer program, including computer-readable code, which when the computer-readable code runs on a device, causes a processor in the device to execute the traffic information identification method of the first aspect , Or implement the smart driving method of the second aspect.

The embodiments of the present application provide a method, device, equipment, and storage medium for traffic information identification and intelligent driving. The method includes: acquiring map data, first positioning information of a movable device, and sensing data collected by at least one sensor for the environment in which the movable device is located; and performing traffic light detection on the sensing data to obtain at least one first location. Perception information of traffic lights; based on the first positioning information, extract storage information of at least one second traffic light in the map data; perception information of the first traffic light and storage information of the second traffic light Perform matching processing to obtain matching information; at least according to the perception information of the first traffic light and the matching information, output a recognition result of the first traffic light in the environment.

Since the storage information of the second traffic light is pre-stored in the map data, the corresponding second traffic light in the map can be obtained after positioning in the map data according to the positioning information of the mobile device. Combining the storage information of the second traffic light and the traffic light detection result of the perception data to determine the recognition result of the traffic information, thereby helping to reduce the missed detection rate or the false detection rate, and improve the accuracy of the traffic information recognition result.

Description of the drawings

Fig. 1 is a flowchart of a method for identifying traffic information provided by an embodiment of the application;

Figure 2 is a schematic diagram of the six degrees of freedom of the vehicle provided by an embodiment of the application;

FIG. 3 is a schematic diagram of a traffic light detection result provided by an embodiment of the application;

FIG. 4 is a schematic structural diagram of a traffic information identification device provided by an embodiment of the application;

Fig. 5 is a schematic structural diagram of a smart driving device provided by an embodiment of the application;

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the application.

Through the above drawings, the specific embodiments of the present disclosure have been shown, which will be described in more detail below. These drawings and text description are not intended to limit the scope of the concept of the present disclosure in any way, but to explain the concept of the present disclosure to those skilled in the art by referring to specific embodiments.

Detailed ways

The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

Fig. 1 is a flowchart of a method for identifying traffic information provided by an embodiment of the application. The embodiments of the present application provide a traffic information identification method in response to the above technical problems in the prior art, and the specific steps of the method are as follows:

Step S101: Obtain map data, positioning information of the movable device, and perception data collected by at least one sensor for the environment in which the movable device is located.

In this embodiment, the environment where the mobile device is located includes at least one first traffic light. Movable devices can be vehicles, robots (such as educational cars), smart home devices and other devices. At least one sensor, location information collection unit and detection module are deployed on the mobile device. Among them, the vehicle may be an autonomous driving vehicle or a vehicle equipped with an Advanced Driving Assistant System (ADAS).

Taking the mobile device being an autonomous vehicle as an example, at least one sensor is used to collect information about the environment where the vehicle is located, and the collection result is the perception data of the at least one first traffic light. The aforementioned sensor may include at least one camera or at least one lidar. When the sensor is a camera, the perception data is the environment image of the environment where the vehicle is collected by the camera. The environment image can be obtained from a single image taken by the camera on the vehicle, or a video containing multiple frames of images collected by the camera. The sequence is obtained; in the case that the sensor is a lidar, the perception data is the laser point cloud data of the environment where the vehicle is collected by the lidar.

Optionally, the location information collection unit includes a global positioning system (Global Positioning System, GPS). The GPS deployed on the vehicle can locate the vehicle in real time and obtain the GPS positioning information of the vehicle. The GPS positioning information of the vehicle includes six degrees of freedom positioning information of the vehicle. As shown in Figure 2, take a certain point on the vehicle, such as the center point, as the origin T of the coordinate system, the direction of travel of the vehicle as the x-axis, and the direction perpendicular to the direction of the vehicle on the same horizontal plane as the y-axis, and The direction perpendicular to the upper surface of the vehicle chassis establishes a coordinate system for the z-axis. The six-degree-of-freedom positioning information of the vehicle refers to the position information (Fx, Fy, Fz) and rotation information (Fx, Fy, Fz) of the vehicle on the x-axis, y-axis, and z-axis, respectively.

(My,ψ), (Mz,θ)).

Optionally, the location information collection unit may also be an inertial measurement unit (IMU). The IMU positioning information of the vehicle can be collected as the positioning information of the vehicle. Of course, GPS positioning information can also be combined with IMU positioning information as vehicle positioning information to obtain more reliable positioning results. The fusion information of GPS positioning information and IMU positioning information may be obtained by Kalman filtering on GPS positioning information and IMU positioning information, or may be obtained by calculating the average value of GPS positioning information and IMU positioning information, or calculating a weighted average.

The map data may be a semantic map, a high-precision map or other types of maps, which is not specifically limited in this embodiment. The map data of this embodiment includes abundant road information and topological connection relationships between roads, as well as labeling information such as lane lines, traffic signs, traffic lights, and street lights on each road. Among them, the map data can be pre-stored on the vehicle, or the vehicle can obtain the map data from a third-party device through the network. Among them, the topological connection relationship between roads refers to the connection relationship between a road and other roads that can be directly accessed from this road. For example, when a vehicle travels to an intersection of road A, it can turn left to road B, go straight to road C, and turn right to road D, so road A has a connection relationship with road B, road C, and road D.

Step S102: Perform traffic light detection on the perception data to obtain the perception information of the first traffic light.

This step is to detect whether there are traffic lights in the perception data and the number, shape, size, location and color information of the traffic lights to obtain the detection result of the traffic lights, that is, to detect the perception information of at least one first traffic light.

Among them, the first traffic light refers to the physical traffic light in the environment. The perception information of the first traffic light refers to the information of the first traffic light collected by the sensor, such as at least one of quantity, shape, size, position, and color information.

In an optional implementation manner, when the sensor includes a camera, performing traffic light detection on the perception data to obtain the perception information of the first traffic light includes: performing traffic light detection on the environment image to obtain the first traffic light Perception information. Optionally, traffic light detection can be performed on the environmental image through the detection model, for example, the environmental image is input into the detection model to perform traffic light detection through the detection model. Among them, the detection model can be obtained by training the neural network. For example, the neural network is trained through a variety of sample images including different traffic lights and their labels to obtain a detection model; where the labels include information on the number, shape, size, location, and color of traffic lights and traffic lights in the sample images Of the label. Input the environment image collected by the camera into the trained neural network, that is, the detection model, which can detect the virtual traffic light in the environment image, that is, the perception information of the first traffic light, including its number, shape, size, location, color and other information . Among them, the color information includes three colors of red, yellow, and green; the shape information includes circles, arrows, and so on.

In another optional implementation manner, when the sensor includes lidar, performing traffic light detection on the sensing data to obtain the sensing information of the first traffic light includes: performing traffic light detection on the laser point cloud data to obtain Perception information of the first traffic light.

Step S103: Extract storage information of at least one second traffic light in the map data based on the positioning information of the movable device.

In this embodiment, the storage information of the second traffic light refers to the information of the virtual traffic light pre-stored in the map data. For example, according to the principle that traffic lights are usually set at intersections in the real environment, the second traffic lights are correspondingly labeled at locations where there is a connection relationship between roads in the map data.

Taking an autonomous vehicle as an example, this step is to determine the map location of the autonomous vehicle based on the location information of the autonomous vehicle, and then query the second traffic light around the map location in the map data. The map location refers to the corresponding location in the map data of the geographic location determined according to the positioning information of the autonomous vehicle.

Optionally, extracting the stored information of at least one second traffic light in the map data based on the location information of the movable device includes: determining the location information of the first traffic light in the environment according to the location information of the movable device; The location information of the first traffic light in the environment, and the corresponding storage information of the second traffic light is searched in the map data.

Optionally, the data involved in the storage information of the second traffic light includes information such as its number, shape, size, and location. For example, after finding the corresponding second traffic light in the map data according to the location information of the first traffic light, the second traffic light is converted to the image coordinate system, and the image is cropped, and the position of the second traffic light is retained as A rectangle with a fixed length and a fixed width in the center, and then input the cropped rectangular image into the detection model for detection, so as to find out the storage information of the second traffic light in the map data, including its number, shape, size, location and other information.

In an optional scenario, the current location of the vehicle in the map data is determined according to the location information of the vehicle, and it is determined whether the distance between the current location and the nearby stop line is less than a preset distance. If the distance between the current location of the vehicle and the stop line is less than the preset distance, step S103 is executed; otherwise, the decision of automatic driving and path planning is performed based on the perception information of the first traffic light. For example, the preset distance is 100 meters, the vehicle location is located in the map data according to the vehicle's location information, the map location of the vehicle is obtained, and the distance between the map location and the nearest stop line near the map location is judged whether the distance is less than 100 meters , If the distance between the map location and the nearest stop line near the map location is less than 100 meters, the storage information of the second traffic light is extracted, otherwise it ends or the autonomous driving decision and path planning are carried out according to the perception information of the first traffic light .

Optionally, if the storage information of the second traffic light is extracted from the map data, step S104 is executed, and if the storage information of the second traffic light is not extracted from the map data, the method ends and the method is executed according to the first traffic light. The perceptual information is used for decision-making and path planning of autonomous driving.

Step S104: Perform matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain matching information.

For example, according to the perception information of the first traffic light including the number, shape, size, location and other information, and the storage information of the second traffic light including the number, shape, size, and location, the first traffic light and the second traffic light One-to-one matching, and the matching information between the perception information of the first traffic light and the stored information of the second traffic light is obtained.

Step S105: Output the recognition result of the first traffic light in the environment at least according to the perception information and matching information of the first traffic light.

Specifically, the matching information is used as the guidance information to correct the perception information of the first traffic light, and the correction result is output to the automatic driving control module of the vehicle as the recognition result of the first traffic light in the environment, so that the automatic driving control module Make decision or path planning based on the recognition result.

For example, according to the color information of the first traffic light, or color information and shape information (arrow or circle), and matching information between the perception information of the first traffic light and the stored information of the second traffic light, the recognition result is obtained.

The embodiment of the application obtains map data, positioning information of a movable device, and perception data collected by at least one sensor for the environment in which the movable device is located; performs traffic light detection on the perception data to obtain at least one first traffic light Perception information; Based on the positioning information of the movable device, extract the stored information of at least one second traffic light in the map data; Perception information of the first traffic light and the stored information of the second traffic light Perform matching processing to obtain matching information; at least according to the perception information of the first traffic light and the matching information, output the recognition result of the first traffic light in the environment.

Optionally, the method of this embodiment further includes: extracting the first road connection relationship indicated by the second traffic light in the map data based on the positioning information of the movable device.

Wherein, outputting the recognition result of the first traffic light in the environment at least according to the perception information and matching information of the first traffic light includes: outputting the recognition result of the first traffic light according to the perception information of the first traffic light, matching information and the first road connection relationship. The recognition result of the first traffic light and the second road connection relationship indicated by it. For example, when the vehicle is driving to a certain intersection, in addition to the perceptual information of the first traffic light in the above embodiment, the second traffic light indicated by the second traffic light can be accurately extracted from the map data based on the positioning information of the vehicle. A road connection relationship.

In this embodiment, the storage information of the second traffic light in the map data also stores the first road connection relationship indicated by the second traffic light. Specifically, in the map data, there is identification information for each lane, for example, each lane is numbered. The connection relationship between roads represents the road turn, that is, the connection relationship between a road and other roads that can be directly accessed from this road. Corresponding to the real world, it can be understood that when a vehicle travels to the intersection of Road1, it can turn left to Road2, can go straight to Road3, and can turn right to Road4, then Road1 and Road2, Road3, Road4 Have a connection relationship. If the road Road1 needs to turn left to Road2 and then turn right to Road5, then Road1 and Road5 do not have the connection relationship described in this embodiment.

In addition, it can also be understood that the road connection relationship may also be a time series connection relationship between one road and the next road. In an embodiment, a two-tuple including time information and road identification information may be used to represent the connection relationship between two roads. For example, the two-tuple is ((t1, Road1), (t2, Road2)), where t1 and t2 represent time information, Road1 and Road2 represent road identification information, and t2>t1, then the above-mentioned two-tuple represents the road Road1 There is a road connection relationship with Road2.

Of course, the above-mentioned representation of the two-tuple is only an example for description, and does not limit this embodiment.

Optionally, there is a corresponding relationship between the second traffic light and the first road connection relationship, and the corresponding relationship includes at least one of the following situations.

The first situation: multiple second traffic lights can collectively indicate the same first road connection relationship.

For example, as shown in Fig. 3, corresponding to the real world, a plurality of first traffic lights (the traffic lights marked A in Fig. 3) together instruct Road1 to turn left to Road2. Then, in the map data, storage information of a plurality of second traffic lights is stored, including the first road connection relationship between Road1 and Road2 indicated by these second traffic lights.

The second situation: a second traffic light indicates a plurality of first road connection relationships.

In one embodiment, corresponding to the real world, suppose that a first traffic light indicates that Road1 turns left to Road2, goes straight to Road3, and turns right to Road4, etc. Then, in the map data, the storage information of the second traffic light will be stored, including the first road connection relationship between the road Road1 indicated by the second traffic light and the roads Road2, Road3, and Road4, respectively.

The third case: a second traffic light indicates a first road connection relationship.

In one embodiment, corresponding to the real world, if a first traffic light only instructs Road1 to turn left to Road2, go straight to Road3, or turn right to Road4, then the map data will store the second traffic light The stored information includes the first road connection relationship between Road1 and Road2, Road3, or Road4 indicated by the second traffic light.

Optionally, the perception information of the first traffic light and the stored information of the second traffic light can be matched one by one to obtain the matching information of the first traffic light and the second traffic light. Then, combining the perception information of the first traffic light and the first road connection relationship indicated by the matching second traffic light, the recognition result of the first traffic light and the second road connection indicated by the first traffic light are determined relationship. For example, the perception information of the first traffic light is circular and the color is green, and the second traffic light that matches it indicates the first road connection relationship between Road1 and Road2, then the first traffic light indicates The second road connection relationship is that it can travel from Road1 to Road2.

In this embodiment, in combination with the first road connection relationship indicated by the second traffic light in the map data, the recognition result of the first traffic light is obtained and the road turning is controlled, which can help the mobile device to accurately understand the traffic light rules. Provide driving decision or path planning for autonomous driving to improve driving safety.

In an optional implementation manner, performing matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain the matching information includes: mapping the second traffic light to the coordinates of the first traffic light Under the system, obtain the second mapped traffic light; in the coordinate system where the first traffic light is located, according to the distance between the first traffic light and the second mapped traffic light, and the ordering principle of the first traffic light and the second traffic light , Quantity, shape and/or size, match the first traffic light with the second traffic light.

In this embodiment, the storage information of the second traffic light stored in the map data includes three-dimensional position information, and the three-dimensional position information is position information in the map coordinate system. When the sensor is a camera and the sensing data is an environmental image, the sensing information of the first traffic light in the environmental image includes two-dimensional position information in the image coordinate system. In order to facilitate subsequent data processing, the first traffic light and the second traffic light are converted to the same coordinate system. For example, map the second traffic light in the map coordinate system to the image coordinate system. Wherein, mapping the second traffic light in the map coordinate system to the image coordinate system includes: converting the three-dimensional position information of the second traffic light in the map coordinate system according to the conversion relationship between the map coordinate system and the camera coordinate system To the camera coordinate system; then according to the camera's internal and external parameters, the position information of the second traffic light in the camera coordinate system is converted to the image coordinate system to obtain the position information of the second traffic light in the image coordinate system. For ease of comparison, the second traffic light in the image coordinate system can be referred to as the second mapped traffic light. Then, in the image coordinate system, according to the location information of the first traffic light and the location information of the second mapped traffic light, the closest first traffic light and the second mapped traffic light are identified as a pair of matching traffic lights. Optionally, for each second mapped traffic light, it may be determined whether the two are matched by the Euclidean distance between the two traffic lights and all the first traffic lights. For example, there are 3 first traffic lights and 3 second mapped traffic lights in the image coordinate system, hereinafter referred to as the first traffic light 1, the first traffic light 2, the first traffic light 3, the second mapped traffic light 1, the first traffic light For the second mapped traffic light 2, the second mapped traffic light 3, for the second mapped traffic light 1, the Euclidean distances between it and the first traffic light 1, the first traffic light 2 and the first traffic light 3 are respectively calculated. And it is determined that the first traffic light corresponding to the minimum Euclidean distance is the first traffic light that matches the second mapped traffic light 1. Similarly, the other second mapped traffic lights can be matched with the first traffic lights through the above method. In one embodiment, in the coordinate system where the first traffic light is located, the first traffic light and the second mapped traffic light can be matched according to a preset distance threshold between the first traffic light and the second mapped traffic light . For example, if the preset distance threshold is set to 2m, any two first traffic lights and second mapped traffic lights with a distance within 2m can be matched. In this embodiment, according to the distance between the first traffic light and the second mapped traffic light, and the ordering principle, quantity, shape and/or size of the first traffic light and the second traffic light, the first traffic light and the second traffic light The two-mapped traffic lights are matched to realize the one-to-one matching of the first traffic light and the second traffic light, and improve the matching accuracy of the first traffic light and the second traffic light. Therefore, in the subsequent process of identifying the first traffic light in the environment by combining the perception information of the first traffic light, the accuracy of the traffic information recognition result is further improved.

In another optional implementation manner, performing matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain the matching information includes: dividing the first traffic light into a first traffic light group, And divide the second traffic light into the second traffic light group; match the first traffic light group and the second traffic light group; match each first traffic light group in the first traffic light group and the second traffic light group that match The lights are matched with each second traffic light one by one.

Optionally, dividing the first traffic light into the first traffic light group and dividing the second traffic light into the second traffic light group includes: clustering the first traffic light and the second traffic light respectively to obtain the first traffic light A traffic light group and a second traffic light group. In actual application scenarios, the first traffic lights located on the same lamp holder can usually be considered as a group of traffic lights. For example, in the process of clustering the first traffic lights, the length and width of the first traffic lights in the same category should be as consistent as possible, and the distance between the first traffic lights is less than the preset value, the preset value can be It is the average of the length and width of the first traffic light. As shown in FIG. 3, the first traffic lights in the two rectangular boxes A and C next to each other in the figure can be considered as a group of first traffic lights. As shown in FIG. 3, there are two groups of first traffic lights.

Among them, the adjacent first traffic lights in the three-dimensional space are also adjacent in the map. Therefore, the second traffic lights can be clustered, including: calculating the Euclidean distance between all the second traffic lights, and The second traffic light whose Euclidean distance is less than or equal to the preset Euclidean distance is identified as a class, that is, the second traffic light group. In this embodiment, when clustering the second traffic lights, a clustering algorithm based on K nearest neighbors, a k-mean clustering algorithm, etc. may also be used. Optionally, in the clustering process, the KDtree data structure may also be used to optimize the efficiency of the above-mentioned clustering process.

Similarly, the first traffic lights can also be clustered based on the clustering of the second traffic lights. For details, please refer to the above-mentioned clustering method of the second traffic lights, which will not be repeated here.

Optionally, matching the first traffic light group with the second traffic light group includes: mapping the second traffic light group to the coordinate system where the first traffic light group is located to obtain the second mapped traffic light group; In the coordinate system where a traffic light group is located, according to the distance between the first traffic light group and the second mapped traffic light group, and each first traffic light and each second traffic light group in the first traffic light group and the second traffic light group The ordering principle, quantity, shape and/or size of traffic lights are matched to the first traffic light group and the second traffic light group.

In this embodiment, when the perception data is an environmental image, after the clustering of the first traffic light and the second traffic light is completed, the clustered second traffic light group can be mapped to the image coordinate system, and then Match the first traffic light category (first traffic light group) and the second traffic light category (second traffic light group). The matching principle is to ensure that each first traffic light and each second traffic light in the first traffic light category and the second traffic light category are consistent with the ordering principles of the traffic lights of the first traffic light category and the second traffic light category The number of the first traffic light and the second traffic light are as close as possible in the image coordinate system (the nearest neighbor principle), and the shape and size of each first traffic light and each second traffic light are as close as possible As consistent as possible. The traffic lights of the first traffic light category and the second traffic light category have the same ordering principle, which means that each first traffic light and each second traffic light in the first traffic light category and the second traffic light category are from left to Sort from the right or from top to bottom. According to the matching principle, the closest first traffic light class and the second traffic light class are selected for matching. If the matching is successful, the number of each first traffic light and each second traffic light is the same and the sorting principle is the same, so the first traffic light can be further matched. Each of the first traffic light and the second traffic light in the traffic light category and the second traffic light category.

For example, after the second traffic light group is mapped to the first traffic light group, each of the first traffic light group and the second traffic light group can be ordered from left to right, and/or from top to bottom. The first traffic light and the second traffic light are sorted separately. Then the first traffic light group and the second traffic light group are matched. The matching principle is that the number of traffic lights in the first traffic light group and the second traffic light group are the same, and the first traffic light group and the second traffic light group are in The Euclidean distance in the image coordinate system is as close as possible (the nearest neighbor principle), and the shape and size of each first traffic light and each second traffic light are as consistent as possible. If the matching is successful, because the number of the first traffic lights and the second traffic lights are the same according to the matching principle and the ordering principle is the same, each first traffic light and the second traffic light can be further matched, that is, according to the order from left to right and/ Or in order from top to bottom, the first traffic light and the second traffic light in the matched first traffic light group and the second traffic light group are matched one by one.

Optionally, in the process of matching the first traffic light and the second traffic light, the following situations may also exist:

The first situation: the number of first traffic lights is less than the number of second traffic lights. In this case, it means that the first traffic light has missed detection. At this time, the first traffic light can be supplemented based on the extra second traffic light.

The second situation: the number of first traffic lights is greater than the number of second traffic lights. In this case, it means that there may be missing labels in the map data. Further, for the relevant geographic location, the number of matching times where the number of the first traffic light is greater than the number of the second traffic light can be counted. If the counted number exceeds the set number, or the preset number of movable devices passes through the location, the first If the number of one traffic light is greater than the number of the second traffic light, it means that the probability of missing the label of the map data is higher, and the update of the map data will be triggered, that is, the map data will be updated according to the matching result. Specifically, the statistics can be performed by the mobile device, and the server can be notified to trigger the update of the map data when the number of statistics for the relevant geographic location exceeds the set number; or the statistics can be performed by the server, that is, whenever the matching result is the first traffic light quantity When the number of second traffic lights is greater than the number of the second traffic lights, all the movable devices report to the server, and the server triggers the update of the map data when the number of counts for the relevant geographic location exceeds the set number or when the report of a preset number of movable devices accumulates.

The third situation: the number of first traffic lights is equal to the number of second traffic lights, then the first traffic light and the second traffic light can be successfully combined one by one according to the shape and size of the first traffic light and the second traffic light match.

Optionally, after clustering the first traffic light and the second traffic light respectively to obtain the first traffic light group and the second traffic light group, the method of the embodiment of the present application further includes: determining to cluster the first traffic light The remaining first traffic lights after the class, and the remaining second traffic lights after clustering the second traffic lights; according to the remaining second traffic lights are mapped to the second remaining in the coordinate system where the remaining first traffic lights are located Map the distance between the traffic light and the remaining first traffic light, as well as the shape and size of the remaining first traffic light and the remaining second traffic light, and match the remaining first traffic light and the remaining second traffic light . For example, after clustering the first traffic light and the second traffic light separately, some of the first or second traffic lights may appear separately. Or, due to the camera angle, the accuracy of the detection model, etc., the matching of the first traffic light category and the second traffic light category fails. For the remaining first traffic lights and the remaining second traffic lights (including all the first traffic lights and all the second traffic lights in the first traffic light category and the second traffic light category released after the matching fails), it can be directly based on The Euclidean distance, shape, and size similarity in the image coordinate system are matched by nearest neighbors, and the remaining first traffic lights and the remaining second traffic lights are matched one by one.

In this embodiment, the first traffic light and the second traffic light are firstly clustered into multiple traffic light categories by means of clustering. The traffic lights indicating different road connection relationships can be grouped into one category. In the process of identifying the traffic information by combining the first traffic light and the matching information, the traffic light group indicating the same road connection relationship is quickly and accurately obtained. In addition, the clustering method can further reduce the false detection rate or the missed detection rate of the first traffic light.

Optionally, in a case where multiple second traffic lights collectively indicate the same first road connection relationship, the method of this embodiment of the present application further includes: clustering multiple second traffic lights that indicate the same first road connection relationship ; According to the result of the clustering, and the matching information of the first traffic light and the second traffic light, determine a plurality of first traffic lights matching the plurality of second traffic lights.

In the real environment, the turning information of the same road may be indicated by multiple first traffic lights. Therefore, it is necessary to cluster the first traffic light and the second traffic light according to the same road connection relationship, that is, the same The first traffic light and the second traffic light of the road turning are clustered into one class respectively.

After that, all turns are enumerated. For all the first traffic lights that indicate the direction of each road, count their signal light types. Because there may be misdetection, the first traffic light signal type indicating the same turn may be different. In one embodiment, the number of signal types of each of these first traffic lights can be counted, and the signal type of the first traffic light with the largest number is selected as the signal type of these first traffic lights. For example, suppose there are 6 first traffic lights instructing the same road Road1 to turn left to Road2, then if 4 of the 6 first traffic lights are all arrows and red in shape, the other 2 The shape of the first traffic light is black and has no shape, then the other two first traffic lights are corrected with the shape and color information of the four first traffic lights, so that the output of the first traffic light after correction can be used for the first traffic light in the environment. The recognition result of a traffic light.

In this embodiment, for the situation where multiple first traffic lights indicate the same road connection relationship, in the case of partial misdetection or missed detection of the first traffic lights, the traffic information identification result can still be accurately obtained.

Optionally, the method of the embodiment of the present application further includes: determining whether the perception information of the first traffic light meets the change rule based on the temporal change rule of the perception information of the first traffic light and the perception information of the historical time period; When the perception information of the first traffic light meets the change rule, the step of outputting the recognition result of the first traffic light in the environment at least according to the perception information and matching information of the first traffic light is executed; If the information does not conform to the change rule, search for the perception information of the historical time period that conforms to the change rule. For example, select the perceptual information that is adjacent in time sequence and conforms to the change rule. In a real environment, according to traffic rules, the first traffic light needs to meet the time sequence change rule of red to green, green to yellow, yellow to red, or the shape and color of the first traffic light change together. The common change of the shape and color of the first traffic light means that when a first traffic light indicates multiple road turning information, for example, the shape of the first traffic light at the first time is an arrow, and the shape of the first traffic light at the second time is a circle. . Among them, at the first moment, the arrow may have three color information of red, green, and yellow respectively; similarly, at the second moment, the circle may have three color information of red, green, and yellow respectively. Therefore, in this embodiment, for each road turning information, the perception information of the first traffic light within a period of time is stored. According to the stored perception information of the historical time period, it is judged whether the perception information of the current first traffic light is reasonable. If it is reasonable, output the latest first traffic light signal type; otherwise, output the sensing information of the first traffic light that is nearest and reasonable to the current time in the historical time period.

Taking the environment image as an example, the perception information of the first traffic light within a period of time is stored, for example, 5 frames of environment images are stored, and the perception information of the first traffic light obtained by performing traffic light detection on the 5 frames of environment images respectively. Assuming that the 4th frame of environment image is the current frame of environment image, compare it with the multiple frames of environment images at different historical moments before the 4th frame of environment image (the 1st to 3rd frame, even the frame image before the 1st frame), judge and detect Whether the perceptual information of the first traffic light obtained from the 4th frame of the environment image satisfies the timing change rule. If the time-sequence change rule is satisfied, the sensory information of the first traffic light obtained by detecting the environment image of the fourth frame is output. If the timing change rule is not met, output the sensing information of the first traffic light that meets the timing change rule obtained by detecting the environmental image of the third frame or the historical moment before the third frame.

In this embodiment, by filtering the illegal data in the perception data according to the time-series change logic of the traffic rules, the accuracy of the traffic information recognition result can be improved.

Optionally, the method of the embodiment of the present application further includes: determining whether the sensing data collected at the current moment has the first time sequence change rule according to the matching information and/or the matching of the time series change rule of the sensing information of the first traffic light and the preset time series change rule. 1. False detection or missed detection of traffic lights; determine whether the cumulative number of false detections or missed detections of the first traffic light in the sensed data collected within a certain period of time exceeds the set threshold; if the cumulative number of times exceeds the set threshold, an alarm is triggered . If the cumulative number of false detections or missed detections of the first traffic light is greater than the set threshold, the automatic driving control will be affected, and even the driving safety will be affected. Therefore, if the cumulative number of false detections or missed detections of the first traffic light is greater than the set threshold, an alarm is triggered. Take the camera to collect information about the surrounding environment of the vehicle as an example. Assuming that the camera collects 30 perception data per second, if there are 30 misdetections or missed detections, an alarm will be issued. The alarm information will be provided to the back-end modules of the automatic driving system, such as a path planning module or a decision control module, and the path planning module or decision control module will combine the alarm information for path planning or decision control. For example, if the arrow lights are detected 30 times within 1 second, but the circular lights are on the map data, the subsequent decision-making control module will control according to the circular lights after receiving the alarm.

Wherein, according to the matching information, it is determined whether there is a misdetection or missed detection of the first traffic light in the collected perception data, which may be that the number of the first traffic light and the second traffic light are inconsistent during the matching process, for example, in the above-mentioned embodiment The first situation is a situation where the number of first traffic lights is less than the number of second traffic lights, and the second situation is a situation where the number of first traffic lights is greater than the number of second traffic lights.

For example, according to the matching of the time series change rule of the first traffic light perception information and the preset time series change rule, it is determined whether there is a misdetection or missed detection of the first traffic light in the collected perception data, including: the first traffic light at the current moment The perceptual information of the light is red, and according to the preset timing change rule, it is inferred that the perceptual information of the first traffic light at the current moment should be green. At this time, it is considered that the perceptual data is misdetected or missed.

In this embodiment, an alarm is triggered when the sensing data is misdetected or missed for more than a predetermined number of times to provide alarm information to the path planning module or decision control module, so that the path planning module or decision control module combines the alarm information to make path planning or decision-making. Control to improve driving safety.

On the basis of the foregoing embodiment, an embodiment of the present application also provides an intelligent driving method, including: acquiring a traffic light image collected by an image acquisition unit; using the traffic information identification method of the foregoing embodiment to perform traffic on the traffic light image Light recognition to obtain a traffic light recognition result; based on the traffic light recognition result, the movable device is controlled to drive. Take an autonomous vehicle as an example. Based on the result of traffic light recognition, the mobile device is controlled to drive, including controlling the autonomous vehicle to drive on the road.

Fig. 4 is a schematic structural diagram of a traffic information identification device provided by an embodiment of the application. The traffic information recognition device provided by the embodiment of the present application can execute the processing flow provided in the embodiment of the traffic information recognition method. As shown in FIG. 4, the traffic information recognition device 40 includes: a first acquisition module 41, a detection module 42, an extraction module 43, The matching module 44 and the output module 45; wherein, the first acquisition module 41 is configured to acquire map data, positioning information of the movable device, and perception data collected by at least one sensor for the environment in which the movable device is located, wherein the The environment includes at least one first traffic light; the detection module 42 is used to perform traffic light detection on the perception data to obtain the perception information of the first traffic light; the extraction module 43 is used to extract map data based on the positioning information of the movable device At least one second traffic light in the storage information; the matching module 44 is used to match the perception information of the first traffic light and the storage information of the second traffic light to obtain the matching information; the output module 45 is used to at least according to the first traffic light Perception information and matching information of a traffic light, and output the recognition result of the first traffic light in the environment.

Optionally, the extraction module 43 is further configured to extract the first road connection relationship indicated by the second traffic light in the map data based on the positioning information of the movable device; the output module 45 is based on at least the perception information of the first traffic light and Matching information, when outputting the recognition result of the first traffic light in the environment, specifically includes: outputting the recognition result of the first traffic light in the environment according to the perception information of the first traffic light, matching information and the first road connection relationship And its indicated second road connection relationship.

Optionally, the sensor includes at least one camera; the sensing data is an environmental image of the environment where the mobile device is located collected by the camera; when the detection module 42 performs traffic light detection on the sensing data to obtain the sensing information of the first traffic light, it specifically includes : Perform traffic light detection on the environment image to obtain the perception information of the first traffic light.

Optionally, the sensor includes at least one lidar; the sensing data is laser point cloud data of the environment where the movable device is collected by lidar; the detection module 42 is performing traffic light detection on the sensing data to obtain the perception of the first traffic light The information includes: detecting traffic lights on the laser point cloud data to obtain the perception information of the first traffic light.

Optionally, when the extracting module 43 extracts the stored information of at least one second traffic light in the map data based on the location information of the movable device, it specifically includes: determining that the first traffic light is in the environment according to the location information of the movable device Location information; according to the determined location information of the first traffic light in the environment, search for the corresponding storage information of the second traffic light in the map data.

Optionally, when the matching module 44 performs matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain the matching information, it specifically includes: mapping the second traffic light to the location where the first traffic light is located. Under the coordinate system, obtain the second mapped traffic light; in the coordinate system where the first traffic light is located, according to the distance between the first traffic light and the second mapped traffic light, and the sorting of the first traffic light and the second traffic light The principle, quantity, shape and/or size, match the first traffic light with the second traffic light.

Optionally, when the matching module 44 performs matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain the matching information, it specifically includes: dividing the first traffic light into a first traffic light group, And divide the second traffic light into the second traffic light group; match the first traffic light group and the second traffic light group; match each first traffic light group in the first traffic light group and the second traffic light group that match The lights are matched with each second traffic light one by one.

Optionally, when the matching module 44 divides the first traffic light into the first traffic light group and divides the second traffic light into the second traffic light group, it specifically includes: dividing the first traffic light and the second traffic light respectively The lights are clustered to obtain the first traffic light group and the second traffic light group.

Optionally, when the matching module 44 matches the first traffic light group with the second traffic light group, it specifically includes: mapping the second traffic light group to the coordinate system where the first traffic light group is located to obtain the second traffic light group. Map the traffic light group; in the coordinate system where the first traffic light group is located, according to the distance between the first traffic light group and the second traffic light group, and each of the first traffic light group and the second traffic light group The ordering principle, quantity, shape and/or size of a traffic light and each second traffic light are matched to the first traffic light group and the second traffic light group.

Optionally, the matching module 44 is further configured to determine the remaining first traffic lights after clustering the first traffic lights, and the remaining second traffic lights after clustering the second traffic lights; according to the remaining second traffic lights The traffic light is mapped to the distance between the second remaining mapped traffic light and the remaining first traffic light in the coordinate system where the remaining first traffic light is located, and the shape of the remaining first traffic light and the remaining second traffic light And size, match the remaining first traffic light with the remaining second traffic light.

Optionally, the device 40 further includes: a clustering module 46, configured to cluster a plurality of second traffic lights indicating the same first road connection relationship; a first determining module 47, configured to perform clustering based on the result of the clustering, and The matching information is used to determine a plurality of first traffic lights that match the plurality of second traffic lights; and based on a preset principle, the plurality of first traffic lights are corrected.

Optionally, the device 40 further includes: a second determining module 48, configured to determine whether the perception information of the first traffic light is based on the time sequence change rule of the perception information of the first traffic light and the perception information of the historical time period Comply with the change rule; in the case that the perception information of the first traffic light meets the change rule, perform the step of outputting the recognition result of the first traffic light in the environment at least according to the perception information and matching information of the first traffic light; and , In the case that the perception information of the first traffic light does not conform to the change rule, search for the perception information of the historical time period that conforms to the change rule, for example, select the perception information of the first traffic light that is adjacent in time sequence and conforms to the change rule, And execute the step of outputting the recognition result of the first traffic light in the environment at least according to the perception information and matching information of the historical time period.

Optionally, when the matching module 44 performs matching processing on the perception information of the first traffic light and the stored information of the second traffic light, it specifically includes: if the number of the first traffic light is equal to the number of the second traffic light, then The first traffic light and the second traffic light are matched one by one; if the number of the first traffic light is greater than the number of the second traffic light, the update of the map data is triggered under the following conditions: the number of the first traffic light is greater than the second traffic light The number of matching times of the number of traffic lights is greater than or equal to the preset number, or when the preset number of movable devices pass through the area where the first traffic light is located, the number of the first traffic light is greater than the number of the second traffic light; If the number of one traffic light is less than the number of second traffic lights, the first traffic light is complemented according to the extra second traffic lights.

Optionally, the device 40 further includes: a third determining module 49, configured to determine the current time collection based on the matching information and/or the match between the timing change rule of the first traffic light perception information and the preset timing change rule Whether the first traffic light misdetection or missed detection exists in the perception data; and whether the cumulative number of misdetection or missed detection of the first traffic light collected within a certain period of time exceeds the set threshold; and when the cumulative number exceeds the set threshold In the case of a threshold value, an alarm is triggered.

The traffic information identification device of the embodiment shown in FIG. 4 can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.

Fig. 5 is a schematic structural diagram of a smart driving device provided by an embodiment of the application. The smart driving device provided in the embodiment of the present application can execute the processing flow provided in the smart driving method embodiment. As shown in FIG. 5, the smart driving device 50 includes: a second acquisition module 51, an identification module 52, and a control module 53; a second acquisition module 51, which acquires traffic light images collected by an image acquisition unit; and an identification module 52, which is used to The traffic information recognition method of the foregoing embodiment performs traffic light recognition on the traffic light image to obtain the traffic light recognition result; the control module 53 is used to control the movable device to drive based on the traffic light recognition result.

The smart driving device of the embodiment shown in FIG. 5 can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.

FIG. 6 is a schematic structural diagram of an electronic device provided by an embodiment of the application. The electronic device may specifically be the traffic information identification device or the smart driving device in the above-mentioned embodiment. The electronic device provided in the embodiment of the present application can execute the processing flow provided in the embodiment of the traffic information recognition method or the intelligent driving method. As shown in FIG. 6, the electronic device 60 includes: a memory 61, a processor 62, a computer program, and a communication interface 63; Wherein, the computer program is stored in the memory 61 and is configured to be executed by the processor 62 to execute the technical solutions of the above embodiments of the traffic information identification method or the intelligent driving method.

The electronic device of the embodiment shown in FIG. 6 can be used to implement the technical solutions of the foregoing method embodiments, and its implementation principles and technical effects are similar, and will not be repeated here.

In addition, an embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the traffic information identification method or the intelligent driving method described in the foregoing embodiment.

The embodiments of the present application also provide a computer program, including computer-readable code, which when the computer-readable code runs on a device, causes a processor in the device to execute to implement the traffic information recognition described in the foregoing embodiment Method or smart driving method.

In the several embodiments provided in this application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.

The above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The above-mentioned software functional unit is stored in a storage medium, and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute the method described in each embodiment of the present application. Part of the steps. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Those skilled in the art can clearly understand that for the convenience and conciseness of the description, only the division of the above-mentioned functional modules is used as an example. In practical applications, the above-mentioned functions can be allocated by different functional modules as required, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not repeated here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application, not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or equivalently replace some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application. range.

Claims

A method for identifying traffic information, including:

Acquiring map data, first positioning information of the movable device, and perception data collected by at least one sensor for the environment in which the movable device is located; wherein the environment includes at least one first traffic light;

Performing traffic light detection on the perception data to obtain the perception information of the first traffic light;

Extracting stored information of at least one second traffic light in the map data based on the first positioning information;

Performing matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain matching information;

At least according to the perception information of the first traffic light and the matching information, outputting a recognition result of the first traffic light in the environment.
The method of claim 1, wherein:

The sensor includes at least one camera;

The perception data is an environmental image of the environment where the movable device is located, which is collected by the camera;

The performing traffic light detection on the perception data to obtain the perception information of the first traffic light includes:

Performing traffic light detection on the environment image to obtain the perception information of the first traffic light.
The method according to claim 1 or 2, characterized in that:

The sensor includes at least one lidar;

The sensing data is laser point cloud data of the environment where the movable device is located, collected by the laser radar;

The performing traffic light detection on the perception data to obtain the perception information of the first traffic light includes:

Perform traffic light detection on the laser point cloud data to obtain the perception information of the first traffic light.
The method according to any one of claims 1 to 3, wherein the extracting storage information of at least one second traffic light in the map data based on the first positioning information comprises:

Determining second positioning information of the first traffic light in the environment according to the first positioning information;

According to the second positioning information, search for corresponding stored information of the second traffic light in the map data.
The method according to any one of claims 1 to 4, wherein the matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain the matching information comprises:

Mapping the second traffic light to the coordinate system where the first traffic light is located to obtain a second mapped traffic light;

In the coordinate system where the first traffic light is located, according to the distance between the first traffic light and the second mapped traffic light, and the ordering principle of the first traffic light and the second traffic light , Quantity, shape and/or size, matching the first traffic light and the second traffic light.
The method according to any one of claims 1 to 4, wherein the matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain the matching information comprises:

Dividing the first traffic light into a first traffic light group, and dividing the second traffic light into a second traffic light group;

Matching the first traffic light group and the second traffic light group;

Match each of the first traffic lights and each of the second traffic lights in the matched first traffic light group and the second traffic light group one by one.
The method according to claim 6, wherein the matching the first traffic light group with the second traffic light group comprises:

Mapping the second traffic light group to the coordinate system where the first traffic light group is located to obtain a second mapped traffic light group;

In the coordinate system where the first traffic light group is located,

According to the distance between the first traffic light group and the second mapped traffic light group, and each of the first traffic light and each of the first traffic light group and the second traffic light group The ordering principle, quantity, shape and/or size of the second traffic lights are matched to the first traffic light group and the second traffic light group.
The method according to claim 6 or 7, wherein the dividing the first traffic light into a first traffic light group and dividing the second traffic light into a second traffic light group comprises:

Clustering the first traffic light and the second traffic light respectively to obtain the first traffic light group and the second traffic light group.
8. The method according to claim 8, wherein the clustering of the first traffic light and the second traffic light is performed to obtain the first traffic light group and the second traffic light group After that, the method further includes:

Determining the remaining first traffic lights after clustering the first traffic lights, and the remaining second traffic lights after clustering the second traffic lights;

According to the distance between the remaining second traffic light mapped to the second remaining mapped traffic light in the coordinate system where the remaining first traffic light is located, and the remaining first traffic light, and the remaining first traffic light The shape and size of the first traffic light and the remaining second traffic light are matched with the remaining first traffic light and the remaining second traffic light.
The method according to any one of claims 1-9, wherein the method further comprises:

Extracting the first road connection relationship indicated by the second traffic light in the map data based on the first positioning information;

The outputting a recognition result of the first traffic light in the environment at least according to the perception information of the first traffic light and the matching information includes:

According to the perception information of the first traffic light, the matching information and the first road connection relationship, outputting the recognition result of the first traffic light in the environment and the second road connection relationship indicated by it.
The method according to claim 10, wherein the method further comprises:

Clustering multiple second traffic lights indicating the same first road connection relationship;

Determine, according to the result of the clustering and the matching information, a plurality of first traffic lights that match the plurality of second traffic lights;

Based on a preset principle, the plurality of first traffic lights are corrected.
The method according to any one of claims 1-11, wherein the method further comprises:

Determine whether the perception information of the first traffic light conforms to the change rule based on the temporal change rule of the perception information of the first traffic light and the perception information of the historical time period;

In the case that the perception information of the first traffic light meets the change rule, it is executed to output a response to the first traffic light in the environment at least according to the perception information of the first traffic light and the matching information. The steps of the recognition result;

In the case that the perception information of the first traffic light does not conform to the change rule, search for the perception information of the historical time period that conforms to the change rule, and execute at least according to the perception information of the historical time period and the change rule. The matching information, the step of outputting the recognition result of the first traffic light in the environment.
The method according to any one of claims 1-12, wherein the matching processing on the perception information of the first traffic light and the stored information of the second traffic light comprises:

If the number of the first traffic lights is equal to the number of the second traffic lights, matching the first traffic lights and the second traffic lights one by one;

If the number of the first traffic lights is greater than the number of the second traffic lights, the update of the map data is triggered under the following conditions:

The number of matching times in which the number of the first traffic lights is greater than the number of the second traffic lights is greater than or equal to a preset number, or

When a preset number of movable devices pass through the area where the first traffic lights are located, the number of the first traffic lights is greater than the number of the second traffic lights;

If the number of the first traffic lights is less than the number of the second traffic lights, the first traffic lights are supplemented according to the extra second traffic lights.
The method according to any one of claims 1-13, wherein the method further comprises:

According to the matching information and/or the matching between the timing change rule of the sensing information of the first traffic light and the preset timing change rule, it is determined whether the sensing data collected at the current moment has a misdetection or omission of the first traffic light. Check

Determining whether the cumulative number of false detections or missed detections of the first traffic light in the perception data collected within a certain period of time exceeds a set threshold;

If the cumulative number of times exceeds the set threshold, an alarm is triggered.
An intelligent driving method, including:

Obtain the traffic light image collected by the image collecting unit;

Use the method according to any one of claims 1 to 14 to perform traffic light recognition on the traffic light image to obtain a traffic light recognition result;

The movable device is controlled to drive based on the traffic light recognition result.
A traffic information recognition device includes:

The first acquisition module is configured to acquire map data, first positioning information of the movable device, and perception data collected by at least one sensor for the environment in which the movable device is located; wherein, the environment includes at least one first traffic light;

A detection module, configured to perform traffic light detection on the perception data to obtain the perception information of the first traffic light;

An extraction module, configured to extract storage information of at least one second traffic light in the map data based on the first positioning information;

A matching module, configured to perform matching processing on the perception information of the first traffic light and the stored information of the second traffic light to obtain matching information;

The output module is configured to output the recognition result of the first traffic light in the environment at least according to the perception information of the first traffic light and the matching information.
An intelligent driving device, including:

The second acquisition module acquires traffic light images collected by the image acquisition unit;

The recognition module is configured to perform traffic light recognition on the traffic light image by using the method according to any one of claims 1-14 to obtain a traffic light recognition result;

The control module is used to control the movable device to drive based on the traffic light recognition result.
A traffic information recognition device, including:

Memory; and processor;

Wherein, the memory stores a computer program, and the computer program is configured to be executed by the processor to implement the method according to any one of claims 1-14.
An intelligent driving equipment, including:

Memory; and processor;

Wherein, the memory stores a computer program, and the computer program is configured to be executed by the processor to implement the method according to claim 15.
A computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1-15 is realized.
A computer program comprising computer readable code, which when the computer readable code runs on a device, causes a processor in the device to execute the method for identifying traffic information according to any one of claims 1-14, or The smart driving method of claim 15 is implemented.