WO2022247299A1

WO2022247299A1 - Indicator lamp state recognition

Info

Publication number: WO2022247299A1
Application number: PCT/CN2022/070699
Authority: WO
Inventors: 黄超; 陈熙; 姚为龙
Original assignee: 上海仙途智能科技有限公司
Priority date: 2021-05-27
Filing date: 2022-01-07
Publication date: 2022-12-01
Also published as: CN115249407B; CN115249407A

Abstract

An indicator lamp state recognition method and apparatus, and an electronic device, a storage medium and a product. The method comprises: projecting location information, which is acquired from an electronic map at the current moment and comprises a target traffic signal lamp, into an on-board captured image, and effectively positioning a region of interest, where a traffic signal lamp is located, in the on-board captured image; recognizing traffic signal lamps in the region of interest, and precisely selecting a traffic signal lamp that needs to be focused on when passing through the current intersection; and then fusing a testing result of the traffic signal lamp that needs to be focused on at the current intersection with an output result of each frame of the traffic signal lamp within a time period before, and recognizing accurately and outputting a display attribute state of the traffic signal lamp, and thereby improving the accuracy of recognizing the state of the traffic signal lamp.

Description

Indicator status identification

technical field

The present invention relates to the field of traffic technology, in particular to a method, device, electronic equipment, computer-readable storage medium and computer program product for state identification of indicator lights.

Background technique

With the continuous improvement of road complexity and the increasing number of vehicles, driving safety has become a topic of widespread concern. For example, how to determine the indication status of a certain crossroad or adjacent crossroad traffic lights on the road to ensure driving safety. sex etc.

In related technologies, the commonly used way to determine the state of traffic lights is to first obtain an image containing traffic lights through the vehicle-mounted camera, and then locate the traffic lights in the image on the electronic map according to the vehicle positioning device. area, and finally detect and identify the current state of traffic lights in the area of interest, and output the current recognition results. However, in the related art, due to various uncertain factors such as the error of the vehicle positioning device, the deviation of the camera parameters of the vehicle-mounted camera, and the change of the size and position of the traffic signal light in the captured image during the driving of the vehicle, it is impossible to accurately identify the vehicle. The current state of the traffic lights that need attention when passing the current intersection.

Therefore, how to accurately identify the state of the traffic signal lights that the vehicle needs to pay attention to when passing the current intersection is a technical problem to be solved at present.

Contents of the invention

The present invention provides a method, device, electronic equipment, computer-readable storage medium and computer program product for identifying the state of an indicator light, so as to at least solve the problem that the current intersection cannot be accurately identified when the vehicle passes the current intersection in the related art. Concerned about technical issues with the status of traffic lights.

Technical scheme of the present invention is as follows:

According to the first aspect of the embodiments of the present invention, there is provided a method for identifying the state of an indicator light, comprising: obtaining the traffic signal image in front of the vehicle taken at the current moment, and the location information of the vehicle at the current moment; if obtained from the electronic map The location information of the traffic lights that the vehicle currently needs to pass through the intersection, then determine the ROI of the location information of the traffic lights that the vehicle currently needs to pass through the intersection on the traffic signal image according to the positioning information of the vehicle at the current moment ; detect and identify the traffic lights in the region of interest ROI to obtain the state of the traffic lights that need to pass through the intersection at the current moment; update the state of the historical tracking target at the current moment, wherein the historical tracking target is the current The detection and recognition results of traffic lights within a period of time before the moment; determine the first matching result between the state of the traffic lights that need to pass through the intersection at the current moment and the state of the historical tracking target at the current moment; if the first matching result is the The historical tracking target matches the state of the traffic lights that need to pass through the intersection at the current moment, then obtain the historical display attribute information of the historical tracking target corresponding to the traffic lights matched at the current moment, wherein the matched traffic lights are the target traffic lights If the position information of the target traffic signal light is included in the electronic map at the current moment, then determine the second matching result between the projected position of the target traffic signal light on the traffic signal light image and the historical tracking target; According to the second matching result, the projected position of the target traffic signal light is matched to the display attribute state of the corresponding historical tracking target, the display attribute state includes the color state and shape state of the target traffic signal light; output the target The display property state of the traffic light.

According to the second aspect of the embodiments of the present invention, there is provided a device for identifying the status of an indicator light, including: a first acquisition module, configured to acquire a traffic signal image in front of a vehicle at the current moment, and the location information of the vehicle at the current moment The second acquisition module is used to acquire the position information of the traffic lights that the vehicle currently needs to pass through the intersection in the electronic map; the first determination module is used to acquire the current position of the vehicle in the electronic map by the second acquisition module. When the location information of the traffic lights passing through the intersection is needed, determine the position information of the traffic lights that the vehicle currently needs to pass through the intersection on the traffic signal image according to the positioning information of the vehicle at the current moment acquired by the first acquisition module Region of interest ROI; Detection and identification module, used to detect and identify the traffic lights in the region of interest ROI, to obtain the status of the traffic lights that need to pass through the intersection at the current moment; Update module, used to update the historical tracking target in the location The state at the current moment, wherein the historical tracking target is the detection and recognition result of the traffic signal lights in the previous period of time at the current moment; The first matching result of the state of the tracking target at the current moment; the third acquisition module is used to match the historical tracking target with the traffic that needs to pass through the intersection at the current moment in the first matching result determined by the second determination module When the traffic signal is in the state of the traffic signal, obtain the historical display attribute information of the historical tracking target corresponding to the traffic signal matched at the current moment, wherein the matched traffic signal is the target traffic signal; the fourth acquisition module is used to obtain the The electronic map includes the position information of the target traffic light at the current moment; the third determination module is configured to determine the position information of the target traffic light in the electronic map when the fourth acquisition module acquires the position information of the target traffic light in the electronic map. The second matching result of the projection position of the target traffic signal light on the traffic signal light image and the historical tracking target; the fifth acquisition module is used to obtain the matching of the projection position of the target traffic signal light according to the second matching result To the display attribute state of the corresponding historical tracking target, the display attribute state includes the color state and shape state of the traffic signal light; an output module, configured to output the display attribute of the target traffic signal light acquired by the fifth acquisition module state.

According to a third aspect of the embodiments of the present invention, there is provided an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement The method for identifying the state of the indicator light as described above.

According to a fourth aspect of the embodiments of the present invention, there is provided a computer-readable storage medium, when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the above-mentioned How to recognize the state of the indicator light.

According to a fifth aspect of the embodiments of the present invention, there is provided a computer program product, including a computer program or an instruction, and when the computer program or instruction is executed by a processor, the method for identifying the status of an indicator light as described above is implemented.

The technical solution provided by the embodiments of the present invention may at least include the following beneficial effects: the method for identifying the state of the indicator light provided by the embodiments of the present invention projects the position information including the target traffic light obtained from the electronic map at the current moment onto the vehicle In the captured image, the area of interest where the traffic signal light is located in the vehicle-mounted image is effectively located, and the traffic signal light in the area of interest is identified to accurately select the traffic signal light that needs to be paid attention to when passing the current intersection , and then fuse the detection results of the current frame with the output results of the previous frames of the traffic light to accurately identify and output the state of the traffic light, thereby improving the accuracy of traffic signal state recognition.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention.

Description of drawings

The accompanying drawings here are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and used together with the specification to explain the principle of the present invention, and do not constitute an improper limitation of the present invention.

Fig. 1 is a flow chart of a method for identifying the state of an indicator light provided by an embodiment of the present invention.

Fig. 2 is a flow chart of an application example of a method for identifying the state of an indicator light provided by an embodiment of the present invention.

Fig. 3 is a block diagram of a device for identifying the state of an indicator light provided by an embodiment of the present invention.

Fig. 4 is a schematic structural diagram of a first determining module provided by an embodiment of the present invention.

FIG. 5 is a schematic structural diagram of a projection module provided by an embodiment of the present invention.

Fig. 6 is a schematic structural diagram of a second determining module provided by an embodiment of the present invention.

Fig. 7 is a schematic structural diagram of a third determining module provided by an embodiment of the present invention.

Fig. 8 is another block diagram of a device for identifying the state of an indicator light provided by an embodiment of the present invention.

Fig. 9 is another block diagram of a device for identifying the state of an indicator light provided by an embodiment of the present invention.

Fig. 10 is a structural block diagram of an electronic device provided by an embodiment of the present invention.

Fig. 11 is a structural block diagram of a device with indicator light status recognition provided by an embodiment of the present invention.

Detailed ways

In order to enable ordinary persons in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples do not represent all implementations consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with aspects of the invention as recited in the appended claims.

Fig. 1 is a flow chart of a method for identifying the status of an indicator light according to an exemplary embodiment. The method for identifying the status of an indicator light can be used in a terminal, and the terminal implementation device can be an electronic device such as a vehicle-mounted terminal, which is not limited here. As shown in FIG. 1 , this embodiment may include the following steps: Step 101 , acquiring the traffic signal image in front of the vehicle at the current moment and the location information of the vehicle at the current moment.

Wherein, in this embodiment, when the vehicle-mounted terminal detects the shooting instruction, it starts the vehicle-mounted camera to take pictures. The vehicle-mounted camera may be integrated on the recorder, or independently deployed on the vehicle, which is not limited in this embodiment.

In this step, the vehicle-mounted terminal can locate the positioning information of the current vehicle through the satellite navigation system, of course, it can also locate the positioning information of the current vehicle in other ways, wherein, the specific process of the vehicle-mounted terminal positioning through the satellite positioning and navigation system, It is a well-known technology for those skilled in the art, so it will not be repeated here.

Step 102, if the location information of the traffic lights that the vehicle needs to pass through the intersection is obtained in the electronic map, then determine the location information of the traffic lights that the vehicle needs to pass through the intersection according to the positioning information of the vehicle at the current moment. A region of interest ROI on a traffic signal image.

In this step, the vehicle-mounted terminal first judges whether the position information of the traffic signal lights that the vehicle currently needs to pass through the intersection in the electronic map has been obtained, and if it is obtained, first according to the positioning information of the vehicle at the current moment, the location information of the traffic lights that the vehicle currently needs to pass through the intersection The position information of the traffic signal lights is projected onto the traffic signal image to obtain the two-dimensional coordinate information of the traffic signal lights that need to pass through the intersection on the traffic signal image; A region of interest (ROI, region of interest) on the traffic signal image of the traffic lights passing through the intersection. The process of obtaining the region of interest on the traffic signal image of the traffic signal light that needs to pass through the intersection with the projected projection point as the center is a well-known technology and will not be repeated here. Among them, ROI, in machine vision and image processing, outlines the area to be processed from the processed image in the form of a box, circle, ellipse, irregular polygon, etc., called the region of interest.

Wherein, according to the positioning information of the vehicle at the current moment, the position information of the traffic lights that the vehicle needs to pass through the intersection is projected onto the traffic signal image, and the traffic signal lights that need to pass the intersection are obtained in the traffic signal image. The two-dimensional coordinate information on the above may include: the distance between the position information of the traffic lights that need to pass the intersection and the location information of the vehicle; according to the distance, map the traffic lights that need to pass the intersection to In the traffic signal light image, the two-dimensional coordinate information projected on the traffic signal light image by the traffic signal light that needs to pass through the intersection currently is obtained.

That is to say, in this embodiment, it is necessary to first obtain the distance between the position of the traffic signal light that needs to pass through the intersection and the current position of the vehicle, and then obtain the position information (ie, the three-dimensional image) of the traffic signal light that needs to pass the intersection. Mapped to the traffic signal image recorded by the on-board camera (the image is a two-dimensional image), thereby obtaining the projection result of the traffic signal light that needs to pass through the intersection on the traffic signal image, wherein the projection result is two-dimensional coordinate information. That is, in this embodiment, in order to realize the mapping of the position information of the traffic lights currently passing through the intersection on the traffic signal image, it is necessary to clarify the relative positional relationship between the traffic lights currently passing through the intersection and the current vehicle. One method adopted in the embodiment of the present invention is to use the endpoint coordinate set of the traffic signal light to describe its spatial position: first obtain the relative positional relationship between each endpoint of the traffic signal light and the current vehicle, and combine the calibrated data obtained by the on-board camera For internal and external parameters, referring to the small hole imaging principle of the camera, the position of the traffic light can be projected from the three-dimensional space to the two-dimensional plane of the traffic signal image to obtain a set of projected coordinates. The coordinates of the center of the projected point are the mean value of the corresponding coordinates of each element in the projected coordinate set.

Step 103 , detecting and identifying the traffic lights in the region of interest ROI to obtain the status of the traffic lights that need to pass through the intersection at the current moment.

In this step, the vehicle-mounted camera can use image detection technology and image recognition technology to detect and identify the position and size of the traffic lights in the region of interest ROI, and obtain the status of the traffic lights that need to pass through the intersection at the current moment. Status, which can include the location and size of individual traffic lights.

Of course, in this embodiment, the images of the region of interest may also be sequentially input to the corresponding detection model and recognition model to perform detection and recognition of traffic lights respectively. Wherein, the traffic signal detection model and recognition model can be trained according to traditional machine learning algorithms, or can be obtained according to deep learning algorithms. The real-time example of the present invention does not limit the methods used in the training process of the series of models. In a specific example, the traffic signal detection model can be trained using centernet's deep learning algorithm, and the traffic signal recognition model can be trained using the error backpropagation network. Wherein, the traffic signal light detection model obtains the position and size of the traffic signal light on the image of the traffic signal light in the region of interest.

Step 104, updating the state of the historical tracking target at the current moment, wherein the historical tracking target is the detection and recognition result of the traffic signal light in a period of time before the current moment.

In this step, the position and size of each historical tracking target in the historical tracking target set are updated so that its state matches the current moment. Among them, the historical tracking target refers to the recognition result of the traffic signal light within a period of time before the current moment.

Step 105, determining a first matching result between the state of the traffic lights that need to pass through the intersection at the current moment and the state of the historical tracking target at the current moment.

Wherein, in this step, first determine the first association matrix (ie association score) of the state of the traffic lights that need to pass through the crossing at the current moment and the state of the historical tracking target; The first corresponding relationship between the traffic lights passing through the intersection and the historical tracking target; and according to the first corresponding relationship, it is determined that the historical tracking target is associated with the state of the traffic lights that need to pass through the intersection at the current moment; and using the current moment The state of the historical tracking target needs to be updated with the state of the traffic lights passing through the intersection.

That is to say, in the embodiment of the present invention, first construct the correlation matrix between the state of the traffic signal light (that is, the detection result) that needs to pass through the intersection at the current moment and the historical tracking target, and then use the optimal matching principle to determine the relationship matrix between the two. One-to-one correspondence (herein referred to as the first correspondence). Specifically, the correlation matrix in this embodiment is the correlation score between the detection result of the traffic light and the historical tracking target. The higher the correlation score, the greater the possibility that the two are the same object. On the contrary, the correlation The lower the score, the more likely the two are different objects.

Among them, the calculation method of the correlation score includes but is not limited to the following: one way is to calculate the distance between the detection result of the traffic light and the historical tracking target, and then use the optimal matching algorithm, such as the Hungarian algorithm, to obtain The correlation matrix is matched, and the corresponding relationship (ie, the first corresponding relationship) between the detection result of the current traffic signal and the historical tracking target is output. It should be noted that due to various unfavorable factors such as the error of the vehicle positioning device, the deviation of the vehicle camera and camera parameters, and the change of the size and position of the traffic light in the captured image during the driving of the vehicle, there may be new traffic lights. When entering the image of the region of interest, or the historical tracking target fails to be detected and recognized at the current moment, or the historical tracking target leaves the image of the region of interest, etc., the matching results usually have three situations, that is, the current traffic signal detection result cannot be associated with the history Tracking target, the historical tracking target cannot be associated with the current traffic signal detection result, and the historical tracking target cannot be associated with the current traffic signal detection result. For ease of description, this embodiment takes the history tracking target not being associated with the current traffic signal detection result as an example for illustration, and the other two cases are detailed in the following embodiments.

In this step, the number of correlation matrices constructed first is the same as that of the target tracking algorithm. In this embodiment, the target tracking algorithm used is divided into method A and method B. It is necessary to set different priorities for the target tracking algorithm used, that is, when the traffic signal detection results are first associated with the historical tracking target, the priority The tracking result obtained by a higher-level tracking algorithm is matched with the detection result of the traffic light. If it fails to match, continue to track the target, use the tracking result obtained by the tracking algorithm with lower priority, and then continue to compare the tracking result obtained by using the lower tracking algorithm with the light detection result of the traffic signal that has not yet been matched. match. And so on, until the tracking results using all tracking algorithms are matched.

Step 106, if the first matching result is that the historical tracking target matches the state of a traffic signal that needs to pass through the intersection at the current moment, then acquire the historical display attribute information of the historical tracking target corresponding to the traffic signal that is matched at the current moment , where the matched traffic signal is the target traffic signal.

Wherein, the historical display attribute information includes the display attribute information of the traffic signal light detection results associated with the tracking target in the previous n frames. Constructing the sequence of display attribute information can be arranged according to time, smoothing it in time, and obtaining the most voting attributes through voting fusion as the output state result of the tracking target at the current moment.

That is to say, if the historical tracking target is associated (or matched) with the current traffic signal detection result, use the historical target tracking algorithm to update the corresponding historical tracking target's motion state and Display attribute information.

Step 107: If the electronic map includes the position information of the target traffic light, determine a second matching result between the projected position of the target traffic light on the traffic signal image and the historical tracking target.

In this step, the vehicle-mounted terminal first judges whether the electronic map includes the position information of the target traffic light, and if so, creates the projection position of the target traffic light on the traffic signal image at the current moment and the historical tracking target information. The second association matrix (i.e. the association score), afterward, the second association matrix is matched to obtain the second corresponding relationship between the current moment target traffic signal projection and the historical tracking target; finally, according to the second corresponding relationship to determine the Match the projection position of the signal light of the above-mentioned target to the corresponding historical tracking target.

Specifically, the correlation matrix is the correlation score between the target traffic light projection and the historical tracking target. The higher the correlation score, the greater the possibility that the two are the same object. Conversely, the lower the correlation score, it means that the two are not The more likely it is the same object. The calculation method of the association score includes but is not limited to calculating the distance between the target traffic signal projection and the historical tracking target. Then use an optimal matching algorithm, such as the Hungarian algorithm, to match the obtained correlation matrix, output the corresponding relationship (ie, the second corresponding relationship) between the target traffic signal projection and the historical tracking target, and determine the target according to the second corresponding relationship Match the projection position of the traffic signal light to the corresponding historical tracking target.

Step 108 , according to the second matching result, obtain that the projected position of the target traffic signal is matched to the display property state of the corresponding historical tracking target, and the display property state includes the color state and shape state of the target traffic signal light.

In this step, if the second matching result is that the projection position of the target traffic signal matches the corresponding historical tracking target, then acquire the display attribute information that the projection position of the target traffic signal matches the corresponding historical tracking target, wherein , the display attribute information may include the color information and shape information of the traffic lights, the color information includes one of the following: red, yellow, green, and no display, etc., and the shape information includes at least one of the following: circle, left-turn arrow , right-turning arrow, straight-going arrow, etc. However, during specific implementation, the display attribute information of the traffic signal light provided by the embodiment of the present invention is not limited thereto.

Step 109, outputting display attribute information of the target traffic signal light.

Wherein, in this step, the selected target traffic signal light display attribute status is output as a basis for vehicle driving planning.

The indicator light state recognition method provided by the embodiment of the present invention projects the location information including the target traffic signal light obtained from the electronic map at the current moment into the vehicle-mounted image, effectively locating the location of the traffic signal light in the vehicle-mounted image The area of interest where the area of interest is located, by identifying the traffic lights in the area of interest, accurately select the traffic lights that need to be paid attention to when passing the current intersection, and then compare the detection results of the current frame with the traffic lights in the previous period The output results of the frame are fused to accurately identify and output the state of the traffic signal light, thereby improving the accuracy of the state recognition of the traffic signal light.

Optionally, in another embodiment, on the basis of the above embodiment, this embodiment may also include: if the position information of the traffic signal lights that the vehicle currently needs to pass through the intersection in the electronic map is not obtained, judging whether there is Historical traffic light tracking target; if it does not exist, then the default detection area is set as an area of interest; if it exists, then predict and update the state of the historical traffic light tracking target at the current moment; according to the vehicle's current state The location information determines the region of interest (ROI) of the historical traffic signal light tracking target on the traffic signal image, wherein the historical traffic signal light tracking target is used as a traffic signal that needs to pass through the intersection currently.

In this embodiment, if the vehicle-mounted terminal does not obtain the position information of the traffic lights that the vehicle currently needs to pass through the intersection in the electronic map, it needs to determine whether there is a historical traffic light tracking target. If it does not exist, set the default detection area as interested Area; if it exists, predict and update the position and size of the historical traffic light tracking target at the current moment according to the tracking algorithm, and then use the historical traffic signal tracking target on the captured traffic signal image (ie, the two-dimensional image taken by the vehicle) The center of the coordinates of is the center to obtain the region of interest. Specifically, the location information of the historical traffic signal light tracking target is projected onto the traffic signal image to obtain two-dimensional coordinate information, and then the corresponding region of interest is obtained centering on the projected point.

Optionally, in another embodiment, on the basis of the above embodiment, this embodiment may also include: the first case, if the first matching result is that the status of the traffic lights that need to pass the intersection at the current moment is not If the above-mentioned historical tracking target can be matched, a new tracking target is created, and the new tracking target is added to the set of historical tracking targets.

That is to say, if the current traffic light detection result fails to associate with the historical tracking target, it means that a new traffic light has entered the area of interest, and then a new tracking target entity is generated for the detection result and added to the historical tracking target set.

In the embodiment of the present invention, if the first matching result is that the state of the traffic signal lights that need to pass through the intersection at the current moment fails to match the historical tracking target, a new tracking target is created to improve the recognition efficiency of the traffic signal light state .

Second, if the first matching result is that the historical tracking target fails to match the state of the traffic lights that need to pass through the intersection at the current moment, determine whether to remove the historical tracking target; if so, remove the historical tracking target Track the target; if not, update the state of the historical tracking target, and the state records information about whether the historical tracking target is detected in the current frame.

That is to say, if the historical tracking target cannot be associated with the current traffic light detection result, it means that the historical tracking target has failed to be detected and recognized at the current moment or the historical tracking target has left the area of interest. In order to ensure the stability of the tracking result , it is necessary to further determine whether to delete the tracking target, and the criteria for the determination may be based on the fact that the historical tracking target has not been detected in the previous m frames as an example. If the deletion criterion is met, the tracking target is deleted from the historical tracking target collection; otherwise, the status of the historical tracking target is updated, and the status includes information indicating whether the tracking target is detected in the current frame.

In the embodiment of the present invention, if the historical tracking target cannot be associated with the current traffic signal detection result, then according to the judgment standard, the unassociated historical tracking target is adaptively deleted, which not only saves storage space, but also increases the number of traceable results. Stability, thereby improving the recognition efficiency of the state of the traffic signal.

Optionally, in another embodiment, on the basis of the above embodiment, this embodiment may further include: if the position information of the target traffic light in the electronic map is not obtained, obtaining the vehicle The planned behavior at the current intersection; according to the planned behavior of the vehicle, the vehicle is matched to the historical tracking target of the corresponding state; the historical display attribute status of the historical tracking target at the current moment is acquired; the historical display of the historical tracking target is output property status.

In this embodiment, if the loaded electronic map does not contain the position information of the target traffic lights that need to be paid attention to at the current crossing, the planned behavior of the vehicle at the current crossing, such as turning left, turning right or going straight, etc., is obtained, and then , match the tracking target of the corresponding output state according to the planned behavior of the vehicle, and obtain the output state result of the tracking target at the current moment. In a specific example, if the vehicle is going to turn left at the intersection, the shape information of the output state of the matching tracking target should be a left-turning arrow, and the matching rule is consistent with the traffic passing rule.

In the embodiment of the present invention, when the loaded electronic map does not contain the position information of the target traffic light that needs attention at the current crossing, by acquiring the planned behavior of the vehicle, the output state of the tracking target is matched according to the planned behavior of the vehicle, Planning for the vehicle's driving planning behavior improves the stability and comfort of the vehicle.

Please also refer to FIG. 2 , which is an application flowchart of a method for identifying the state of an indicator light provided by an embodiment of the present invention. In this embodiment, taking the detection and identification of the state of a traffic signal light as an example, this embodiment includes: Step 201, obtaining The traffic signal image captured by the vehicle-mounted camera at the current moment, the traffic signal image includes traffic signal lights.

In this step, when a shooting instruction is detected, the shooting of the vehicle-mounted camera can be started. The vehicle-mounted camera can be integrated on the recorder, or independently deployed on the vehicle, which is not limited in this embodiment.

Step 202, acquiring the location information of the vehicle at the current moment.

In this step, the positioning information of the current vehicle can be located through the satellite navigation system, and the specific positioning process through the satellite positioning and navigation system is a well-known technology for those skilled in the art, and will not be repeated here.

It should be noted that, in a specific embodiment, step 201 and step 202 can be separated in no order, and can also be executed at the same time, which is not limited in this embodiment.

Step 203 , judging whether the loaded electronic map includes the location information of traffic lights that need to pass through the intersection at present, if yes, execute step 204 , otherwise, execute step 206 .

Step 204 , projecting the position information of the traffic signal light onto the traffic signal image to obtain two-dimensional coordinate information on the traffic signal image; and performing step 205 .

In this step, it is necessary to obtain the distance between the position of the traffic light at the current moment and the position of the vehicle, and map the position of the traffic light to the image, so as to obtain the projection result of the traffic light on the image. In order to realize the mapping of traffic lights on the traffic signal image, it is first necessary to clarify the relative positional relationship between the traffic lights and the current vehicle. The embodiment of the present invention uses the endpoint coordinate set of the traffic signal light to describe its spatial position. After obtaining the relative positional relationship between each end point of the traffic signal light and the current vehicle, combined with the internal and external parameters that have been calibrated by the vehicle-mounted camera, and referring to the small hole imaging principle of the camera, the traffic signal light can be projected from the three-dimensional space to the two-dimensional plane of the image to get the set of projected coordinates. The coordinates of the center of the projected point are the mean value of the corresponding coordinate sum of each element in the projected coordinate set.

Step 205 , centering on the projected projection point, obtain the region of interest ROI on the traffic signal image of the traffic lights that currently need to pass through the intersection; after that, step 210 is executed.

Step 206 , judging whether there is a historical traffic signal light tracking target on the electronic map, if yes, execute step 207 , if not, execute step 209 .

Wherein, the historical tracking target refers to the recognition result of the traffic signal light in a period of time before the current moment.

Step 207, predicting and updating the two-dimensional coordinates and size of the historical traffic signal light tracking target at the current moment.

In this step, the prediction can be made according to the tracking algorithm. For example, considering the running speed and tracking accuracy, the historical target tracking algorithm in step 207 includes but not limited to Kalman filter algorithm and kernel correlation filter algorithm. The Kalman filter algorithm is to calculate the motion state of the tracking target. The motion state includes but not limited to the size, size and change rate of the position coordinates. According to the difference between the current moment and the state of the historical tracking target, the tracking target motion state is obtained , so as to predict the state of the historical tracking target at the current moment. It is worth noting that, for the sake of illustration, this implementation example uses a simple motion model at a constant speed. Therefore, the motion state includes the position and speed information of the tracking target as an example, but it is not limited to this in practical applications. It should be noted that the present invention can also use other motion state information, such as acceleration, angular velocity, etc., so as to be applicable to other more complex motion models. The kernel correlation filter algorithm is used to calculate the historical image features of the historical tracking target. The historical image features include but not limited to: grayscale features, directional gradient features, histogram features, etc., search on the traffic signal image at the current moment, and get The area most similar to the features of the historical image is the predicted tracking result of the tracking target at the current moment. The central coordinate of the coordinates of the historical traffic signal light tracking target on the two-dimensional image is the mean value of the corresponding coordinate sum of each element in the tracking result set.

Step 208 , taking the coordinates of the historical traffic signal light tracking target on the traffic signal image as the center to obtain a region of interest ROI; after that, step 210 is executed.

Step 209, setting the default detection region as the region of interest.

The position of the set default detection area can be set according to empirical parameters, that is, the position of the default detection area can be obtained by hypothetical projection based on the estimation of the relative position of the vehicle camera and the traffic signal light, and the internal and external parameters of the vehicle camera.

Step 210, detect the traffic lights in the image of the region of interest ROI, and obtain the detection results of the traffic lights that need to pass through the intersection on the traffic signal image; after that, execute step 212.

Wherein, the detection result may include position and size and display attribute information.

Wherein, the step 210 uses the image of the region of interest as an input to detect and recognize traffic lights. The traffic signal light detection model and recognition model can be trained according to traditional machine learning algorithms, and can also be obtained according to deep learning algorithms. The present invention does not limit the methods used in the training process of the series of models. In a specific example, the traffic signal detection model can be trained using centernet's deep learning algorithm, and the traffic signal recognition model can be trained using the error backpropagation network. The traffic signal light detection model obtains the position and size of the traffic signal light on the image of the region of interest. The traffic signal recognition model obtains the display attribute information of the traffic signal according to the display image of the traffic signal on the region of interest. In a specific example, the display attribute information may include the color information and shape information of traffic lights, where the color information may include red, yellow, green, and no display, etc., and the shape information may include circle, left-turn arrow, and right-turn arrow and straight arrows etc. The present invention does not limit the content contained in the display attribute information of the traffic signal light.

Step 211 , updating the position and size of the historical tracking target at the current moment, so that the position and size of the historical tracking target match the detection results of the traffic lights that need to pass through the intersection currently on the traffic signal light image.

Step 212 , constructing a correlation matrix between the detection results of the traffic lights that currently need to pass through the intersection on the traffic signal image and the position and size of the historical tracking target at the current moment.

Wherein, the association matrix may also be referred to as a first management matrix.

In the step, first construct the correlation matrix between the current traffic light detection results and the historical tracking targets. Among them, the correlation matrix is the correlation score between the traffic light detection result and the historical tracking target. The higher the correlation score, the greater the possibility that the two are the same object. On the contrary, it represents the possibility that the two are the same object. smaller. The calculation method of the association score includes but not limited to the method of calculating the distance between the detection result of the traffic light and the historical tracking target.

Step 213, correlating the corresponding relationship between the position and size of the current signal light and the position and size of the historical tracking target according to the constructed correlation matrix, if the correlation fails, perform step 214, and if the correlation is successful, perform step 220; this step In , the optimal matching algorithm is used to determine the one-to-one correspondence between the two. Among them, the optimal matching algorithm, such as the Hungarian algorithm, matches the obtained correlation matrix, and outputs the corresponding relationship between the current traffic signal detection result and the historical tracking target. Due to various unfavorable factors such as the error of the vehicle positioning device, the deviation of the camera parameters of the on-board camera, and the change of the size and position of the traffic light in the captured image during the driving of the vehicle, there may be new traffic lights entering the image of the region of interest, If the historical tracking target fails to be detected and recognized at the current moment or the historical tracking target leaves the image of the region of interest, etc., the matching result of the optimal matching algorithm in step 213 may appear in two situations, one is successful association, and the other is Association failed.

In a specific example, the number of association matrices constructed in step 213 is the same as the number of target tracking algorithms. If there are two target tracking algorithms used and recorded as method A and method B respectively, it is necessary to set different priorities for the target tracking algorithms used first, that is, when associating the detection results of traffic lights with historical tracking targets, first use The tracking result obtained by the tracking algorithm with a higher priority matches the detection result of the traffic signal light. If it fails to match, the tracking algorithm with a higher priority is used (the tracking algorithm has a lower priority than the tracking algorithm with a higher priority above). ) is matched with the unmatched traffic signal detection results. And so on, until the tracking results of all tracking algorithms are used.

Step 214: Judging whether the association of the current signal lamp or the historical tracking target fails, if the association of the current signal lamp fails, perform step 215, and if the association of the historical tracking target fails, perform step 217.

Step 215, creating a new tracking target entity.

In this step, if the current traffic light detection result fails to associate with the historical tracking target, it means that a new traffic light has entered the image of the region of interest, and a new tracking target entity is generated for the detection result.

Step 216 , adding the new tracking target to the historical tracking target set, and then performing step 221 .

Step 217, determine whether to delete the historical tracking target, if not, execute step 219, if yes, execute step 218.

In this step, if the historical tracking target fails to be associated with the current traffic light detection result, it means that the historical tracking target has not been successfully detected and recognized at the current moment or the historical tracking target has left the image of the region of interest. In order to ensure the accuracy of the tracking result For stability, it is necessary to determine whether to delete the tracking target according to the judgment standard. The judgment standard can be based on the proportion of the tracking target that was not detected in the previous m frames as an example, but in actual application, it is not limited to this. If the criterion for deletion is met, execute step 218; otherwise, execute step 219; step 218, delete the historical tracking target, and end the current process.

Step 219, update the state of the historical tracking target, and then execute step 221.

Wherein, the updated state may include information marking whether the tracking target is observed in the current frame.

Step 220, updating the motion state and display attribute information of the corresponding historical tracking target.

In this step, when the two are associated, the motion state and display attribute information of the corresponding historical tracking target is updated according to the historical target tracking algorithm in the above step using the traffic signal detection result matched at the current moment.

Step 221, acquire the state of the existing historical tracking target, including its historical display attribute, that is, the display attribute information of the traffic signal detection results associated with the tracking target in the previous n frames.

Among them, in this step, the display attributes can be constructed according to the sequence arranged before and after time, time smoothed, and the most voted attributes can be obtained through voting fusion as the output state result of the tracking target at the current moment.

Step 222 , judging whether the loaded map contains the location information of the target traffic lights that need to be paid attention to at the current crossing; if so, go to step 223 ; otherwise, go to step 228 .

Step 223, constructing a correlation matrix between the projected position of the target traffic light on the vehicle camera image and the existing historical tracking target.

Step 224: Use the optimal matching algorithm to correlate the correspondence between the two according to the correlation matrix. If the correlation is successful, go to step 225; if the correlation fails, go to step 227.

Specifically, according to the correlation matrix, the optimal matching algorithm is used to correlate the one-to-one correspondence between the two. If successful (that is, the association is successful), the corresponding relationship between the target traffic light projection and the historical tracking target is output. If the association between the target traffic signal light projection and the existing tracking target fails, step 227 is executed.

Step 225, acquiring the display attribute status of the target traffic signal light projection matched to the corresponding historical tracking target.

Wherein, the display attribute state may include the color state and shape state of the target traffic signal light.

Step 226 , outputting the display attribute status of the target traffic signal light, and ending the current process.

Step 227, there is no traffic signal output at the current moment, and the current process ends.

Step 228, acquire the planned behavior of the vehicle at the current intersection.

Among them, planned behaviors, such as turning left, turning right, or going straight.

Step 229 , matching the historical tracking target of the corresponding output state according to the planned behavior of the vehicle, and obtaining the historical display attribute status of the historical tracking target at the current moment.

In a specific example, if the vehicle is about to turn left at the intersection, the shape information of the output state of the matched historical tracking target should be a left-turn arrow. Matched rules match the traffic rules.

Step 230, outputting the acquired historical reality attribute status of the historical tracking target, and ending the current process.

In this step, the historical reality attribute state of the historical tracking target can be used as the basis for vehicle driving planning.

It should be noted that, for the method embodiment, for the sake of simple description, it is expressed as a series of action combinations, but those skilled in the art should know that this implementation disclosure is not limited by the described action sequence, because according to In the present invention, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification belong to preferred embodiments, and the actions involved are not necessarily required by the present invention.

Fig. 3 is a block diagram of a device for identifying the state of an indicator light according to an exemplary embodiment. Referring to Fig. 3, the device comprises a first acquisition module 301, a second acquisition module 302, a first determination module 303, a detection identification module 304, an update module 305, a second determination module 306, a third acquisition module 307, and a fourth acquisition module 308 , the third determination module 309 , the fifth acquisition module 310 and the output module 311 . Wherein, the first obtaining module 301 is used to obtain the traffic signal image taken in front of the vehicle at the current moment, and the location information of the vehicle at the current moment; the second obtaining module 302 is used to obtain the information described in the electronic map. The position information of the traffic lights that the vehicle currently needs to pass through the intersection; the first determining module 303 is used to determine the position information of the traffic lights that the vehicle currently needs to pass through the intersection when the second acquisition module 302 obtains the position information of the traffic lights that the vehicle currently needs to pass through the intersection in the electronic map. The location information of the vehicle at the current moment acquired by the first acquisition module 301 determines the ROI of the location information of the traffic lights that the vehicle needs to pass through the intersection on the traffic signal image; the detection and identification module 304, It is used to detect and identify the traffic lights in the region of interest ROI to obtain the state of the traffic lights that need to pass through the intersection at the current moment; the update module 305 is used to update the state of the historical tracking target at the current moment, wherein , the historical tracking target is the detection and recognition result of traffic lights in a period of time before the current moment; the second determination module 306 is used to determine the state of the traffic lights that need to pass through the intersection at the current moment and the historical tracking target at the current moment The first matching result of the state; the third obtaining module 307 is used to determine the first matching result determined by the second determining module 306 as the historical tracking target matching the traffic lights that need to pass through the intersection at the current moment state, obtain the historical display attribute information of the historical tracking target corresponding to the traffic signal light matched at the current moment, wherein the traffic signal light matched is the target traffic signal light; the fourth obtaining module 308 is used to obtain the The electronic map includes the position information of the target traffic light at the current moment; the third determining module 309 is configured to obtain the position information of the target traffic light in the electronic map when the fourth obtaining module 308 acquires , determine the second matching result between the projected position of the target traffic signal light on the traffic signal light image and the historical tracking target; the fifth obtaining module 310 is configured to obtain the target traffic light according to the second matching result The projection position of the signal light is matched to the display attribute state of the corresponding historical tracking target, and the display attribute state includes the color state and shape state of the traffic signal light; the output module 311 is used to output the display attribute state of the target traffic signal light .

Optionally, in another embodiment, on the basis of the above embodiment, the first determination module 303 includes: a projection module 401 and an area acquisition module 402, the structural diagram of which is shown in FIG. 4 , wherein , the projection module 401 is configured to project the position information of the traffic lights that the vehicle needs to pass through the intersection to the traffic signal image according to the positioning information of the vehicle at the current moment, and obtain the position information of the traffic lights that need to pass the intersection currently. The two-dimensional coordinate information on the traffic signal image; the region acquisition module 402 is configured to obtain the region of interest ROI on the traffic signal image of the traffic signal lights that currently need to pass through the intersection centering on the projected projection point.

Optionally, in another embodiment, which is based on the above-mentioned embodiment, the projection module 401 includes: a calculation module 501 and a mapping module 502, the structure diagram of which is shown in FIG. 5 , wherein the calculation The module 501 is used to calculate the distance between the position information of the traffic lights that need to pass the intersection and the location information of the vehicle; the mapping module 502 is used to map the traffic lights that need to pass the intersection according to the distance. The position information of the traffic signal is mapped to the traffic signal image, and the two-dimensional coordinate information projected on the traffic signal image by the position information of the traffic signal that needs to pass through the intersection is obtained.

Optionally, in another embodiment, on the basis of the above embodiment, the second determining module 306 includes: a first correlation matrix determining module 601, a first matching module 602, and a first judging module 603 And the first state update module 604, its structure schematic diagram is as shown in Figure 6, this first association matrix determination module 601, is used for determining the first association matrix of the state of the traffic signal light that needs to pass through the intersection at the current moment and the state of the historical tracking target The first matching module 602 is used to match the first correlation matrix to obtain the first correspondence between the traffic lights that need to pass through the intersection at the current moment and the historical tracking target; the first judgment module 603 is used to According to the first correspondence, it is judged whether the historical tracking target is associated with the state of the traffic lights that need to pass through the intersection at the current moment; When the state of the traffic signal light passing through the intersection needs to be passed at the current moment, the state of the corresponding historical tracking target is updated by using the state of the traffic signal light matched at the current moment.

Optionally, in another embodiment, on the basis of the above embodiment, the third determining module 309 includes: a second correlation matrix determining module 701, a second matching module 702, and a second judging module 703 and tracking target determination module 704, its structural diagram is as shown in Figure 7, wherein, the second correlation matrix determination module 701 is used to determine the projection position and historical tracking of the target traffic light on the traffic signal image at the current moment The second correlation matrix of the target; the second matching module 702 is used to match the second correlation matrix to obtain the second corresponding relationship between the target traffic signal light projection at the current moment and the historical tracking target; the second judgment module 703, It is used to determine whether the projected position of the target traffic signal lamp matches the corresponding historical tracking target according to the second correspondence relationship; The projected position of the target traffic light is matched to the corresponding historical tracking target.

Optionally, in another embodiment, on the basis of the above embodiment, the device further includes: a third judging module 801, a setting module 802, a prediction updating module 803 and a fourth determining module 804, which A schematic diagram of the structure is shown in FIG. 8 , wherein the third judging module 801 is used to judge whether there is a traffic signal light at an intersection when the first determining module 303 determines that the position information of the traffic signal light that the vehicle currently needs to pass through the intersection in the electronic map has not been obtained. Historical traffic signal light tracking target; the setting module 802, used for the judgment module to determine that there is no historical traffic signal light tracking target, the default detection area is set to the region of interest; the prediction update module 803 is used for When the third determination module 801 determines that there is a historical traffic signal light tracking target, predict and update the state of the historical traffic signal light tracking target at the current moment; the fourth determination module 804 is used to Determining a region of interest (ROI) of the historical traffic signal light tracking target on the traffic signal image, wherein the historical traffic signal light tracking target is used as a traffic signal that currently needs to pass through an intersection.

Optionally, in another embodiment, on the basis of the above embodiments, the device may further include: a creation module, and/or a fourth judgment module, a removal module and a second status update module , wherein, the creating module is used to create a new tracking when the first matching result determined by the second determining module is that the state of the traffic lights that need to pass through the intersection at the current moment fails to match the historical tracking target target, and add the new tracking target to the set of historical tracking targets; the fourth judging module is used to determine that the first matching result determined by the second determining module is that the historical tracking target fails to match When checking the state of the traffic lights that need to pass through the intersection at the current moment, judge whether to remove the historical tracking target; the removal module is used to remove the historical tracking target when the fourth judging module History tracking target; the second state update module is used to update the state of the history tracking target when the fourth judging module determines not to remove the history tracking target, and the state records the history in the current frame Tracks information about whether the target is detected or not.

Optionally, in another embodiment, on the basis of the above embodiments, the device may further include: a behavior acquisition module 901, a matching module 902 and a state acquisition module 903, the schematic diagram of which is shown in Figure 9 shown, wherein the behavior acquiring module 901 is configured to acquire the location information of the target traffic light in the electronic map when the second matching result determined by the third determining module 309 is not acquired. The planned behavior of the current intersection; the matching module 902 is used to match the vehicle to the historical tracking target of the corresponding state according to the planned behavior of the vehicle; the state acquisition module 903 is used to obtain the historical tracking target at the current moment Historical display attribute status; the output module 311 is also configured to output the historical display attribute status of the historical tracking target acquired by the status acquisition module 903 .

Regarding the device in the above embodiment, the specific way in which each module executes the operation has been described in detail in the embodiment of the method. For relevant parts, please refer to the part of the description of the method embodiment, and will not be elaborated here. illustrate.

An embodiment of the present invention also provides an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the above-mentioned instructions Light state identification method.

An embodiment of the present invention also provides a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can execute the above-mentioned method for identifying the state of the indicator light. . Alternatively, the computer-readable storage medium may be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk and optical data storage devices, etc.

An embodiment of the present invention also provides a computer program product, including a computer program or an instruction. When the computer program or instruction is executed by a processor, the above-mentioned method for identifying the state of the indicator light is implemented.

Optionally, an embodiment of the present invention also provides an electronic device, as shown in FIG. The mutual communication is completed through the communication bus 1004, wherein the memory 1003 is used to store computer programs; the processor 1001 is used to realize the above-mentioned status recognition of the indicator lights when executing the programs stored in the memory 1003 method.

The communication bus mentioned in the terminal above may be a Peripheral Component Interconnect (PCI for short) bus or an Extended Industry Standard Architecture (EISA for short) bus or the like. The communication bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

The communication interface is used for communication between the terminal and other devices.

The memory may include a random access memory (Random Access Memory, RAM for short), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far away from the aforementioned processor.

The above-mentioned processor can be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (Digital Signal Processing, referred to as DSP) , Application Specific Integrated Circuit (ASIC for short), Field Programmable Gate Array (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Fig. 11 is a block diagram of an apparatus 1100 for identifying an indicator light state according to an exemplary embodiment. For example, apparatus 1100 may be provided as a terminal. Referring to FIG. 11 , apparatus 1100 includes processing component 1122 , which further includes one or more processors, and a memory resource represented by memory 1132 for storing instructions executable by processing component 1122 , such as application programs. The application program stored in memory 1132 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1122 is configured to execute instructions to perform the above method.

Device 1100 may also include a power component 1126 configured to perform power management of device 1100 , a wired or wireless network interface 1150 configured to connect device 1100 to a network, and an input-output (I/O) interface 1158 . The device 1100 can operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

Other embodiments of the invention will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present invention, these modifications, uses or adaptations follow the general principles of the present invention and include common knowledge or conventional technical means in the technical field not disclosed in the present invention . The specification and examples are to be considered exemplary only, with a true scope and spirit of the invention being indicated by the following claims.

It should be understood that the present invention is not limited to the precise constructions which have been described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the invention is limited only by the appended claims.

Claims

A method for identifying the state of an indicator light, characterized in that it comprises:

Obtaining the traffic signal image in front of the vehicle captured at the current moment, and the positioning information of the vehicle at the current moment;

If the position information of the traffic signal lights that the vehicle currently needs to pass through the intersection in the electronic map is obtained, then determine the position information of the traffic signal lights that the vehicle currently needs to pass through the intersection according to the positioning information of the vehicle at the current moment in the traffic signal image ROI on ROI;

Detecting and identifying the traffic lights in the region of interest ROI to obtain the status of the traffic lights that need to pass through the intersection at the current moment;

Updating the state of the historical tracking target at the current moment, wherein the historical tracking target is the detection and recognition result of the traffic signal light within a period of time before the current moment;

Determining the first matching result between the state of the traffic lights that need to pass through the intersection at the current moment and the state of the historical tracking target at the current moment;

If the first matching result is that the historical tracking target matches the state of a traffic signal that needs to pass through the intersection at the current moment, then acquire the historical display attribute information of the historical tracking target corresponding to the traffic signal that is matched at the current moment, wherein, The matched traffic light is the target traffic light;

If the position information of the target traffic light in the electronic map is acquired at the current moment, then determine a second matching result between the projected position of the target traffic light on the traffic signal image and the historical tracking target;

According to the second matching result, the projected position of the target traffic signal light is matched to the display attribute state of the corresponding historical tracking target, and the display attribute state includes the color state and shape state of the target traffic signal light;

Outputting the display property state of the target traffic signal light.
The method for identifying the state of an indicator lamp according to claim 1, wherein the position information of the traffic signal lights that the vehicle needs to pass through the intersection is determined according to the vehicle positioning information at the current moment on the traffic signal image Region of interest ROI, including:

Projecting the position information of the traffic lights that the vehicle needs to pass through the intersection to the traffic signal image according to the positioning information of the vehicle at the current moment, and obtaining the two-dimensional image of the traffic lights that need to pass the intersection currently on the traffic signal image coordinate information;

A region of interest (ROI) on the traffic signal image of a traffic signal light that currently needs to pass through the intersection is obtained centering on the projected projection point.
The method for identifying the state of an indicator light according to claim 2, wherein the location information of the traffic signal lights that the vehicle needs to pass through the intersection at present is projected onto the traffic signal image according to the location information of the vehicle at the current moment , to obtain the two-dimensional coordinate information of the traffic lights that currently need to pass through the intersection on the traffic signal image, including:

Calculating the distance between the current position information of the traffic lights that need to pass through the intersection and the location information of the vehicle;

According to the distance, map the position information of the traffic signal lights that need to pass the intersection to the traffic signal light image, and obtain the two-dimensional coordinate information projected on the traffic signal light image by the position information of the traffic lights that need to pass the intersection currently.
The method for identifying the state of an indicator lamp according to claim 1, wherein the first matching result of determining the state of the traffic signal light that needs to pass through the intersection at the current moment and the state of the historical tracking target at the current moment comprises:

Determine the first correlation matrix between the state of the traffic lights that need to pass through the intersection and the state of the historical tracking target at the current moment;

The first association matrix is matched to obtain the first correspondence between the traffic lights that need to pass through the crossing at the current moment and the historical tracking target;

According to the first correspondence, it is determined that the historical tracking target is associated with the state of the traffic lights that need to pass through the intersection at the current moment;

The state of the corresponding historical tracking target is updated by using the state of the traffic signal light matched at the current moment.
The method for identifying the state of an indicator lamp according to claim 1, wherein said determining the second matching result between the projected position of the target traffic signal light on the traffic signal light image and the historical tracking target comprises :

Determining the second correlation matrix between the projected position of the target traffic signal light on the traffic signal light image at the current moment and the historical tracking target;

Matching the second correlation matrix to obtain the second corresponding relationship between the target traffic signal light projection at the current moment and the historical tracking target;

According to the second correspondence, it is determined that the projection position of the target traffic light matches the corresponding historical tracking target.
The method for identifying the status of an indicator light according to any one of claims 1 to 5, wherein the method further comprises:

If the position information of the traffic lights that the vehicle currently needs to pass through the intersection is not obtained in the electronic map, it is judged whether there is a historical traffic lights tracking target;

If it does not exist, setting the default detection area as the area of interest;

If it exists, predict and update the state of the historical traffic light tracking target at the current moment;

The region of interest ROI of the historical traffic signal light tracking target on the traffic signal image is determined according to the positioning information of the vehicle at the current moment, wherein the historical traffic signal light tracking target is used as a traffic signal that needs to pass through an intersection currently.
The method for identifying the status of an indicator light according to any one of claims 1 to 5, wherein the method further comprises:

If the first matching result is that the state of the traffic lights that need to pass through the intersection at the current moment fails to match the historical tracking target, then create a new tracking target, and add the new tracking target to the historical tracking target set ;

If the first matching result is that the historical tracking target fails to match the state of the traffic lights that need to pass through the intersection at the current moment, determine whether to remove the historical tracking target;

If so, remove the historical tracking target;

If not, update the state of the historical tracking target, where the state records information about whether the historical tracking target is detected in the current frame.
The method for identifying the status of an indicator light according to any one of claims 1 to 5, wherein the method further comprises:

If the position information including the target traffic light in the electronic map is not obtained, then obtain the planned behavior of the vehicle at the current intersection;

matching the vehicle to a historical tracking target of a corresponding state according to the planned behavior of the vehicle;

Obtain the historical display attribute status of the historical tracking target at the current moment;

Output the historical display attribute status of the historical tracking target.
A device for identifying the state of an indicator light, characterized in that it comprises:

The first acquisition module is used to acquire the traffic signal image taken in front of the vehicle at the current moment, and the positioning information of the vehicle at the current moment;

The second acquisition module is used to acquire the position information of the traffic lights that the vehicle currently needs to pass through the intersection in the electronic map;

The first determination module is configured to, when the second acquisition module acquires the location information of the traffic lights at the intersections that the vehicle currently needs to pass through in the electronic map, according to the position of the vehicle at the current moment acquired by the first acquisition module The information determines the ROI of the position information of the traffic lights that the vehicle currently needs to pass through the intersection on the traffic signal image;

The detection and identification module is used to detect and identify the traffic lights in the region of interest ROI to obtain the status of the traffic lights that need to pass through the intersection at the current moment;

An update module, configured to update the state of the historical tracking target at the current moment, wherein the historical tracking target is the detection and recognition result of the traffic signal light in a period of time before the current moment;

The second determination module is used to determine the first matching result between the state of the traffic lights that need to pass through the intersection at the current moment and the state of the historical tracking target at the current moment;

The third acquiring module is configured to acquire the matching traffic signal at the current moment when the first matching result determined by the second determining module is that the historical tracking target matches the state of the traffic signal light that needs to pass through the intersection at the current moment. The historical display attribute information of the historical tracking target corresponding to the traffic signal light, wherein the matched traffic signal light is the target traffic signal light;

A fourth acquiring module, configured to acquire the position information including the target traffic light at the current moment in the electronic map;

The third determining module is configured to determine the projected position of the target traffic signal on the image of the traffic signal and the location information of the target traffic signal when the fourth obtaining module obtains the position information of the target traffic signal in the electronic map. The second matching result of the above-mentioned historical tracking target;

The fifth acquisition module is used to acquire the display attribute state of matching the projection position of the target traffic signal light to the corresponding historical tracking target according to the second matching result, and the display attribute state includes the color state and shape of the traffic signal light state;

An output module, configured to output the display attribute state of the target traffic signal light acquired by the fifth acquisition module.
An electronic device, characterized in that it comprises:

processor;

memory for storing said processor-executable instructions;

Wherein, the processor is configured to execute the instructions, so as to realize the method for identifying the state of the indicator light according to any one of claims 1-8.
A computer-readable storage medium, characterized in that, when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the The method for identifying the status of the indicator light described above.
A computer program product, including computer programs or instructions, characterized in that, when the computer programs or instructions are executed by a processor, the method for identifying the status of an indicator light according to any one of claims 1 to 8 is implemented.