CN115249407B

CN115249407B - Indicator light state identification method and device, electronic equipment, storage medium and product

Info

Publication number: CN115249407B
Application number: CN202110587049.1A
Authority: CN
Inventors: 黄超; 陈熙; 姚为龙
Original assignee: Shanghai Xiantu Intelligent Technology Co Ltd
Current assignee: Shanghai Xiantu Intelligent Technology Co Ltd
Priority date: 2021-05-27
Filing date: 2021-05-27
Publication date: 2023-09-26
Anticipated expiration: 2041-05-27
Also published as: WO2022247299A1; CN115249407A

Abstract

The embodiment of the invention provides an indicator lamp state identification method, an indicator lamp state identification device, electronic equipment, a storage medium and a product.

Description

Indicator light state identification method and device, electronic equipment, storage medium and product

Technical Field

The present invention relates to the field of traffic technologies, and in particular, to a method and apparatus for identifying a status of an indicator light, an electronic device, a computer readable storage medium, and a computer program product.

Background

With the continuous increase of road complexity and the increasing of vehicles, traffic safety is a topic of current widespread attention, for example, how to determine the indication state of a certain intersection or a traffic signal lamp adjacent to the intersection on a road to ensure the traffic safety.

In the related art, a common determination mode of the state of a traffic signal lamp is that an image containing the traffic signal lamp is firstly obtained through a vehicle-mounted camera on a vehicle, then an interested region of the traffic signal lamp in the image, which appears on an electronic map, is positioned according to a vehicle positioning device, and finally the current state of the traffic signal lamp is detected and identified in the interested region, and a current identification result is output. However, in the related art, due to various uncertain factors such as errors of the vehicle positioning device, deviations of camera parameters of the vehicle-mounted camera, and changes of the size and the position of the traffic signal lamp in the photographed image during the running of the vehicle, the current state of the traffic signal lamp which needs to be focused when the vehicle passes through the current intersection cannot be accurately identified.

Therefore, how to accurately identify the state of the traffic signal lamp which needs to be concerned when the vehicle passes through the current intersection is a technical problem to be solved at present.

Disclosure of Invention

The invention provides an indicator lamp state identification method, an indicator lamp state identification device, electronic equipment, a computer readable storage medium and a computer program product, which at least solve the technical problem that the state of a traffic signal lamp which needs to be focused on at a current intersection cannot be accurately identified when a vehicle passes through the current intersection in the related technology. The technical scheme of the invention is as follows:

according to a first aspect of an embodiment of the present invention, there is provided an indicator light status recognition method, including:

acquiring a traffic signal image shot in front of a vehicle at the current moment and positioning information of the vehicle at the current moment;

if the position information of the traffic signal lamp of the vehicle which is required to pass through the intersection currently in the electronic map is obtained, determining a region of interest (ROI) of the position information of the traffic signal lamp of the vehicle which is required to pass through the intersection currently on the traffic signal image according to the positioning information of the vehicle at the current moment;

detecting and identifying the traffic signal lamp in the region of interest (ROI) to obtain the state of the traffic signal lamp which needs to pass through the intersection at the current moment;

updating the state of a history tracking target at the current moment, wherein the history tracking target is the detection and identification result of a traffic signal lamp in a period of time before the current moment;

Determining a first matching result of the state of a traffic signal lamp which needs to pass through an intersection at the current moment and the state of the history tracking target at the current moment;

if the first matching result is that the history tracking target matches the state of the traffic signal lamp which needs to pass through the intersection at the current moment, acquiring the history display attribute information of the history tracking target corresponding to the traffic signal lamp matched at the current moment, wherein the traffic signal lamp matched is the target traffic signal lamp;

if the current time in the electronic map comprises the position information of the target traffic signal lamp, determining a second matching result of the projection position of the target traffic signal lamp on the traffic signal lamp image and the historical tracking target;

acquiring a display attribute state of the history tracking target corresponding to the projection position of the target traffic signal lamp according to the second matching result, wherein the display attribute state comprises a color state and a shape state of the target traffic signal lamp;

and outputting the display attribute state of the target traffic signal lamp.

According to a second aspect of an embodiment of the present invention, there is provided an indicator light status recognition device, including:

The first acquisition module is used for acquiring traffic signal images shot in front of a vehicle at the current moment and positioning information of the vehicle at the current moment;

the second acquisition module is used for acquiring the position information of the traffic signal lamp of the vehicle which is required to pass through the intersection currently in the electronic map;

the first determining module is used for determining a region of interest (ROI) on the traffic signal image according to the positioning information of the vehicle at the current moment, which is acquired by the first acquiring module, when the second acquiring module acquires the position information of the traffic signal which is required to pass through the intersection currently by the vehicle in the electronic map;

the detection and identification module is used for detecting and identifying the traffic signal lamps in the region of interest (ROI) to obtain the states of the traffic signal lamps which need to pass through the intersection at the current moment;

the updating module is used for updating the state of the historical tracking target at the current moment, wherein the historical tracking target is the detection and identification result of the traffic signal lamp in a period of time before the current moment;

the second determining module is used for determining a first matching result of the state of the traffic signal lamp which needs to pass through the intersection at the current moment and the state of the history tracking target at the current moment;

A third obtaining module, configured to obtain, when the first matching result determined by the second determining module is that the history tracking target matches a state of a traffic signal lamp that needs to pass through an intersection at a current moment, history display attribute information of the history tracking target corresponding to the traffic signal lamp that is matched at the current moment, where the matched traffic signal lamp is a target traffic signal lamp;

a fourth obtaining module, configured to obtain location information including the target traffic signal lamp at the current time in the electronic map;

the third determining module is used for determining a second matching result of the projection position of the target traffic signal lamp on the traffic signal lamp image and the history tracking target when the fourth obtaining module obtains the position information of the target traffic signal lamp in the electronic map;

a fifth obtaining module, configured to obtain, according to the second matching result, that the projection position of the target traffic signal lamp matches to a display attribute state of the corresponding history tracking target, where the display attribute state includes a color state and a shape state of the traffic signal lamp;

and the output module is used for outputting the display attribute state of the target traffic signal lamp acquired by the fifth acquisition module.

According to a third aspect of an embodiment of the present invention, there is provided an electronic apparatus including:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the indicator light state recognition method as described above.

According to a fourth aspect of embodiments of the present invention, there is provided a computer-readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the indicator light state recognition method as described above.

According to a fifth aspect of embodiments of the present invention, there is provided a computer program product comprising a computer program or instructions which, when executed by a processor, implements an indicator light state recognition method as described above.

The technical scheme provided by the embodiment of the invention at least can comprise the following beneficial effects:

according to the indicating lamp state identification method provided by the embodiment of the invention, the position information comprising the target traffic signal lamp obtained from the electronic map at the current moment is projected into the vehicle-mounted shooting image, the region of interest of the traffic signal lamp in the vehicle-mounted shooting image is effectively positioned, the traffic signal lamp passing through the current intersection is accurately selected by identifying the traffic signal lamp of the region of interest, then the detection result of the current frame is fused with the output result of each frame of the traffic signal lamp before a period of time, and the state of the traffic signal lamp is accurately identified and output, so that the accuracy of identifying the state of the traffic signal lamp is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention and do not constitute a undue limitation on the invention.

Fig. 1 is a flowchart of an indicator light status recognition method according to an embodiment of the present invention.

Fig. 2 is a flowchart of an application example of an indicator light status recognition method according to an embodiment of the present invention.

Fig. 3 is a block diagram of an indicator light status recognition device according to an embodiment of the present invention.

Fig. 4 is a schematic structural diagram of a first determining module according to an embodiment of the present invention.

Fig. 5 is a schematic structural diagram of a projection module according to an embodiment of the present invention.

Fig. 6 is a schematic structural diagram of a second determining module according to an embodiment of the present invention.

Fig. 7 is a schematic structural diagram of a third determining module according to an embodiment of the present invention.

Fig. 8 is another block diagram of an indicator light status recognition device according to an embodiment of the present invention.

Fig. 9 is a further block diagram of an indicator light status recognition device according to an embodiment of the present invention.

Fig. 10 is a block diagram of an electronic device according to an embodiment of the present invention.

Fig. 11 is a block diagram of an apparatus with status recognition of indicator lights according to an embodiment of the present invention.

Detailed Description

In order to enable a person skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. The implementations described in the following exemplary examples do not represent all implementations consistent with the invention. Rather, they are merely examples of apparatus and methods consistent with aspects of the invention as detailed in the accompanying claims.

Fig. 1 is a flowchart illustrating a method for identifying a status of an indicator light according to an exemplary embodiment, where the method may be used in a terminal, and a terminal implementation device may be an electronic device such as a vehicle-mounted terminal, and the embodiment may include the following steps, as shown in fig. 1:

And 101, acquiring a traffic signal image shot in front of a vehicle at the current moment and positioning information of the vehicle at the current moment.

In this embodiment, when the vehicle-mounted terminal detects the shooting instruction, the vehicle-mounted camera is started to shoot, and the vehicle-mounted camera may be integrated on the recorder or may be independently deployed on the vehicle, which is not limited in this embodiment.

In this step, the vehicle-mounted terminal may be located to the positioning information of the current vehicle through the satellite navigation system, or may be located to the positioning information of the current vehicle in other manners, where the specific process of locating the vehicle-mounted terminal through the satellite navigation system is a well-known technology for those skilled in the art, and will not be described herein.

Step 102, if the position information of the traffic signal lamp of the vehicle which is required to pass through the intersection currently in the electronic map is obtained, determining a region of interest (ROI) of the position information of the traffic signal lamp of the vehicle which is required to pass through the intersection currently on the traffic signal image according to the positioning information of the vehicle at the current moment.

In the step, a vehicle-mounted terminal firstly judges whether the position information of a traffic signal lamp of the vehicle which is required to pass through an intersection currently in an electronic map is acquired, and if so, the position information of the traffic signal lamp of the vehicle which is required to pass through the intersection currently is projected onto a traffic signal image according to the positioning information of the vehicle at the current moment, so that two-dimensional coordinate information of the traffic signal lamp which is required to pass through the intersection currently on the traffic signal image is obtained; then, a region of interest (ROI, region of interest) of the traffic signal lamp currently required to pass through the intersection on the traffic signal image is acquired centering on the projected point of the projection. The process of acquiring the region of interest of the traffic signal lamp currently required to pass through the intersection on the traffic signal image by taking the projected point as the center is a well-known technology, and is not described herein again. In machine vision and image processing, the ROI outlines a region to be processed, called a region of interest, from a processed image in a manner of a square, a circle, an ellipse, an irregular polygon, and the like.

The projecting the position information of the traffic signal lamp of the vehicle which needs to pass through the intersection on the traffic signal image according to the positioning information of the vehicle at the current moment to obtain the two-dimensional coordinate information of the traffic signal lamp which needs to pass through the intersection on the traffic signal image, may include:

the distance between the position information of the traffic signal lamp which needs to pass through the intersection at present and the position of the positioning information of the vehicle; and mapping the traffic signal lamp which is required to pass through the intersection currently into the traffic signal lamp image according to the distance, and obtaining the two-dimensional coordinate information projected on the traffic signal lamp image by the traffic signal lamp which is required to pass through the intersection currently.

That is, in this embodiment, the distance between the position of the traffic light currently required to pass through the intersection and the position of the vehicle is required to be obtained first, and the position information (i.e., the three-dimensional image) of the traffic light currently required to pass through the intersection is mapped to the traffic light image recorded by the vehicle-mounted camera (the image is a two-dimensional image), so that the projection result of the traffic light currently required to pass through the intersection on the traffic light image is obtained, where the projection result is two-dimensional coordinate information. That is, in this embodiment, in order to realize the mapping of the position information of the traffic signal lamp that needs to pass through the intersection on the traffic signal lamp image, the relative position relationship between the traffic signal lamp that needs to pass through the intersection and the current vehicle needs to be clarified. The embodiment of the invention adopts a mode that the space position of the traffic signal lamp is described by adopting the endpoint coordinate set of the traffic signal lamp: after the relative position relation between each endpoint of the traffic signal lamp and the current vehicle is acquired, the position of the traffic signal lamp can be projected from a three-dimensional space to a two-dimensional plane of an traffic signal lamp image by referring to the small hole imaging principle of a camera by combining the internal and external parameters calibrated by the vehicle-mounted camera so as to acquire a projection coordinate set. The coordinates of the center of the projection point are the average value of the corresponding coordinates of each element in the projection coordinate set.

And 103, detecting and identifying the traffic signal lamp in the region of interest (ROI) to obtain the state of the traffic signal lamp which needs to pass through the intersection at the current moment.

In this step, the vehicle-mounted camera may use an image detection technique and an image recognition technique to detect and recognize the position and the size of the traffic signal lamp in the region of interest ROI, so as to obtain the state of the traffic signal lamp that needs to pass through the intersection at the current moment, where the state may include the position and the size of each traffic signal lamp.

Of course, in this embodiment, the image of the region of interest may be sequentially input to the corresponding detection model and the corresponding identification model, so as to detect and identify the traffic signal. The traffic signal lamp detection model and the recognition model can be obtained by training according to a traditional machine learning algorithm, and can also be obtained according to a deep learning algorithm. In a specific example, the traffic light detection model may be trained using a centrex deep learning algorithm, and the traffic light identification model may be trained using an error back propagation network. The traffic signal detection model obtains the position and the size of a traffic signal on a traffic signal image of a region of interest.

And 104, updating the state of the historical tracking target at the current moment, wherein the historical tracking target is the detection and identification result of the traffic signal lamp in a period of time before the current moment.

In the step, the position and the size of each history tracking target in the history tracking target set are updated to enable the state of each history tracking target to be matched with the current moment. The historical tracking target points to the traffic signal lamp identification result in a period of time before the current moment.

Step 105, determining a first matching result of the state of the traffic signal lamp which needs to pass through the intersection at the current moment and the state of the history tracking target at the current moment.

In the step, a first incidence matrix (namely an incidence score) of the state of a traffic signal lamp which needs to pass through an intersection at the current moment and the state of a historical tracking target is determined; matching the first incidence matrix to obtain a first corresponding relation between traffic signal lamps which need to pass through the intersection at the current moment and a historical tracking target; judging the state of a traffic signal lamp which needs to pass through an intersection at the current moment on the history tracking target association according to the first corresponding relation; and updating the state of the history tracking target by using the state of the traffic signal lamp which needs to pass through the intersection at the current moment.

That is, in the embodiment of the present invention, firstly, an association matrix between the state (i.e., the detection result) of the traffic signal lamp that needs to pass through the intersection at the current moment and the history tracking target is constructed, and then, the one-to-one correspondence (referred to herein as a first correspondence) between the two is determined by adopting an optimal matching principle. Specifically, the correlation matrix in this embodiment is a correlation score of the correlation between the detection result of the traffic signal lamp and the history tracking target, and the higher the correlation score is, the greater the possibility that the two are the same object, and conversely, the lower the correlation score is, the greater the possibility that the two are different objects is.

The calculation method of the association score includes, but is not limited to, the following: one way is to calculate the distance between the detection result of the traffic signal lamp and the history tracking target, then match the obtained correlation matrix by using an optimal matching algorithm, such as a hungarian algorithm, and the like, and output the corresponding relationship (i.e., the first corresponding relationship) between the detection result of the current traffic signal lamp and the history tracking target. It should be noted that, due to various adverse factors such as errors of the vehicle positioning device, deviations of parameters of the vehicle-mounted camera and the camera, and changes of the size and the position of the traffic signal lamp in the photographed image during the running process of the vehicle, there may be situations that a new traffic signal lamp enters the image of the region of interest, or that the history tracking target fails to be detected and identified at the current moment or leaves the image of the region of interest, etc., the matching result usually has three situations, that is, the current traffic signal lamp detection result fails to be associated with the history tracking target, the history tracking target fails to be associated with the current traffic signal lamp detection result. For convenience of description, this embodiment is described by taking an example that a history tracking target fails to be associated with a current traffic light detection result, and the other two cases are described in detail in the following embodiments.

In the step, the number of the correlation matrixes constructed first is the same as that of the target tracking algorithm. The adopted target tracking algorithm is divided into a method A and a method B, different priorities are required to be set for the adopted target tracking algorithm, namely when the traffic signal lamp detection result and the historical tracking target are associated, the tracking result obtained by adopting the tracking algorithm with higher priority is matched with the traffic signal lamp detection result. If the traffic signal is not matched, continuing to track the target, adopting a tracking result obtained by a tracking algorithm with lower priority, and continuing to match the tracking result obtained by the tracking algorithm with the lamp detection result of the traffic signal which is not matched. And so on until the tracking results are matched using all tracking algorithms.

And 106, if the first matching result is that the history tracking target matches the state of the traffic signal lamp which needs to pass through the intersection at the current moment, acquiring the history display attribute information of the history tracking target corresponding to the traffic signal lamp matched at the current moment, wherein the traffic signal lamp matched is the target traffic signal lamp.

The historical display attribute information comprises display attribute information of traffic signal lamp detection results on the association of the first n frames and the tracking target. The construction display attribute information can be time-smoothed according to a sequence arranged before and after time, and the maximum voting attribute is obtained through voting fusion and is used as an output state result of the tracking target at the current moment.

That is, if the history tracking target is associated with (or matched with, etc.) the current traffic light detection result, the motion state and display attribute information of the corresponding history tracking target are updated for the current traffic light detection result matched at the moment by using the history target tracking algorithm.

And step 107, if the electronic map includes the position information of the target traffic signal lamp, determining a second matching result of the projection position of the target traffic signal lamp on the traffic signal lamp image and the history tracking target.

In the step, the vehicle-mounted terminal firstly judges whether the electronic map comprises the position information of the target traffic signal lamp, if so, a second incidence matrix (namely incidence score) of the projection position of the target traffic signal lamp on the traffic signal lamp image at the current moment and the history tracking target is created, and then the second incidence matrix is matched to obtain a second corresponding relation between the projection of the target traffic signal lamp at the current moment and the history tracking target; and finally, judging that the projection position of the target communication signal lamp is matched with a corresponding history tracking target according to the second corresponding relation.

Specifically, the correlation matrix is the correlation score between the projection of the target traffic signal and the historical tracking target, the higher the correlation score, the greater the probability that the target traffic signal projection and the historical tracking target represent the same object, and conversely, the lower the correlation score, the greater the probability that the target traffic signal projection and the historical tracking target represent the same object. The manner in which the association score is calculated includes, but is not limited to, calculating the distance between the projection of the traffic signal at the target and the historical tracking target. And then, matching the obtained incidence matrix by using an optimal matching algorithm, such as a Hungary algorithm, outputting a corresponding relation (namely a second corresponding relation) between the projection of the target traffic signal lamp and the history tracking target, and judging that the projection position of the target traffic signal lamp is matched with the corresponding history tracking target according to the second corresponding relation.

And step 108, acquiring a display attribute state of the history tracking target corresponding to the projection position of the target traffic signal lamp according to the second matching result, wherein the display attribute state comprises a color state and a shape state of the target traffic signal lamp.

In the step, if the second matching result is that the projection position of the target communication signal lamp is matched with the corresponding historical tracking target, display attribute information of the projection position of the target traffic signal lamp matched with the corresponding historical tracking target is obtained, wherein the display attribute information can comprise color information and shape information of the traffic signal lamp, and the color information comprises one of the following: red, yellow, green, no display, etc., the shape information includes at least one of: circular, left turn arrow, right turn arrow, straight arrow, etc. However, in a specific implementation, the embodiment of the present invention provides display attribute information of the traffic signal lamp is not limited thereto.

And step 109, outputting the display attribute information of the target traffic signal lamp.

In the step, the display attribute state of the selected target traffic signal lamp is output and used as the basis of vehicle driving planning.

Optionally, in another embodiment, on the basis of the foregoing embodiment, the method may further include:

if the position information of the traffic signal lamp of the vehicle which needs to pass through the intersection currently in the electronic map is not obtained, judging whether a historical traffic signal lamp tracking target exists or not; if not, setting the default detection area as a region of interest; if yes, predicting and updating the state of the historical traffic signal lamp tracking target at the current moment; and determining a region of interest (ROI) of the historical traffic signal lamp tracking target on the traffic signal image according to the positioning information of the vehicle at the current moment, wherein the historical traffic signal lamp tracking target is used as a traffic signal lamp which needs to pass through an intersection at present.

In this embodiment, if the vehicle-mounted terminal does not acquire the position information of the traffic signal lamp of the intersection, which is needed to be passed by the vehicle currently, in the electronic map, it is needed to determine whether there is a historical traffic signal lamp tracking target, and if there is no, a default detection area is set as the region of interest; and if the target exists, predicting and updating the position and the size of the historical traffic signal lamp tracking target at the current moment according to a tracking algorithm, and then acquiring the region of interest by taking the center of the coordinates of the historical traffic signal lamp tracking target on the shot traffic signal image (namely the vehicle-mounted shot two-dimensional image) as the center. The method comprises the following steps: and projecting the position information of the historical traffic signal lamp tracking target on the traffic signal image to acquire two-dimensional coordinate information, and then acquiring a corresponding region of interest by taking a projection point as a center.

in the first case, if the first matching result is that the state of the traffic signal lamp which needs to pass through the intersection at the current moment cannot match the historical tracking target, a new tracking target is created, and the new tracking target is added into the historical tracking target set.

That is, if the current traffic signal detection result fails to correlate with the historical tracking target, it is indicated that a new traffic signal enters the region of interest, and then a new tracking target entity is generated for the detection result and added to the historical tracking target set.

In the embodiment of the invention, if the first matching result is that the state of the traffic signal lamp which needs to pass through the intersection at the current moment cannot be matched with the historical tracking target, the recognition efficiency of the state of the traffic signal lamp is improved by creating a new tracking target.

Secondly, if the first matching result is that the history tracking target fails to match the state of the traffic signal lamp which needs to pass through the intersection at the current moment, judging whether to remove the history tracking target; if yes, removing the history tracking target; if not, updating the state of the history tracking target, wherein the state records the information of whether the history tracking target is detected in the current frame.

That is, if the history tracking target is not associated with the current traffic light detection result, it is indicated that the history tracking target is not successfully detected and identified at the current moment or the history tracking target leaves the region of interest, and in order to ensure the stability of the tracking result, it is necessary to further determine whether to delete the tracking target, and the criterion of the determination may be, for example, that the history tracking target is not detected in the previous m frames. If the tracking target meets the deleting standard, deleting the tracking target from the historical tracking target set; otherwise, the state of the historical tracking target is updated, the state including information that marks whether the tracking target was detected in the current frame.

In the embodiment of the invention, if the history tracking target is not associated with the current traffic signal lamp detection result, the history tracking target which is not associated is adaptively deleted according to the judgment standard, so that the storage space is saved, the stability of the trackable result is also improved, and the recognition efficiency of the state of the traffic signal is improved.

if the position information of the target traffic signal lamp is not acquired from the electronic map, acquiring the planning behavior of the vehicle at the current intersection; according to the planned behavior of the vehicle, the vehicle is matched with a historical tracking target in a corresponding state; acquiring a history display attribute state of the history tracking target at the current moment; and outputting the historical display attribute state of the historical tracking target.

In this embodiment, if the loaded electronic map does not include the position information of the target traffic signal lamp that needs to be focused on by the current intersection, the planned behavior of the vehicle at the current intersection, such as left turn, right turn or straight run, is obtained, and then the tracking target of the corresponding output state is matched according to the planned behavior of the vehicle, and the output state result of the tracking target at the current moment is obtained. In a specific example, if the vehicle turns left at the intersection, the shape information of the output state of the matched tracking target should be a left turn arrow, and the matching rule thereof conforms to the traffic rule.

In the embodiment of the invention, when the loaded electronic map does not contain the position information of the target traffic signal lamp which is required to be focused by the current crossing, the planning behavior of the vehicle is acquired, the output state of the tracking target is matched according to the planning behavior of the vehicle, and the planning of the running planning behavior of the vehicle is performed, so that the running stability and comfort of the vehicle are improved.

Referring to fig. 2, an application flowchart of an indicator light status recognition method according to an embodiment of the present invention is shown, where in this embodiment, detection and recognition of a traffic light status are taken as an example, and the embodiment includes:

step 201, acquiring a traffic signal image shot by a vehicle-mounted camera at the current moment, wherein the traffic signal image comprises a traffic signal lamp.

In this step, when a shooting instruction can be detected, shooting of the vehicle-mounted camera is started, and the vehicle-mounted camera can be integrated on the recorder or can be independently deployed on the vehicle, which is not limited in this embodiment.

Step 202, obtaining the positioning information of the vehicle at the current moment.

In this step, the positioning information of the current vehicle may be positioned by the satellite navigation system, and the specific positioning process of the satellite positioning navigation system is well known to those skilled in the art, and will not be described herein.

It should be noted that, in the embodiment, the steps 201 and 202 may be performed simultaneously without any sequence, which is not limited in this embodiment.

Step 203, determining whether the loaded electronic map includes the position information of the traffic signal lamp which needs to pass through the intersection currently, if yes, executing step 204, otherwise, executing step 206.

Step 204, projecting the position information of the traffic signal lamp onto the traffic signal image to obtain two-dimensional coordinate information on the traffic signal image; step 205 is performed.

In the step, the distance between the position of the traffic light at the current moment and the position of the vehicle is required to be obtained, and the position of the traffic light is mapped into an image, so that the projection result of the traffic light on the image is obtained. In order to realize the mapping of the traffic signal lamp on the traffic signal image, the relative position relation between the traffic signal lamp and the current vehicle needs to be defined first. The embodiment of the invention adopts the endpoint coordinate set of the traffic signal lamp to describe the space position. After the relative position relation between each endpoint of the traffic signal lamp and the current vehicle is obtained, the traffic signal lamp can be projected onto the two-dimensional plane of the image from the three-dimensional space by combining the internal and external parameters calibrated by the vehicle-mounted camera and referring to the small hole imaging principle of the camera so as to obtain a projection coordinate set. The coordinates of the center of the projection point are the average value of the corresponding coordinates of each element in the projection coordinate set.

Step 205, acquiring a region of interest ROI of a traffic signal lamp currently required to pass through an intersection on the traffic signal image by taking the projected point of projection as a center; thereafter, step 210 is performed.

Step 206, judging whether a historical traffic light tracking target exists on the electronic map, if so, executing step 207, and if not, executing step 209.

The historical tracking target refers to a traffic signal lamp identification result in a period of time before the current moment.

Step 207, predicting and updating the two-dimensional coordinates and the size of the historical traffic signal lamp tracking target at the current moment.

In this step, predictions may be made based on tracking algorithms, such as from the standpoint of operating speed and tracking accuracy, the historical target tracking algorithm in step 207 includes, but is not limited to, a Kalman filter algorithm, a kernel correlation filter algorithm, and the like. The Kalman filter algorithm is used for calculating the motion state of the tracking target, wherein the motion state comprises, but is not limited to, the change rate of the size, the dimension and the position coordinates, and the change value of the motion state of the tracking target is obtained according to the difference value between the current moment and the state moment of the history tracking target, so that the state of the history tracking target at the current moment is predicted. It should be noted that, for convenience of explanation, the present embodiment uses a simple constant velocity motion model, and thus, the motion state is exemplified by position and velocity information including the tracking target, but is not limited thereto in practical application. It should be noted that other motion state information, such as acceleration, angular velocity, etc., may also be used in the present invention, and thus may be applied to other more complex motion models. The kernel-dependent filtering algorithm is used to calculate historical image features of the historical tracking target, including but not limited to: and searching on the traffic signal image at the current moment to obtain a region most similar to the characteristic of the historical image, namely a predicted tracking result of the tracking target at the current moment. The central coordinate of the coordinates of the historical traffic signal lamp tracking target on the two-dimensional image is the mean value of the sum of the corresponding coordinates of all elements in the tracking result set.

Step 208, acquiring a region of interest ROI with the coordinates of the historical traffic signal lamp tracking target on the traffic signal image as the center; thereafter, step 210 is performed.

In step 209, the default detection region is set as the region of interest.

The position of the default detection area can be set according to experience parameters, namely, the position of the default detection area is obtained by carrying out hypothesis projection according to estimation of the relative positions of the vehicle-mounted camera and the traffic signal lamp and the internal and external parameters of the vehicle-mounted camera.

Step 210, detecting a traffic signal lamp in the image of the region of interest ROI to obtain a detection result of the traffic signal lamp currently required to pass through the intersection on the traffic signal lamp image; thereafter, step 212 is performed.

The detection result may include position and size and display attribute information, among others.

The step 210 takes an image of the region of interest as an input to detect and identify the traffic signal lamp. The traffic signal lamp detection model and the recognition model can be obtained through training according to a traditional machine learning algorithm, and can also be obtained through a deep learning algorithm. In a specific example, the traffic light detection model may be trained using a centrex deep learning algorithm, and the traffic light identification model may be trained using an error back propagation network. The traffic light detection model obtains the position and size of the traffic light on an image of the region of interest. The traffic signal lamp identification model obtains display attribute information of the traffic signal lamp according to the display image of the traffic signal lamp on the region of interest. In a specific example, the display attribute information may include color information and shape information of the traffic signal, where the color information may include red, yellow, green, no display, etc., and the shape information may include a circle, a left turn arrow, a right turn arrow, a straight arrow, etc. The invention does not limit the content contained in the display attribute information of the traffic signal lamp.

Step 211, updating the position and the size of the history tracking target at the current moment, so that the position and the size of the history tracking target are matched with the detection result of the traffic signal lamp which is required to pass through the intersection at present on the traffic signal lamp image.

And 212, constructing an incidence matrix of the detection result of the traffic signal lamp which is required to pass through the intersection on the traffic signal lamp image and the position and the size of the historical tracking target at the current moment.

Wherein the association matrix may also be referred to as a first management matrix.

In the step, firstly, an incidence matrix between the current traffic signal lamp detection result and the historical tracking target is constructed. The correlation matrix is the correlation score between the traffic signal lamp detection result and the historical tracking target, and the higher the correlation score is, the greater the probability that the traffic signal lamp detection result and the historical tracking target are the same object is, and the lower the probability that the traffic signal lamp detection result and the historical tracking target are the same object is. The way of calculating the association score includes, but is not limited to, the way of calculating the distance between the detection result of the traffic signal lamp and the history tracking target.

Step 213, according to the constructed correlation matrix, correlating the corresponding relation between the position and the size of the current signal lamp and the position and the size of the historical tracking target, if the correlation fails, executing step 214, and if the correlation succeeds, executing step 220;

In the step, an optimal matching algorithm is adopted to determine the one-to-one correspondence relationship between the two. The optimal matching algorithm, such as Hungary algorithm, matches the obtained incidence matrix and outputs the corresponding relation between the current traffic signal lamp detection result and the historical tracking target. Due to various adverse factors such as errors of the vehicle positioning device, deviation of camera parameters of the vehicle-mounted camera, changes of the size and the position of the traffic signal lamp in the shot image during running of the vehicle, and the like, situations such as that a new traffic signal lamp enters the region of interest image, that a history tracking target cannot be successfully detected and identified at the current moment or that the history tracking target leaves the region of interest image, and the like, can occur, and the matching result of adopting an optimal matching algorithm in step 213 can occur, wherein one situation is that the association is successful, and the other situation is that the association is failed.

In a specific example, the number of correlation matrices constructed in step 213 is the same as the number of target tracking algorithms employed. If two target tracking algorithms are adopted and are respectively marked as a method A and a method B, different priorities are required to be set for the adopted target tracking algorithm, namely when the traffic signal detection result and the historical tracking target are associated, the tracking result obtained by adopting the tracking algorithm with higher priority is matched with the traffic signal detection result, and if the tracking result is not matched, the tracking result obtained by adopting the tracking algorithm with higher priority (the tracking algorithm has lower priority than the tracking algorithm with higher priority) is matched with the traffic signal detection result which is not matched. And so on until the tracking results of all tracking algorithms are used.

Step 214: whether the current signal lamp association fails or the history tracking target association fails is judged, if the current signal lamp association fails, step 215 is executed, and if the history tracking target association fails, step 217 is executed.

Step 215, create a new tracking target entity.

In the step, if the current traffic signal lamp detection result fails to be related to the historical tracking target, a new traffic signal lamp entering the interested area image is indicated, and a new tracking target entity is generated for the detection result.

Step 216, adding the new tracking target to the historical tracking target set, and then executing step 221.

Step 217, it is determined whether the history tracking target is deleted, if not, step 219 is executed, and if yes, step 218 is executed.

In this step, if the history tracking target fails to be associated with the current traffic signal detection result, it is indicated that the history tracking target fails to be detected and identified at the current moment or the history tracking target leaves the image of the region of interest, and in order to ensure stability of the tracking result, it is required to determine whether to delete the tracking target according to a determination criterion, which may be, for example, a proportion of the tracking target that is not detected in the previous m frames, but is not limited to this in practical applications. If the deletion criterion is met, go to step 218; otherwise, step 219 is performed;

Step 218, deleting the history tracking target, and ending the current flow.

Step 219, the state of the history tracking target is updated, and step 221 is executed.

Wherein the updated state may include information that marks whether the tracking target was observed in the current frame.

Step 220, the motion state and display attribute information of the corresponding history tracking target are updated.

In the step, when the two are related, the motion state and the display attribute information of the corresponding historical tracking target are updated according to the historical target tracking algorithm in the step by utilizing the detection result of the traffic signal lamp matched at the current moment.

Step 221, the state of the existing history tracking target is obtained, including the history display attribute thereof, that is, the display attribute information of the traffic signal detection result on the correlation of the previous n frames and the tracking target.

In the step, display attributes can be constructed according to sequences arranged before and after time, time smoothing is performed on the display attributes, and the maximum voting attributes are obtained through voting fusion to serve as output state results of the tracking target at the current moment.

Step 222, judging whether the loaded map contains the position information of the target traffic signal lamp which needs to pay attention to the current crossing; if yes, go to step 223; otherwise, step 228 is performed.

Step 223, constructing an incidence matrix between the projection position of the target traffic light on the vehicle-mounted camera image and the existing history tracking target.

Step 224: and (3) associating the corresponding relation of the two by adopting an optimal matching algorithm according to the association matrix, if the association is successful, executing step 225, and if the association is failed, executing step 227.

Specifically, a one-to-one correspondence between the two is associated by adopting an optimal matching algorithm according to the association matrix, which is similar to the above process, wherein the optimal matching algorithm can be a hungarian algorithm, matches the obtained association matrix, and if the matching is successful (namely the association is successful), outputs the correspondence between the projection of the target traffic signal lamp and the history tracking target. If the target traffic light projection fails to correlate with an existing tracking target, step 227 is performed.

And 225, acquiring the display attribute state of the history tracking target, which is matched with the projection of the target traffic signal lamp.

Wherein the display attribute status may include a color status and a shape status of the target traffic signal.

And 226, outputting the display attribute state of the target traffic signal lamp, and ending the current flow.

Step 227, no traffic signal lamp is output at the current moment, and the current flow is ended.

Step 228, obtain the planned behavior of the vehicle at the current intersection.

In which behaviors such as left turn, right turn, straight run, etc. are planned.

Step 229, matching the history tracking target of the corresponding output state according to the vehicle planning behavior, and obtaining the history display attribute state of the current time of the history tracking target.

In a specific example, if the vehicle is turning left at the intersection, the shape information for the output state of the matched history tracking target should be a left-turn arrow. The matching rules are in accordance with traffic rules.

And 230, outputting the acquired historical reality attribute state of the historical tracking target, and ending the current flow.

In the step, the historical real attribute state of the historical tracking target can be used as the basis of vehicle driving planning.

It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the present disclosure is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required for the present invention.

Fig. 3 is a block diagram of an indicator light status recognition device, according to an exemplary embodiment. Referring to fig. 3, the apparatus includes a first acquisition module 301, a second acquisition module 302, a first determination module 303, a detection and identification module 304, an update module 305, a second determination module 306, a third acquisition module 307, a fourth acquisition module 308, a third determination module 309, a fifth acquisition module 310, and an output module 311. Wherein, the liquid crystal display device comprises a liquid crystal display device,

the first acquiring module 301 is configured to acquire a traffic signal image captured in front of a vehicle at a current time and positioning information of the vehicle at the current time;

the second obtaining module 302 is configured to obtain location information of a traffic signal lamp of the electronic map, where the vehicle currently needs to pass through an intersection;

the first determining module 303 is configured to determine, when the second obtaining module 302 obtains location information of a traffic signal lamp of the vehicle that is currently required to pass through an intersection in the electronic map, a region of interest ROI on the traffic signal image of the location information of the traffic signal lamp of the vehicle that is currently required to pass through the intersection according to the location information of the vehicle at the current time obtained by the first obtaining module 301;

The detection and identification module 304 is configured to detect and identify traffic signals in the ROI to obtain a state of a traffic signal that needs to pass through an intersection at a current moment;

the updating module 305 is configured to update a state of a history tracking target at the current time, where the history tracking target is a detection and identification result of a traffic signal lamp in a period of time before the current time;

the second determining module 306 is configured to determine a first matching result of a state of a traffic signal lamp that needs to pass through an intersection at a current moment and a state of the history tracking target at the current moment;

the third obtaining module 307 is configured to obtain, when the first matching result determined by the second determining module 306 is that the history tracking target matches a state of a traffic signal lamp that needs to pass through an intersection at a current time, historical display attribute information of the history tracking target corresponding to the traffic signal lamp that is matched at the current time, where the matched traffic signal lamp is a target traffic signal lamp;

the fourth obtaining module 308 is configured to obtain location information including the target traffic signal lamp at the current time in the electronic map;

The third determining module 309 is configured to determine, when the fourth obtaining module 308 obtains that the electronic map includes the position information of the target traffic signal lamp, a second matching result of the projection position of the target traffic signal lamp on the traffic signal lamp image and the history tracking target;

the fifth obtaining module 310 is configured to obtain, according to the second matching result, that the projection position of the target traffic signal lamp matches to a display attribute state of the corresponding history tracking target, where the display attribute state includes a color state and a shape state of the traffic signal lamp;

the output module 311 is configured to output a display attribute status of the target traffic signal lamp.

Optionally, in another embodiment, based on the foregoing embodiment, the first determining module 303 includes: a schematic structure of the projection module 401 and the region acquisition module 402 is shown in fig. 4, in which,

the projection module 401 is configured to project, according to positioning information of the vehicle at the current moment, position information of a traffic signal lamp of the vehicle that is currently required to pass through an intersection onto the traffic signal image, so as to obtain two-dimensional coordinate information of the traffic signal lamp that is currently required to pass through the intersection on the traffic signal image;

The region acquiring module 402 is configured to acquire a region of interest ROI of a traffic signal lamp currently required to pass through an intersection on the traffic signal image with the projected point as a center.

Optionally, in another embodiment, based on the above embodiment, the projection module 401 includes: a schematic diagram of the computing module 501 and the mapping module 502 is shown in fig. 5, in which,

the calculating module 501 is configured to calculate a distance between position information of a traffic signal lamp that needs to pass through an intersection at present and a position where positioning information of the vehicle is located;

the mapping module 502 is configured to map, according to the distance, position information of a traffic signal lamp that is currently required to pass through an intersection into the traffic signal lamp image, and obtain two-dimensional coordinate information projected on the traffic signal lamp image by the position information of the traffic signal lamp that is currently required to pass through the intersection.

Optionally, in another embodiment, based on the foregoing embodiment, the second determining module 306 includes: a first correlation matrix determining module 601, a first matching module 602, a first judging module 603 and a first status updating module 604, the structure of which is schematically shown in fig. 6,

The first correlation matrix determining module 601 is configured to determine a first correlation matrix of a state of a traffic signal lamp that needs to pass through an intersection at a current moment and a state of a history tracking target;

the first matching module 602 is configured to match the first correlation matrix to obtain a first correspondence between traffic signals that need to pass through an intersection at a current moment and a history tracking target;

the first judging module 603 is configured to judge, according to the first correspondence, whether the history tracking target is associated with a state of a traffic signal lamp that needs to pass through an intersection at a current moment;

the first state updating module 604 is configured to update, when the judging module judges that the state of the traffic signal lamp that needs to pass through the intersection at the current time is associated with the history tracking target, the state of the corresponding history tracking target by using the state of the traffic signal lamp that is matched at the current time.

Optionally, in another embodiment, based on the foregoing embodiment, the third determining module 309 includes: a second correlation matrix determining module 701, a second matching module 702, a second judging module 703 and a tracking target determining module 704, the structure of which is schematically shown in fig. 7, wherein,

The second incidence matrix determining module 701 is configured to determine a second incidence matrix of a history tracking target and a projection position of the target traffic signal lamp on the traffic signal lamp image at the current moment;

the second matching module 702 is configured to match the second correlation matrix to obtain a second corresponding relationship between the projection of the traffic signal lamp of the target at the current moment and the history tracking target;

the second judging module 703 is configured to judge whether the projection position of the target communication signal lamp matches a corresponding history tracking target according to the second correspondence;

the tracking target determining module 704 is configured to determine, at the second determining module, that the projection position of the target communication signal lamp matches the corresponding historical tracking target according to the second correspondence.

Optionally, in another embodiment, on the basis of the foregoing embodiment, the apparatus further includes: a third judging module 801, a setting module 802, a prediction updating module 803 and a fourth determining module 804, the structure of which is schematically shown in fig. 8, wherein,

the third judging module 801 is configured to judge whether a historical traffic signal lamp tracking target exists when the first determining module 303 determines that no position information of a traffic signal lamp of an intersection is acquired in the electronic map, where the traffic signal lamp is currently required to pass through the intersection;

The setting module 802 is configured to set the default detection area as an area of interest when the judging module determines that there is no history of tracking the historical traffic signal lamp;

the prediction updating module 803 is configured to predict and update a state of the historical traffic signal light tracking target at a current time when the third judging module 801 judges that the historical traffic signal light tracking target exists;

the fourth determining module 804 is configured to determine, according to the positioning information of the vehicle at the current moment, a region of interest ROI of the historical traffic light tracking target on the traffic signal image, where the historical traffic light tracking target is used as a traffic light that needs to pass through an intersection at present.

Optionally, in another embodiment, based on the foregoing embodiment, the apparatus may further include: a creation module, and/or a fourth judgment module, a removal module and a second status update module, wherein,

the creating module is used for creating a new tracking target and adding the new tracking target into a history tracking target set when the first matching result determined by the second determining module is that the state of the traffic signal lamp which needs to pass through the intersection at the current moment cannot be matched with the history tracking target;

The fourth judging module is used for judging whether to remove the history tracking target when the first matching result determined by the second determining module is that the history tracking target fails to match the state of the traffic signal lamp which needs to pass through the intersection at the current moment;

the removing module is used for judging that the history tracking target is removed in the fourth judging module and removing the history tracking target;

the second state updating module is configured to update a state of the history tracking target when the fourth judging module judges that the history tracking target is not removed, where the state records information about whether the history tracking target is detected in a current frame.

Optionally, in another embodiment, based on the foregoing embodiment, the apparatus may further include: the behavior acquisition module 901, the matching module 902 and the state acquisition module 903 are schematically shown in fig. 9, in which,

the behavior acquisition module 901 is configured to acquire a planned behavior of the vehicle at a current intersection when the second matching result determined by the third determination module 309 does not acquire the position information of the target traffic signal lamp included in the electronic map;

The matching module 902 is configured to match the vehicle to a history tracking target of a corresponding state according to a planned behavior of the vehicle;

the state obtaining module 903 is configured to obtain a history display attribute state of the history tracking target at a current time;

the output module 311 is further configured to output the history display attribute state of the history tracking target acquired by the state acquiring module 903.

The specific manner in which the operations of the respective modules are performed in the apparatus of the above embodiments has been described in detail in the embodiments related to the method, and the relevant points are only required to be referred to in the description of the embodiments related to the method, which will not be described in detail herein.

The embodiment of the invention also provides electronic equipment, which comprises:

a processor;

a memory for storing the processor-executable instructions;

Embodiments of the present invention also provide a computer-readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the indicator light state recognition method as described above. Alternatively, the computer readable storage medium may be a non-transitory computer readable storage medium, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

Embodiments of the present invention also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement the indicator light state recognition method as described above.

Optionally, an electronic device is further provided according to an embodiment of the present invention, as shown in fig. 10, including a processor 1001, a communication interface 1002, a memory 1003, and a communication bus 1004, where the processor 1001, the communication interface 1002, and the memory 1003 complete communication with each other through the communication bus 1004, where,

the memory 1003 is used for storing a computer program;

the processor 1001 is configured to implement the indicator light state recognition method as described above when executing the program stored in the memory 1003.

The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (Peripheral Component Interconnect, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.

The communication interface is used for communication between the terminal and other devices.

The memory may include random access memory (Random Access Memory, RAM) or non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.

The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU for short), a network processor (Network Processor, NP for short), etc.; but also digital signal processors (Digital Signal Processing, DSP for short), application specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), field-programmable gate arrays (Field-Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

Fig. 11 is a block diagram illustrating an apparatus 1100 for indicator light status recognition, according to an example embodiment. For example, the apparatus 1100 may be provided as a terminal. Referring to FIG. 11, apparatus 1100 includes a processing component 1122 that further includes one or more processors and memory resources, represented by memory 1132, for storing instructions, such as application programs, executable by processing component 1122. The application programs stored in memory 1132 may include one or more modules each corresponding to a set of instructions. Further, processing component 1122 is configured to execute instructions to perform the methods described above.

The apparatus 1100 may also include a power component 1126 configured to perform power management of the apparatus 1100, a wired or wireless network interface 1150 configured to connect the apparatus 1100 to a network, and an input-output (I/O) interface 1158. The device 1100 may operate based on an operating system stored in memory 932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.

Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims

1. An indicator light status recognition method, comprising:

If the electronic map comprises the position information of the target traffic signal lamp, determining a second matching result according to the distance between the projection position of the target traffic signal lamp on the traffic signal image and the historical tracking target;

and outputting the display attribute state of the target traffic signal lamp.

2. The method for identifying the status of the indicator light according to claim 1, wherein the determining the region of interest ROI on the traffic signal image of the position information of the traffic signal lamp which the vehicle currently needs to pass through the intersection according to the positioning information of the vehicle at the current moment comprises:

projecting the position information of the traffic signal lamp of the vehicle which is required to pass through the intersection on the traffic signal image according to the positioning information of the vehicle at the current moment to obtain the two-dimensional coordinate information of the traffic signal lamp which is required to pass through the intersection on the traffic signal image;

And acquiring a region of interest (ROI) of a traffic signal lamp which is required to pass through an intersection currently on the traffic signal image by taking the projected point of projection as the center.

3. The method for identifying the status of an indicator light according to claim 2, wherein projecting the position information of the traffic signal lamp of the vehicle that is currently required to pass through the intersection onto the traffic signal image according to the positioning information of the vehicle at the current moment, to obtain the two-dimensional coordinate information of the traffic signal lamp that is currently required to pass through the intersection on the traffic signal image, comprises:

calculating the distance between the position information of the traffic signal lamp which needs to pass through the intersection at present and the position of the positioning information of the vehicle;

and according to the distance, mapping the position information of the traffic signal lamp which is required to pass through the intersection currently into the traffic signal lamp image, and obtaining the two-dimensional coordinate information projected on the traffic signal lamp image by the position information of the traffic signal lamp which is required to pass through the intersection currently.

4. The method for identifying the status of the indicator light according to claim 1, wherein the determining the first matching result of the status of the traffic light which needs to pass through the intersection at the current time and the status of the history tracking target at the current time comprises:

Determining a first incidence matrix of the state of a traffic signal lamp which needs to pass through an intersection at the current moment and the state of a historical tracking target;

matching the first incidence matrix to obtain a first corresponding relation between traffic signal lamps which need to pass through the intersection at the current moment and a historical tracking target;

judging the state of a traffic signal lamp which needs to pass through an intersection at the current moment on the history tracking target association according to the first corresponding relation;

and updating the state of the corresponding historical tracking target by using the state of the traffic signal lamp matched with the current moment.

5. The method of claim 1, wherein determining a second match based on a distance between a projected location of the target traffic signal on the traffic signal image and a history tracking target comprises:

determining a second incidence matrix of the projection position of the target traffic signal lamp on the traffic signal lamp image at the current moment and a historical tracking target;

matching the second incidence matrix to obtain a second corresponding relation between the projection of the traffic signal lamp of the target at the current moment and the historical tracking target;

and judging that the projection position of the target communication signal lamp is matched with a corresponding history tracking target according to the second corresponding relation.

6. The indicator light status recognition method of any one of claims 1 to 5, further comprising:

if the position information of the traffic signal lamp of the vehicle which needs to pass through the intersection currently in the electronic map is not obtained, judging whether a historical traffic signal lamp tracking target exists or not;

if not, setting a default detection area as the region of interest;

if yes, predicting and updating the state of the historical traffic signal lamp tracking target at the current moment;

and determining a region of interest (ROI) of the historical traffic signal lamp tracking target on the traffic signal image according to the positioning information of the vehicle at the current moment, wherein the historical traffic signal lamp tracking target is used as a traffic signal lamp which needs to pass through an intersection at present.

7. The indicator light status recognition method of any one of claims 1 to 5, further comprising:

if the first matching result is that the state of the traffic signal lamp which needs to pass through the intersection at the current moment fails to match the historical tracking target, a new tracking target is created, and the new tracking target is added into a historical tracking target set;

If the first matching result is that the history tracking target fails to match the state of the traffic signal lamp which needs to pass through the intersection at the current moment, judging whether the history tracking target is removed;

if yes, removing the history tracking target;

if not, updating the state of the history tracking target, wherein the state records the information of whether the history tracking target is detected in the current frame.

8. The indicator light status recognition method of any one of claims 1 to 5, further comprising:

if the position information of the target traffic signal lamp is not acquired from the electronic map, acquiring the planning behavior of the vehicle at the current intersection;

according to the planned behavior of the vehicle, the vehicle is matched with a historical tracking target in a corresponding state;

acquiring a history display attribute state of the history tracking target at the current moment;

and outputting the historical display attribute state of the historical tracking target.

9. An indicator light condition recognition device, comprising:

the third determining module is used for determining a second matching result according to the distance between the projection position of the target traffic signal lamp on the traffic signal image and the historical tracking target when the fourth obtaining module obtains the position information of the target traffic signal lamp in the electronic map;

10. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the indicator light status recognition method of any one of claims 1 to 8.

11. A computer readable storage medium, characterized in that instructions in the computer readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the indicator light state recognition method of any one of claims 1 to 8.

12. A computer program product comprising a computer program or instructions which, when executed by a processor, implements the method of identifying the status of a light as claimed in any one of claims 1 to 8.