CN111310708A - Traffic signal lamp state identification method, device, equipment and storage medium - Google Patents
Traffic signal lamp state identification method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN111310708A CN111310708A CN202010130208.0A CN202010130208A CN111310708A CN 111310708 A CN111310708 A CN 111310708A CN 202010130208 A CN202010130208 A CN 202010130208A CN 111310708 A CN111310708 A CN 111310708A
- Authority
- CN
- China
- Prior art keywords
- traffic signal
- signal lamp
- vehicle
- target
- mapping
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
Abstract
The invention discloses a traffic signal lamp state identification method, a traffic signal lamp state identification device, traffic signal lamp state identification equipment and a storage medium. The method comprises the following steps: determining at least one alternative traffic signal lamp detection result according to an image acquired by a vehicle; determining a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps according to the map data; mapping the target traffic signal lamp to the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result; and determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp. By using the technical scheme of the embodiment of the invention, the state of the traffic signal lamp in front of the vehicle can be accurately identified.
Description
Technical Field
The embodiment of the invention relates to a data processing technology, in particular to a traffic signal lamp state identification method, a traffic signal lamp state identification device, traffic signal lamp state identification equipment and a storage medium.
Background
The traffic signal lamp state identification technology is widely applied to the fields of vehicle automatic driving, auxiliary driving, driving safety and the like.
The existing traffic signal lamp state identification method is mainly based on positioning and identification of an interested area of an image, and the method needs to position the interested area where a traffic signal lamp appears in a vehicle-mounted shot image and then position the traffic signal lamp in the interested area for identification.
In the process of implementing the invention, the inventor finds that the prior art has the following defects: due to factors such as deviation of vehicle positioning, variation of parameters of a vehicle-mounted camera, variation of size and position of a traffic signal lamp in an image during vehicle traveling, and the like, an error interest area is easily positioned. In addition, the method for identifying the state of the traffic signal lamp in the prior art cannot eliminate the interference of a non-traffic signal lamp or other traffic signal lamps, and cannot accurately identify the state of the traffic signal lamp under the condition of low resolution of a vehicle-mounted shot image.
Disclosure of Invention
The embodiment of the invention provides a traffic signal lamp state identification method, a traffic signal lamp state identification device, traffic signal lamp state identification equipment and a storage medium, so that the state of a traffic signal lamp in front of a vehicle can be accurately identified.
In a first aspect, an embodiment of the present invention provides a method for identifying a traffic signal lamp state, where the method includes:
determining at least one alternative traffic signal lamp detection result according to an image acquired by a vehicle;
determining a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps according to the map data;
mapping the target traffic signal lamp to the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result;
and determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp.
In a second aspect, an embodiment of the present invention further provides a traffic signal lamp status identification device, where the device includes:
the alternative traffic signal lamp detection result acquisition module is used for determining at least one alternative traffic signal lamp detection result according to the image acquired by the vehicle;
the target traffic signal lamp acquisition module is used for determining a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps according to the map data;
the traffic signal lamp mapping result acquisition module is used for mapping the target traffic signal lamp into the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result;
and the traffic signal lamp state determining module is used for determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp.
In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the traffic signal status identification method according to any one of the embodiments of the present invention.
In a fourth aspect, embodiments of the present invention further provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are used to perform the traffic signal light status identification method according to any one of the embodiments of the present invention.
According to the embodiment of the invention, after the detection result of the alternative traffic signal lamp is determined according to the image acquired by the vehicle, the target traffic signal lamp actually existing in front of the vehicle is mapped to the image acquired by the vehicle, and the detection result of the alternative traffic signal lamp in the image is subjected to auxiliary verification through the position of the target traffic signal lamp in the image, so that the problem of high recognition error rate when the state of the traffic signal lamp is directly recognized according to the image acquired by the vehicle in the prior art is solved, the interference of a non-traffic signal lamp or other traffic signal lamps can be effectively eliminated, and the state of the traffic signal can be accurately recognized under the condition of low resolution of the vehicle-mounted shot image. The method realizes accurate identification of the state of the traffic signal lamp in front of the vehicle, and further ensures the safety of the vehicle in the driving process.
Drawings
Fig. 1 is a flowchart of a traffic signal status recognition method according to a first embodiment of the present invention;
fig. 2a is a flowchart of a traffic signal status recognition method according to a second embodiment of the present invention;
FIG. 2b is a flow chart of a method of identifying the status of a traffic signal suitable for use with embodiments of the present invention;
FIG. 2c is a flowchart of a method of projecting traffic light locations in a three-dimensional laser point cloud onto a camera;
FIG. 2d is a flow chart of a method for similarity matching of traffic light detection results and traffic light projections in an image;
fig. 3 is a schematic structural diagram of a traffic signal lamp state identification device in a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a computer device in the fourth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a traffic light state identification method according to an embodiment of the present invention, where the embodiment is applicable to a situation where a vehicle accurately identifies a traffic light state in front of the vehicle during driving of the vehicle, and the method may be executed by a traffic light state identification device, which may be implemented by software and/or hardware, and is generally integrated in the vehicle and used in cooperation with an on-vehicle camera configured in front of the vehicle.
As shown in fig. 1, the technical solution of the embodiment of the present invention specifically includes the following steps:
and S110, determining at least one alternative traffic signal lamp detection result according to the image acquired by the vehicle.
The image acquired by the vehicle may be an image captured by a vehicle-mounted camera, and the candidate traffic signal light detection result may be a position, an outline, display attribute information, and the like of a traffic signal light detected from the image acquired by the vehicle. The display attribute information may include color information of the traffic signal, shape information of the traffic signal, and the like, and in a specific example, the color information may be red, yellow, green, and the like, and the shape information may be a circle, a left arrow, a right arrow, an upper arrow, and the like. The embodiment of the invention does not limit the specific content of the detection result of the alternative traffic signal lamp, and does not limit the specific content of the display attribute information.
In the embodiment of the present invention, at least one candidate traffic light detection result is determined according to an image acquired by a vehicle, and the embodiment does not limit the manner and the specific process for determining the candidate traffic light detection result according to the image acquired by the vehicle.
In an optional implementation manner of this embodiment, determining at least one alternative traffic signal detection result according to an image acquired by a vehicle may include: inputting an image acquired by a vehicle into a pre-trained traffic signal lamp identification model, and acquiring the detection result of the at least one alternative traffic signal lamp; and the detection result of the alternative traffic signal lamp comprises the position, the outline and the display attribute information of the alternative traffic signal lamp in the image.
The traffic signal lamp recognition model can be obtained by training according to a traditional machine learning algorithm or a deep learning algorithm, and the training process of the traffic signal lamp recognition model is not limited by the embodiment of the invention. In one specific example, the traffic light recognition model may be trained according to conventional machine learning algorithms such as logistic regression, support vector machine, and the like. In another specific example, the traffic light recognition model may be trained according to a deep learning algorithm such as a deep convolutional neural network. And inputting the images acquired by the vehicle into the traffic signal lamp identification model to obtain the detection result of the alternative traffic signal lamp. Optionally, the traffic signal lamp may be labeled in the image acquired by the vehicle, and then the alternative traffic signal lamp detection result is acquired through the traffic signal lamp identification model.
In another optional implementation manner of this embodiment, determining at least one alternative traffic signal detection result according to the image acquired by the vehicle may include:
acquiring a standard traffic signal lamp image set, wherein the standard traffic signal lamp image set comprises typical images of various different traffic signal lamps;
and searching and matching each standard traffic signal image included in the standard traffic signal image set in the images acquired by the vehicle, and if the target standard traffic signal image is matched with a first area in the images acquired by the vehicle, determining that the target standard traffic signal image is identified in the first area.
And S120, determining a target traffic light matched with the vehicle position information from the alternative traffic lights according to the map data.
The map data may be data in a road area where the vehicle is located, the vehicle position information may be related information of a position of the vehicle on the road where the vehicle is located, and the target traffic signal lamp may be a traffic signal lamp located in front of the vehicle and whose state needs to be determined.
In the embodiment of the present invention, a traffic light located in front of the vehicle is determined in the map as a target traffic light.
Alternatively, a positioning module configured on the vehicle, such as a GPS module, may acquire a real-time positioning result (i.e., vehicle position information) of the vehicle, and when it is determined that the image acquired by the vehicle identifies the alternative traffic light, identify the traffic light located in front of the vehicle in the map data according to the real-time positioning result.
In an alternative embodiment of the present invention, determining the target traffic signal matching the vehicle position information from the alternative traffic signals according to the map data may include: determining a search area according to the vehicle position information, and acquiring the geographical position information of at least one traffic signal lamp in the search area from map data; and screening out at least one traffic signal lamp of which the geographic position information and the vehicle position information meet the distance correlation condition as the target traffic signal lamp.
The search area may be an area within a certain position range of the vehicle, and the geographic position information of the traffic signal lamp may be information related to the position of the traffic signal lamp in a map. The distance association condition may be a condition satisfied by a distance between the vehicle position and the traffic light position, for example, a relative distance between the two is 5 meters or less, or 10 meters or less.
Alternatively, the distance correlation condition may be that the distance between the traffic light position and the vehicle position is the closest. The advantage of this arrangement is that the traffic signal light closest to the vehicle can be preferentially identified, providing convenience to the driver. In a specific example, a nearest neighbor search algorithm K-d Tree (K-dimensional Tree) may be used to obtain the traffic signal light closest to the vehicle position, and the embodiment does not limit the manner of obtaining the traffic signal light closest to the vehicle position.
In the embodiment of the invention, the traffic signal lamp meeting the distance correlation condition is screened out from the traffic signal lamps within a certain position range from the vehicle on the map to serve as the target traffic signal lamp.
S130, mapping the target traffic signal lamp to the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result.
The traffic signal lamp mapping result may be a result obtained by projecting the target traffic signal lamp in the image.
In the embodiment of the invention, the target traffic signal lamp is mapped into the image according to the distance between the position of the target traffic signal lamp and the position of the vehicle, and the traffic signal lamp mapping result of the target traffic signal lamp in the image is obtained. In order to map the target traffic signal lamp into the image, a relative position relationship between the target traffic signal lamp and the vehicle, that is, a relative distance between each point in the target traffic signal lamp and the vehicle, needs to be determined first, and after the relative position relationship is obtained, the target traffic signal lamp can be mapped into the image by combining internal and external parameters of the vehicle-mounted camera and an aperture imaging principle of the camera.
Optionally, a laser radar signal may be sent to a surrounding environment (mainly a front side) by a vehicle, point cloud data in front of the vehicle may be obtained based on an echo signal of the received laser radar signal, and the target traffic signal lamp is identified in the point cloud data, so that a relative position relationship between each point in the target traffic signal lamp and the vehicle may be obtained;
alternatively, the vehicle and the target traffic signal lamp may be mapped into a three-dimensional street view map according to vehicle position information and geographic position information of the target traffic signal lamp, and a relative distance between the target point on the vehicle and each point on the target traffic signal lamp may be calculated in the three-dimensional street view map as a relative position relationship between each point in the target traffic signal lamp and the vehicle.
In an optional embodiment of the present invention, mapping the target traffic signal into the image according to a relative position relationship between the target traffic signal and the vehicle to obtain a traffic signal mapping result may include: marking the target traffic signal lamp in point cloud data acquired by a vehicle in real time according to the geographical position information of the target traffic signal lamp; and mapping the marked target traffic signal lamp into the image according to the internal and external parameters of the vehicle-mounted camera of the vehicle to obtain a traffic signal lamp mapping result.
The point cloud data may be data recorded in the form of points including three-dimensional coordinates. The internal and external parameters of the vehicle-mounted camera may include internal parameters of the vehicle-mounted camera, the internal parameters may include a focal length of the camera, a pixel size and the like, the external parameters may include parameters related to a position of the camera in the map, and in a specific example, the external parameters may be a position, a rotation direction, an offset direction and the like of the camera in the map.
In the embodiment of the invention, the vehicle collects the point cloud data in real time, and marks the target traffic signal lamp in the point cloud data according to the geographical position information of the target traffic signal lamp. And mapping the marked target traffic signal lamp according to the internal parameters and the external parameters of the vehicle-mounted camera. The embodiment does not limit the manner of collecting the point cloud data.
And S140, determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp.
Wherein the traffic light state in front of the vehicle may be a state determined according to color information and control information of the traffic light in front of the vehicle. In a specific example, if the traffic light color information in front of the vehicle is green and the control information is straight, the traffic light state in front of the vehicle may be such that the traffic light closest to the vehicle is displayed as straight ahead.
In the embodiment of the invention, the state of the traffic signal lamp is determined according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp, but not only according to the detection result of the alternative traffic signal lamp. The advantage that sets up like this lies in, improves the degree of accuracy of the discernment of traffic signal lamp state, reduces the error, improves user experience.
In an optional embodiment of the present invention, determining the traffic light status according to the traffic light mapping result and the alternative traffic light detection result may include: and acquiring a target detection result matched with the mapping result of the traffic signal lamp in the detection result of the alternative traffic signal lamp, and determining the state of the traffic signal lamp according to the target detection result.
In the embodiment of the invention, the alternative traffic signal lamp detection result matched with the traffic signal lamp mapping result is selected as the target detection result from all the alternative traffic signal lamp detection results, and the traffic signal lamp state is determined according to the target detection result. In an optional embodiment of the present invention, determining the traffic signal lamp status according to the target detection result may include: and determining the state of the traffic signal lamp according to the target detection result and the control information of the target traffic signal lamp.
The control information may be information determined according to shape information of the target traffic signal. In a specific example, when the shape information of the target traffic signal is an up arrow, the control information may be determined to be straight. When the shape information of the target traffic signal light is a left arrow, it may be determined that the control information is a left turn.
In an optional implementation manner of the embodiment of the present invention, the target detection result is selected from the candidate traffic light detection results, and therefore includes the position, the contour, the display attribute information, and the like of the traffic light detected from the image acquired by the vehicle. After the target detection result is selected, the state of the traffic signal lamp can be determined directly according to the target detection result.
According to the technical scheme, after the detection result of the alternative traffic signal lamp is determined according to the image acquired by the vehicle, the target traffic signal lamp actually existing in front of the vehicle is mapped to the image acquired by the vehicle, the detection result of the alternative traffic signal lamp in the image is subjected to auxiliary verification through the position of the target traffic signal lamp in the image, the problem that in the prior art, when the state of the traffic signal lamp is directly identified according to the image acquired by the vehicle, the identification error rate is high is solved, the interference of non-traffic signal lamps or other traffic signal lamps can be effectively eliminated, and the state of the traffic signal can be accurately identified under the condition that the resolution of a vehicle-mounted shot image is low. The method realizes accurate identification of the state of the traffic signal lamp in front of the vehicle, and further ensures the safety of the vehicle in the driving process.
Example two
Fig. 2a is a flowchart of a traffic signal lamp state identification method according to a second embodiment of the present invention, and the second embodiment of the present invention further embodies an acquisition process of a candidate traffic signal lamp detection result, an acquisition process of a target traffic signal lamp, an acquisition process of a traffic signal lamp mapping result, and an acquisition process of a traffic signal lamp state on the basis of the above embodiments.
Correspondingly, as shown in fig. 2a, the technical solution of the embodiment of the present invention specifically includes the following steps:
s210, inputting the images acquired by the vehicle into a traffic signal lamp recognition model trained in advance, and acquiring the at least one alternative traffic signal lamp detection result.
S220, determining a search area according to the vehicle position information, and acquiring the geographical position information of at least one traffic signal lamp in the search area from map data.
S230, screening out at least one traffic signal lamp of which the geographic position information and the vehicle position information meet the distance correlation condition as the target traffic signal lamp.
And S240, point cloud data are obtained.
The laser sensor can be a sensor which is arranged on a vehicle, measures by using a laser technology and is used for collecting point cloud data.
In the embodiment of the invention, the point cloud data is acquired by transmitting and receiving laser information through a laser sensor arranged on a vehicle. The advantage that sets up like this lies in, through laser sensor, can collect vehicle itself and all around various accurate data constantly, carries out the accurate positioning to the vehicle, and the cloud data real-time nature that laser sensor gathered is better, and the precision is higher.
And S250, marking the target traffic signal lamp in the point cloud data according to the vehicle position information and the geographic position information of the target traffic signal lamp.
In the embodiment of the invention, after the point cloud data is acquired, the target traffic signal lamp can be marked according to the data attribute of the point cloud data and the position information of the vehicle and the target traffic signal lamp.
Optionally, after the point cloud data of the road is acquired by the laser sensor on the vehicle, the position information of the vehicle may include, for example, position coordinates of the vehicle, a yaw angle, and the like according to the current position information of the vehicle. And marking the position of the target traffic signal lamp in the point cloud data of the road according to the position relation between the screened target traffic signal lamp and the vehicle. This arrangement has an advantage of facilitating the acquisition of the coordinates of the apex of the target traffic signal.
And S260, acquiring each vertex coordinate of each target traffic signal lamp in the point cloud data.
The vertex coordinates may be three-dimensional coordinates of a vertex of the target traffic signal lamp. In one specific example, 8 three-dimensional coordinates { P } may be employedw＝(xi,yi,zi) And i is 1 … 8, which represents 8 vertexes of the traffic signal lamp.
In the embodiment of the invention, the target traffic signal lamp can be projected according to the vertex coordinates of the target traffic signal lamp, and the vertex coordinates can be obtained according to the point cloud data and the geographical position information of the target traffic signal lamp.
And S270, mapping each vertex coordinate to the image according to the internal and external parameters of the vehicle-mounted camera of the vehicle to obtain pixel point coordinates corresponding to each vertex coordinate of each target traffic signal lamp.
The internal and external parameters of the vehicle-mounted camera comprise internal parameters and external parameters of the camera, the internal parameters of the camera can comprise a camera focal length, an imaged pixel size and the like, and the external parameters of the camera can comprise a position, a rotation direction and a deviation of the camera in a three-dimensional coordinate systemDirection of movement, etc. The process of determining the internal and external parameters of the camera is also called camera calibration, and the embodiment does not limit the specific implementation method of the camera calibration. The pixel point coordinates can be coordinates of each vertex of the target traffic signal lamp in the image after the target traffic signal lamp is mapped into the image. In one specific example, each vertex can be mapped to a pixel P on the camera two-dimensional image by a transformationuv＝(u,v)。
In the embodiment of the invention, the vertex of the target traffic signal lamp is mapped into the image according to the internal and external parameters of the vehicle-mounted camera and the known three-dimensional coordinates of the vertex of the target traffic signal lamp by using a small hole imaging principle, so that the coordinates of the pixel points of the vertexes are obtained.
In an optional embodiment of the present invention, mapping each vertex coordinate to the image according to the internal and external parameters of the vehicle-mounted camera of the vehicle to obtain a pixel point coordinate corresponding to each vertex coordinate of each target traffic signal lamp may include: by the following formula: puv＝K(RPw+ t), coordinates P of each vertexwMapping the image to obtain a pixel point coordinate P corresponding to each vertex coordinate of each target traffic signal lampuv(ii) a Wherein, Pw＝(xi,yi,zi)，i∈[1,N]N denotes the number of vertices of a target traffic signal, PuvU, v, u, v representing coordinates of a pixel, K representing an internal reference matrix of the onboard camera, R representing a rotation matrix of the onboard camera, and t representing a translation vector of the onboard camera;
the internal parameters of the vehicle-mounted camera can be represented by an internal parameter matrix K, and the external parameters of the vehicle-mounted camera can be represented by a rotation matrix R and a translation vector t.
Illustratively, the reference matrix may be used
Is shown, wherein fx、fyDenotes the focal length, x, of the camera0、y0Denotes the principal point shift of the camera and s denotes the pixel size. The rotation matrix can be used
Where the columns of the rotation matrix R represent the directions of the world coordinate system axes in the camera coordinate system. The translation vector can be used
To indicate that the translation vector t represents the position of the world coordinate system origin in the camera coordinate system.
S280, calculating the average value of the pixel point coordinates of the adjacent pixel points to obtain the coordinates of each mapping point of each target traffic signal lamp in the image, and taking the coordinates of each mapping point as the mapping result of the traffic signal lamp.
The pixel point coordinate average value can be an average value of pixel point coordinates of two adjacent mapped vertexes. In a specific example, when the coordinates of two adjacent pixels are P respectivelyuv1＝(u1，v1)，Puv2＝(u2，v2) Then the coordinate of the mapping point is Puv＝(u1+u2/2,v1+v2/2). The mapping point coordinates may be coordinates of mapping points obtained by averaging coordinates of adjacent pixel points.
In the embodiment of the invention, three-dimensional vertex coordinates are mapped into an image, after corresponding pixel points are obtained, each pixel point has a nearest adjacent pixel point, the coordinates of two adjacent pixel points are averaged to be combined into one pixel point, the combined pixel point is used as a mapping point, and the coordinates of each mapping point are the mapping result of the traffic signal lamp.
In an optional embodiment of the present invention, the obtaining an average value of pixel coordinates of adjacent pixels to obtain coordinates of each mapping point of each target traffic signal lamp in the image may include: according to the coordinate P of each pixel pointuvTwo adjacent pixel points are used as pixel point pairs to obtain
A pixelPoint pair; calculating the average value of the pixel point coordinates of each pixel point pair to obtain the mapping point coordinates P of each target traffic signal lamp in the imageuvi(ii) a Wherein, Puvi＝(ui,vi)，i∈[1,M]M represents the coordinate number of the traffic signal mapping result of a target traffic signal in the image,
in the embodiment of the present invention, two nearest neighboring pixel points may be divided into a pixel point pair, and the coordinates of each pixel point of the pixel point pair are averaged to obtain the mapping point coordinates. In one specific example, 8 three-dimensional coordinates { P }are usedw＝(xi,yi,zi) And i is 1 … 8, representing 8 vertexes of the traffic signal lamp, mapping each vertex to an image to obtain 8 pixel point coordinates { P }uvi＝(ui,vi) I is 1 … 8, the coordinates of the pixels at adjacent positions are averaged, and finally 4 mapping point coordinates { P } are useduvi＝(ui,vi) And i is 1 … 4, which represents the mapping result of the target traffic signal lamp in the image.
And S290, respectively calculating the similarity between the detection result of each alternative traffic signal lamp and the mapping result of each traffic signal lamp.
Wherein, the similarity can be used to compare the similarity of two things. Similarity is generally expressed by calculating the distance between features of objects, and if the distance is small, the similarity is large, and if the distance is large, the similarity is small.
In an optional embodiment of the present invention, calculating the similarity between the candidate traffic signal light detection result and the traffic signal light mapping result may include: acquiring a first vertex pixel coordinate set and a second vertex pixel coordinate set which respectively correspond to a currently processed alternative traffic signal lamp detection result and a currently processed traffic signal lamp mapping result; calculating at least one two-dimensional image attribute parameter respectively corresponding to the first vertex pixel coordinate set and the second vertex pixel coordinate set; calculating at least one similarity calculation result between the currently processed alternative traffic signal lamp detection result and the currently processed traffic signal lamp mapping result according to the two-dimensional image attribute parameters; and according to the at least one item of similarity calculation result, obtaining the similarity between the currently processed alternative traffic signal lamp detection result and the currently processed traffic signal lamp mapping result.
The first vertex pixel coordinate set may be a set of pixel coordinates corresponding to detection results of the alternative traffic signal lamps. The second vertex pixel coordinate set may be a set of pixel coordinates corresponding to each traffic signal lamp mapping result. The two-dimensional image attribute parameters may include length, height, and center point distance, etc.
In a specific example, the candidate traffic signal light detection result Pa corresponds to a pixel coordinate Pa ═ Pa (Pa)1，pa2，pa3，pa4) And the pixel coordinate corresponding to the traffic signal lamp mapping result Pb is Pb ═ Pb (Pb)1，pb2，pb3，pb4). And calculating two-dimensional image attribute parameters of Pa and Pb, specifically calculating the center point distance of Pa and Pb, the height and length of Pa and the height and length of Pb. Calculating a first similarity according to the ratio of the distance between the center points of Pa and Pb and the distance between the diagonals of Pa, calculating a second similarity according to the ratio of the height of Pa to the height of Pb, and calculating a third similarity according to the ratio of the length of Pa to the length of Pb. And finally, calculating the similarity between the detection result Pa of the alternative traffic signal lamp and the mapping result Pb of the traffic signal lamp according to the numerical values of the similarities. Optionally, the numerical values of the similarity degrees may be directly added, and the sum of the numerical values of the similarity degrees is used as the similarity between the candidate traffic signal light detection result Pa and the traffic signal light mapping result Pb. When the similarity degrees have different weights, the sum of the products of the similarity degree values and the weights can be used as the similarity degree between the alternative traffic signal lamp detection result Pa and the traffic signal lamp mapping result Pb.
In the embodiment of the invention, the similarity is calculated according to the position information of the detection result of the alternative traffic signal lamp and the mapping result of the traffic signal lamp. The embodiment of the invention does not limit the attribute information used for calculating the similarity, and can also calculate the similarity by using other attributes of the alternative traffic signal lamp detection result and the traffic signal lamp mapping result.
In another optional embodiment of the present invention, when the position information of the candidate traffic light detection result and the traffic light mapping result is not enough to describe the similarity between the candidate traffic light detection result and the traffic light mapping result, the similarity may be calculated by combining the control information of the candidate traffic light detection result and the traffic light mapping result. Firstly, calculating a similarity basis according to the position information of the detection result of the alternative traffic signal lamp and the mapping result of the traffic signal lamp, and then judging whether the control information of the alternative traffic signal lamp and the control information of the traffic signal lamp are the same. In one particular example, if both are displayed as a right turn, the control information for both is the same. If the two pieces of control information are the same, weighting a coefficient w1 on the basis of the similarity, wherein w1 is greater than 1; and if the two pieces of control information are different, weighting a coefficient w2 on the basis of the similarity, wherein w2 is less than 1, so as to obtain the similarity between the final alternative traffic signal lamp detection result and the traffic signal lamp mapping result.
S2100, matching the alternative traffic signal lamp detection result and the traffic signal lamp mapping result according to the similarity calculation result and a preset matching algorithm, and taking the matched alternative traffic signal lamp detection result as a target detection result.
The matching algorithm is used for searching the optimal matching between the detection result of each alternative traffic signal lamp and the mapping result of each traffic signal lamp. Optionally, the matching algorithm may be a KM (Kuhn-Munkres) algorithm, or a greedy strategy may be used, and the selection of the matching algorithm is not limited in the embodiment of the present invention.
Optionally, the optimal matching between the alternative traffic signal lamp detection result and the traffic signal lamp mapping result may be obtained through a bipartite graph maximum weight matching algorithm. And respectively taking the detection result of each alternative traffic signal lamp and the mapping result of each traffic signal lamp as nodes of the bipartite graph, and taking the similarity between the detection result of the alternative traffic signal lamp and the mapping result of the traffic signal lamp as the weight between the nodes. And when the mapping result of each traffic signal lamp finds the corresponding alternative traffic signal lamp detection result and the sum of the weights between the mapping result of each traffic signal lamp and the corresponding alternative traffic signal lamp detection result is maximum, the alternative traffic signal lamp detection result which is optimally matched with the mapping result of each traffic signal lamp can be obtained as the target detection result.
In the embodiment of the invention, the detection results of all the alternative traffic signal lamps and the mapping results of all the traffic signal lamps are matched according to the similarity and the matching algorithm, and the detection results of the alternative traffic signal lamps matched with the mapping results of all the traffic signal lamps are used as target detection results.
And S2110, acquiring color information of the traffic signal lamp from the display attribute information of the target detection result.
In the embodiment of the invention, the color information of the traffic signal lamp is acquired according to the display attribute information of the target detection result and is used for indicating the forward running or stopping state displayed by the traffic signal lamp.
S2120, determining the state of a traffic light in front of the vehicle according to the color information and the control information of the target traffic light.
In an alternative embodiment of the present invention, fig. 2b provides a flowchart of a method for identifying a traffic signal status, which is suitable for use in the embodiment of the present invention, and as shown in fig. 2b, the steps of the method for identifying a traffic signal status include:
and S1, detecting and identifying the traffic signal lamp in the vehicle-mounted camera image by adopting a machine learning method.
And S2, acquiring the information of the nearest traffic signal lamp in front of the vehicle through positioning inquiry, and projecting the position of the traffic signal lamp in the three-dimensional laser point cloud onto a camera image by utilizing the position relation between the camera and the laser sensor.
Fig. 2c is a flowchart of a method for projecting the position of the traffic light in the three-dimensional laser point cloud onto the camera, and as shown in fig. 2c, the method for projecting the position of the traffic light in the three-dimensional laser point cloud onto the camera includes the following steps:
s201, collecting three-dimensional laser point cloud data of a road by using a laser sensor, and marking the position and the attribute of a traffic signal lamp in the three-dimensional laser point cloud.
S202, obtaining internal and external parameters of the camera through calibration.
And S203, projecting the position of the traffic signal lamp in the three-dimensional laser point cloud onto a two-dimensional image shot by the vehicle-mounted camera based on the parameters of the camera.
And S3, performing similarity matching based on the traffic signal lamp detection result in the image and the projection of the traffic signal lamp to obtain the best matching result.
Fig. 2d is a flowchart of a method for matching the similarity between the traffic signal detection result and the traffic signal projection in the image, and as shown in fig. 2d, the method for matching the similarity between the traffic signal detection result and the traffic signal projection includes the following steps:
s301, calculating the similarity between each traffic signal lamp detection result in the camera image and the traffic signal lamp projection.
And S302, calculating an optimal matching strategy according to the similarity to obtain the optimal matching between the traffic signal lamp and the detection result.
And S4, in the matching alignment, combining the detection result and the labeling information of the traffic signal lamp corresponding to the projection in the three-dimensional point cloud to obtain the state of the traffic signal.
The technical proposal of the embodiment of the invention obtains the detection result of the alternative traffic signal lamp by inputting the image obtained by the vehicle into the traffic signal lamp identification model, acquiring geographical position information of the traffic signal lamp in the map data through a search area determined by the vehicle position information, screening to obtain a target traffic signal lamp, marking the target traffic signal lamp in the point cloud data through the point cloud data acquired in real time and the position information of the vehicle and the target traffic signal lamp, and mapping the target traffic signal lamp into the image according to the camera parameters to obtain a traffic signal lamp mapping result, the target detection result is obtained by calculating the similarity between the detection result of each alternative traffic signal lamp and the mapping result of each traffic signal lamp and carrying out optimal matching, and determining the state of the traffic signal lamp according to the target detection result and the control information of the target traffic signal lamp. The problem of among the prior art when directly discerning the traffic signal lamp state according to the image that the vehicle obtained, the discernment error rate is higher is solved, can effectively get rid of the interference of non-traffic signal lamp or other traffic signal lamps, also can accurately discern the state of traffic signal under the condition that on-vehicle image resolution ratio of shooing is low. The method realizes accurate identification of the state of the traffic signal lamp in front of the vehicle, and further ensures the safety of the vehicle in the driving process.
EXAMPLE III
Fig. 3 is a schematic structural diagram of a traffic signal lamp status identification device according to a third embodiment of the present invention, where the device includes: an alternative traffic signal light detection result acquisition module 310, a target traffic signal light acquisition module 320, a traffic signal light mapping result acquisition module 330, and a traffic signal light status determination module 340.
Wherein:
the alternative traffic signal lamp detection result acquisition module 310 is configured to determine at least one alternative traffic signal lamp detection result according to an image acquired by a vehicle;
a target traffic signal lamp obtaining module 320, configured to determine, according to the map data, a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps;
a traffic signal lamp mapping result obtaining module 330, configured to map the target traffic signal lamp into the image according to a relative position relationship between the target traffic signal lamp and the vehicle, so as to obtain a traffic signal lamp mapping result;
and the traffic signal lamp state determining module 340 is configured to determine the state of the traffic signal lamp according to the traffic signal lamp mapping result and the alternative traffic signal lamp detection result.
According to the technical scheme, after the detection result of the alternative traffic signal lamp is determined according to the image acquired by the vehicle, the target traffic signal lamp actually existing in front of the vehicle is mapped to the image acquired by the vehicle, the detection result of the alternative traffic signal lamp in the image is subjected to auxiliary verification through the position of the target traffic signal lamp in the image, the problem that in the prior art, when the state of the traffic signal lamp is directly identified according to the image acquired by the vehicle, the identification error rate is high is solved, the interference of non-traffic signal lamps or other traffic signal lamps can be effectively eliminated, and the state of the traffic signal can be accurately identified under the condition that the resolution of a vehicle-mounted shot image is low. The method realizes accurate identification of the state of the traffic signal lamp in front of the vehicle, and further ensures the safety of the vehicle in the driving process.
On the basis of the above embodiment, the traffic signal status determining module 340 includes:
and the target detection result acquisition unit is used for acquiring a target detection result matched with the traffic signal lamp mapping result in the alternative traffic signal lamp detection result and determining the traffic signal lamp state according to the target detection result.
On the basis of the above embodiment, the traffic signal lamp mapping result obtaining module 330 includes:
the target traffic signal lamp marking unit is used for marking the target traffic signal lamp in the point cloud data acquired by the vehicle in real time according to the geographical position information of the target traffic signal lamp;
and the target traffic signal lamp mapping unit is used for mapping the marked target traffic signal lamp into the image according to the internal and external parameters of the vehicle-mounted camera of the vehicle to obtain a traffic signal lamp mapping result.
On the basis of the above embodiment, the alternative traffic signal light detection result obtaining module 310 includes:
the alternative traffic signal lamp detection result acquisition unit is used for inputting the images acquired by the vehicles into a traffic signal lamp recognition model trained in advance and acquiring at least one alternative traffic signal lamp detection result;
and the detection result of the alternative traffic signal lamp comprises the position, the outline and the display attribute information of the alternative traffic signal lamp in the image.
On the basis of the above embodiment, the target traffic signal acquiring module 320 includes:
the geographical position information acquisition unit is used for determining a search area according to the vehicle position information and acquiring geographical position information of at least one traffic signal lamp in the search area from map data;
and the target traffic signal lamp acquisition unit is used for screening out at least one traffic signal lamp of which the geographic position information and the vehicle position information meet the distance correlation condition as the target traffic signal lamp.
On the basis of the above embodiment, the target traffic signal labeling unit includes:
the point cloud data acquisition subunit is used for acquiring point cloud data;
and the target traffic signal lamp labeling subunit is used for labeling the target traffic signal lamp in the point cloud data according to the vehicle position information and the geographic position information of the target traffic signal lamp.
On the basis of the above embodiment, the target traffic signal lamp mapping unit includes:
the vertex coordinate acquisition subunit is used for acquiring each vertex coordinate of each target traffic signal lamp in the point cloud data;
the vertex coordinate mapping subunit is used for mapping each vertex coordinate into the image according to the internal and external parameters of the vehicle-mounted camera of the vehicle to obtain pixel point coordinates corresponding to each vertex coordinate of each target traffic signal lamp;
and the mapping point coordinate obtaining subunit is used for solving the average value of the pixel point coordinates of the adjacent pixel points to obtain the mapping point coordinates of each target traffic signal lamp in the image, and taking the mapping point coordinates as the mapping result of the traffic signal lamp.
On the basis of the foregoing embodiment, the vertex coordinate mapping subunit is specifically configured to:
by the following formula: puv＝K(RPw+ t), coordinates P of each vertexwMapping the image to obtain a pixel point coordinate P corresponding to each vertex coordinate of each target traffic signal lampuv；
Wherein, Pw＝(xi,yi,zi)，i∈[1,N]N represents a target trafficNumber of signal lamp vertices, PuvU, v, u, v representing coordinates of a pixel, K representing an internal reference matrix of the onboard camera, R representing a rotation matrix of the onboard camera, and t representing a translation vector of the onboard camera;
the mapping point coordinate obtaining subunit is specifically configured to:
according to the coordinate P of each pixel pointuvTwo adjacent pixel points are used as pixel point pairs to obtain
A pixel point pair;
calculating the average value of the pixel point coordinates of each pixel point pair to obtain the mapping point coordinates P of each target traffic signal lamp in the imageuvi；
Wherein, Puvi＝(ui,vi)，i∈[1,M]M represents the coordinate number of the traffic signal mapping result of a target traffic signal in the image,
on the basis of the above embodiment, the target detection result acquiring unit includes:
the similarity calculation unit is used for calculating the similarity between the detection result of each alternative traffic signal lamp and the mapping result of each traffic signal lamp;
and the target detection result acquisition subunit is used for matching the alternative traffic signal lamp detection result with the traffic signal lamp mapping result according to the calculation result of the similarity and a preset matching algorithm, and taking the matched alternative traffic signal lamp detection result as a target detection result.
On the basis of the foregoing embodiment, the similarity calculation unit is specifically configured to:
acquiring a first vertex pixel coordinate set and a second vertex pixel coordinate set which respectively correspond to a currently processed alternative traffic signal lamp detection result and a currently processed traffic signal lamp mapping result;
calculating at least one two-dimensional image attribute parameter respectively corresponding to the first vertex pixel coordinate set and the second vertex pixel coordinate set;
calculating at least one similarity calculation result between the currently processed alternative traffic signal lamp detection result and the currently processed traffic signal lamp mapping result according to the two-dimensional image attribute parameters;
and according to the at least one item of similarity calculation result, obtaining the similarity between the currently processed alternative traffic signal lamp detection result and the currently processed traffic signal lamp mapping result.
On the basis of the above embodiment, the target detection result acquiring unit includes:
and the traffic signal lamp state determining subunit is used for determining the state of the traffic signal lamp according to the target detection result and the control information of the target traffic signal lamp.
On the basis of the above embodiment, the traffic signal lamp state determining subunit is specifically configured to:
acquiring color information of a traffic signal lamp from the display attribute information of the target detection result;
and determining the state of the traffic signal lamp in front of the vehicle according to the color information and the control information of the target traffic signal lamp.
The traffic signal lamp state identification device provided by the embodiment of the invention can execute the traffic signal lamp state identification method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example four
Fig. 4 is a schematic structural diagram of a computer apparatus according to a fourth embodiment of the present invention, as shown in fig. 4, the computer apparatus includes a processor 70, a memory 71, an input device 72, and an output device 73; the number of processors 70 in the computer device may be one or more, and one processor 70 is taken as an example in fig. 4; the processor 70, the memory 71, the input device 72 and the output device 73 in the computer apparatus may be connected by a bus or other means, and the connection by the bus is exemplified in fig. 4.
The memory 71, as a computer-readable storage medium, may be used for storing software programs, computer-executable programs, and modules, such as the modules corresponding to the traffic signal status recognition method in the embodiment of the present invention (e.g., the alternative traffic signal detection result acquisition module 310, the target traffic signal acquisition module 320, the traffic signal mapping result acquisition module 330, and the traffic signal status determination module 340 in the traffic signal status recognition apparatus). The processor 70 executes various functional applications and data processing of the computer device by executing software programs, instructions and modules stored in the memory 71, that is, implements the traffic signal state recognition method described above. The method comprises the following steps:
determining at least one alternative traffic signal lamp detection result according to an image acquired by a vehicle;
determining a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps according to the map data;
mapping the target traffic signal lamp to the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result;
and determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp.
The memory 71 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 71 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 71 may further include memory located remotely from the processor 70, which may be connected to a computer device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 72 may be used to receive input numeric or character information and generate key signal inputs relating to user settings and function controls of the computer apparatus. The output device 73 may include a display device such as a display screen.
EXAMPLE five
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a traffic signal status identification method, including:
determining at least one alternative traffic signal lamp detection result according to an image acquired by a vehicle;
determining a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps according to the map data;
mapping the target traffic signal lamp to the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result;
and determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp.
Of course, the storage medium containing the computer-executable instructions provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the traffic signal status recognition method provided by any embodiments of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the traffic signal lamp state identification device, the included units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (12)
1. A traffic signal lamp state identification method is characterized by comprising the following steps:
determining at least one alternative traffic signal lamp detection result according to an image acquired by a vehicle;
determining a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps according to the map data;
mapping the target traffic signal lamp to the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result;
and determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp.
2. The method of claim 1, wherein determining a traffic signal status based on the traffic signal mapping results and the alternative traffic signal detection results comprises:
and acquiring a target detection result matched with the mapping result of the traffic signal lamp in the detection result of the alternative traffic signal lamp, and determining the state of the traffic signal lamp according to the target detection result.
3. The method of claim 1, wherein mapping the target traffic signal into the image according to a relative position relationship between the target traffic signal and the vehicle to obtain a traffic signal mapping result comprises:
marking the target traffic signal lamp in point cloud data acquired by a vehicle in real time according to vehicle position information and the geographical position information of the target traffic signal lamp;
and mapping the marked target traffic signal lamp into the image according to the internal and external parameters of the vehicle-mounted camera of the vehicle to obtain a traffic signal lamp mapping result.
4. The method of claim 1, wherein determining a target traffic signal from the alternative traffic signals that matches vehicle location information based on map data comprises:
determining a search area according to the vehicle position information, and acquiring the geographical position information of at least one traffic signal lamp in the search area from map data;
and screening out at least one traffic signal lamp of which the geographic position information and the vehicle position information meet the distance correlation condition as the target traffic signal lamp.
5. The method of claim 3, wherein mapping the labeled target traffic signal into the image according to internal and external parameters of a vehicle-mounted camera of the vehicle to obtain a traffic signal mapping result, comprises:
acquiring each vertex coordinate of each target traffic signal lamp in the point cloud data;
mapping each vertex coordinate to the image according to the internal and external parameters of the vehicle-mounted camera of the vehicle to obtain a pixel point coordinate corresponding to each vertex coordinate of each target traffic signal lamp;
and solving the average value of the pixel point coordinates of the adjacent pixel points to obtain the coordinates of each mapping point of each target traffic signal lamp in the image, and taking the coordinates of each mapping point as the mapping result of the traffic signal lamp.
6. The method of claim 5, wherein mapping each vertex coordinate into the image according to an inside-outside parameter of a vehicle-mounted camera of the vehicle to obtain a pixel point coordinate corresponding to each vertex coordinate of each target traffic signal lamp comprises:
by the following formula: puv＝K(RPw+ t), coordinates P of each vertexwMapping the image to obtain a pixel point coordinate P corresponding to each vertex coordinate of each target traffic signal lampuv；
Wherein, Pw＝(xi,yi,zi)，i∈[1,N]N denotes the number of vertices of a target traffic signal, PuvU, v, u, v representing coordinates of a pixel, K representing an internal reference matrix of the onboard camera, R representing a rotation matrix of the onboard camera, and t representing a translation vector of the onboard camera;
solving the average value of the pixel point coordinates of the adjacent pixel points to obtain the coordinates of each mapping point of each target traffic signal lamp in the image, comprising the following steps:
according to the coordinate P of each pixel pointuvTwo adjacent pixel points are used as pixel point pairs to obtain
A pixel point pair;
for each pixelCalculating the average value of the pixel point coordinates by the point pairs to obtain the mapping point coordinates P of each target traffic signal lamp in the imageuvi；
Wherein, Puvi＝(ui,vi)，i∈[1,M]M represents the coordinate number of the traffic signal mapping result of a target traffic signal in the image,
7. the method of claim 2, wherein obtaining, among the alternative traffic signal detection results, a target detection result matching the traffic signal mapping result comprises:
respectively calculating the similarity between the detection result of each alternative traffic signal lamp and the mapping result of each traffic signal lamp;
and matching the detection result of the alternative traffic signal lamp with the mapping result of the traffic signal lamp according to the calculation result of the similarity and a preset matching algorithm, and taking the matched detection result of the alternative traffic signal lamp as a target detection result.
8. The method of claim 7, wherein calculating a similarity between alternative traffic signal detection results and the traffic signal mapping results comprises:
acquiring a first vertex pixel coordinate set and a second vertex pixel coordinate set which respectively correspond to a currently processed alternative traffic signal lamp detection result and a currently processed traffic signal lamp mapping result;
calculating at least one two-dimensional image attribute parameter respectively corresponding to the first vertex pixel coordinate set and the second vertex pixel coordinate set;
calculating at least one similarity calculation result between the currently processed alternative traffic signal lamp detection result and the currently processed traffic signal lamp mapping result according to the two-dimensional image attribute parameters;
and according to the at least one item of similarity calculation result, obtaining the similarity between the currently processed alternative traffic signal lamp detection result and the currently processed traffic signal lamp mapping result.
9. The method of claim 2, wherein determining a traffic signal light status based on the target detection result comprises:
acquiring color information of a traffic signal lamp from the display attribute information of the target detection result;
and determining the state of the traffic signal lamp according to the color information and the control information of the target traffic signal lamp.
10. A traffic signal status recognition apparatus, comprising:
the alternative traffic signal lamp detection result acquisition module is used for determining at least one alternative traffic signal lamp detection result according to the image acquired by the vehicle;
the target traffic signal lamp acquisition module is used for determining a target traffic signal lamp matched with the vehicle position information from the alternative traffic signal lamps according to the map data;
the traffic signal lamp mapping result acquisition module is used for mapping the target traffic signal lamp into the image according to the relative position relation between the target traffic signal lamp and the vehicle to obtain a traffic signal lamp mapping result;
and the traffic signal lamp state determining module is used for determining the state of the traffic signal lamp according to the mapping result of the traffic signal lamp and the detection result of the alternative traffic signal lamp.
11. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the traffic signal status recognition method according to any one of claims 1 to 9 when executing the program.
12. A storage medium containing computer-executable instructions for performing the traffic signal status identification method of any one of claims 1-9 when executed by a computer processor.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2020100927107 | 2020-02-14 | ||
| CN202010092710 | 2020-02-14 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111310708A true CN111310708A (en) | 2020-06-19 |
Family
ID=71162046
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010130208.0A Pending CN111310708A (en) | 2020-02-14 | 2020-02-28 | Traffic signal lamp state identification method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111310708A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112101272A (en) * | 2020-09-23 | 2020-12-18 | 北京百度网讯科技有限公司 | Traffic light detection method and device, computer storage medium and road side equipment |
| CN112489466A (en) * | 2020-11-27 | 2021-03-12 | 恒大新能源汽车投资控股集团有限公司 | Traffic signal lamp identification method and device |
| CN113343872A (en) * | 2021-06-17 | 2021-09-03 | 湖北亿咖通科技有限公司 | Traffic light identification method, device, equipment, medium and product |
| WO2022037693A1 (en) * | 2020-08-20 | 2022-02-24 | 华为技术有限公司 | Traffic signal identification method and apparatus |
| CN114972824A (en) * | 2022-06-24 | 2022-08-30 | 小米汽车科技有限公司 | Rod detection method and device, vehicle and storage medium |
| WO2023273358A1 (en) * | 2021-06-30 | 2023-01-05 | 上海商汤智能科技有限公司 | Traffic sign recognition method and apparatus, electronic device, and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108305475A (en) * | 2017-03-06 | 2018-07-20 | 腾讯科技（深圳）有限公司 | A kind of traffic lights recognition methods and device |
| CN110543814A (en) * | 2019-07-22 | 2019-12-06 | 华为技术有限公司 | Traffic light identification method and device |
| CN110705485A (en) * | 2019-10-08 | 2020-01-17 | 东软睿驰汽车技术（沈阳）有限公司 | Traffic signal lamp identification method and device |
-
2020
- 2020-02-28 CN CN202010130208.0A patent/CN111310708A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108305475A (en) * | 2017-03-06 | 2018-07-20 | 腾讯科技（深圳）有限公司 | A kind of traffic lights recognition methods and device |
| CN110543814A (en) * | 2019-07-22 | 2019-12-06 | 华为技术有限公司 | Traffic light identification method and device |
| CN110705485A (en) * | 2019-10-08 | 2020-01-17 | 东软睿驰汽车技术（沈阳）有限公司 | Traffic signal lamp identification method and device |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022037693A1 (en) * | 2020-08-20 | 2022-02-24 | 华为技术有限公司 | Traffic signal identification method and apparatus |
| CN112101272A (en) * | 2020-09-23 | 2020-12-18 | 北京百度网讯科技有限公司 | Traffic light detection method and device, computer storage medium and road side equipment |
| CN112489466A (en) * | 2020-11-27 | 2021-03-12 | 恒大新能源汽车投资控股集团有限公司 | Traffic signal lamp identification method and device |
| CN113343872A (en) * | 2021-06-17 | 2021-09-03 | 湖北亿咖通科技有限公司 | Traffic light identification method, device, equipment, medium and product |
| WO2023273358A1 (en) * | 2021-06-30 | 2023-01-05 | 上海商汤智能科技有限公司 | Traffic sign recognition method and apparatus, electronic device, and storage medium |
| CN114972824A (en) * | 2022-06-24 | 2022-08-30 | 小米汽车科技有限公司 | Rod detection method and device, vehicle and storage medium |
| CN114972824B (en) * | 2022-06-24 | 2023-07-14 | 小米汽车科技有限公司 | Rod detection method, device, vehicle and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111310708A (en) | Traffic signal lamp state identification method, device, equipment and storage medium | |
| CN111291676B (en) | Lane line detection method and device based on laser radar point cloud and camera image fusion and chip | |
| US11094112B2 (en) | Intelligent capturing of a dynamic physical environment | |
| JP5168601B2 (en) | Own vehicle position recognition system | |
| JP5062498B2 (en) | Reference data generation system and position positioning system for landscape matching | |
| JP5388082B2 (en) | Stationary object map generator | |
| JP5404861B2 (en) | Stationary object map generator | |
| CN113673282A (en) | Target detection method and device | |
| CN110969064B (en) | Image detection method and device based on monocular vision and storage equipment | |
| CN108645375B (en) | Rapid vehicle distance measurement optimization method for vehicle-mounted binocular system | |
| JP5286653B2 (en) | Stationary object map generator | |
| US10872246B2 (en) | Vehicle lane detection system | |
| US10996337B2 (en) | Systems and methods for constructing a high-definition map based on landmarks | |
| CN114051628B (en) | Method and device for determining target object point cloud set | |
| CN114413881A (en) | Method and device for constructing high-precision vector map and storage medium | |
| JP2018077162A (en) | Vehicle position detection device, vehicle position detection method and computer program for vehicle position detection | |
| JP2015194397A (en) | Vehicle location detection device, vehicle location detection method, vehicle location detection computer program and vehicle location detection system | |
| JP2017181476A (en) | Vehicle location detection device, vehicle location detection method and vehicle location detection-purpose computer program | |
| JP2021149863A (en) | Object state identifying apparatus, object state identifying method, computer program for identifying object state, and control apparatus | |
| CN112255604B (en) | Method and device for judging accuracy of radar data and computer equipment | |
| CN116343165A (en) | 3D target detection system, method, terminal equipment and storage medium | |
| CN113611008B (en) | Vehicle driving scene acquisition method, device, equipment and medium | |
| CN112424568A (en) | System and method for constructing high-definition map | |
| JP2011214961A (en) | Reference pattern information generating device, method, program and general vehicle position specifying device | |
| CN112598736A (en) | Map construction based visual positioning method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |