CN111079680A - Temporary traffic signal lamp detection method and device and automatic driving equipment - Google Patents
Temporary traffic signal lamp detection method and device and automatic driving equipment Download PDFInfo
- Publication number
- CN111079680A CN111079680A CN201911342645.2A CN201911342645A CN111079680A CN 111079680 A CN111079680 A CN 111079680A CN 201911342645 A CN201911342645 A CN 201911342645A CN 111079680 A CN111079680 A CN 111079680A
- Authority
- CN
- China
- Prior art keywords
- traffic signal
- signal lamp
- temporary traffic
- point cloud
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/584—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of vehicle lights or traffic lights
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V20/582—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of traffic signs
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/09—Arrangements for giving variable traffic instructions
- G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
- G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
- G08G1/096708—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control
- G08G1/096725—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control where the received information generates an automatic action on the vehicle control
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Atmospheric Sciences (AREA)
- Traffic Control Systems (AREA)
Abstract
The application discloses a temporary traffic signal lamp detection method and device and automatic driving equipment. The method comprises the following steps: acquiring a current image and a corresponding three-dimensional point cloud; identifying a temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in an image coordinate system; projecting the three-dimensional point cloud to the image coordinate system, and determining the point cloud matched with the temporary traffic signal lamp; and determining the position of the temporary traffic signal lamp in a world coordinate system according to the point cloud matched with the temporary traffic signal lamp. The traffic signal lamp detection method has the advantages that on the basis that road facilities are not required to be improved, based on the existing road infrastructure conditions, the temporary traffic signal lamp can be automatically identified and detected, the spatial position of the temporary traffic signal lamp is accurately determined, a good auxiliary effect is achieved on road driving, particularly automatic driving of unmanned equipment, and driving safety and automation degree are effectively improved.
Description
Technical Field
The application relates to the field of automatic driving, in particular to a temporary traffic signal lamp detection method and device and automatic driving equipment.
Background
In the field of automatic driving, accurate detection and identification of traffic signal lamps are indispensable and are key technologies which are urgently needed to be broken through at present. In the prior art, a map is generally used for identifying traffic lights on roads, but in a real scene, not only fixed traffic lights but also temporary traffic lights (for example, fig. 5) which are temporarily placed due to faults of the fixed traffic lights at intersections or failure in timely deploying the fixed traffic lights exist, but the map does not contain information of the temporary traffic lights, so that the temporary traffic lights cannot be effectively identified in the prior art, and the reliability of automatic driving is further influenced.
Disclosure of Invention
In view of the above, the present application is made to provide a temporary traffic signal detection method, apparatus and autopilot device that overcome or at least partially address the above-mentioned problems.
According to a first aspect of the present application, there is provided a temporary traffic signal detection method, including:
acquiring a current image and a corresponding three-dimensional point cloud;
identifying a temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in an image coordinate system;
projecting the three-dimensional point cloud to the image coordinate system, and determining the point cloud matched with the temporary traffic signal lamp;
and determining the position of the temporary traffic signal lamp in a world coordinate system according to the point cloud matched with the temporary traffic signal lamp.
Optionally, the method is performed in case that a temporary traffic signal detection condition is satisfied.
Optionally, the condition that the temporary traffic signal detection condition is satisfied includes at least one of: the current position is an intersection, and the intersection is not provided with a fixed traffic signal lamp; the current position is a crossing, and the crossing is provided with a fixed traffic signal lamp, but the fixed traffic signal lamp has a fault.
Optionally, the image is acquired by a camera of the autonomous device, and the three-dimensional point cloud is acquired by a laser radar of the autonomous device;
the camera and the laser radar are subjected to pre-calibration treatment.
Optionally, the identifying the temporary traffic signal from the image includes identifying the temporary traffic signal from the image by using a traffic signal detection model including a bezel detection network, a bulb detection network, and an ensemble detection network, specifically:
identifying a plurality of lamp frames from the image by using a lamp frame detection network, identifying a plurality of light-emitting lamps from the image by using a lamp bulb detection network, and identifying a plurality of temporary traffic signal lamp integrals from the image by using the integral detection network;
matching the identified lamp frame with the identified light-emitting lamp bulb to obtain the lamp frame containing the light-emitting lamp bulb;
and matching the lamp frame containing the light-emitting bulb with the temporary traffic signal lamp integrally to obtain the temporary traffic signal lamp.
Optionally, the traffic signal detection model further comprises an indication shape detection network, and the identifying a number of light emitting bulbs from the image using the bulb detection network comprises:
identifying the identified light-emitting bulb by using an indication shape detection network, and determining the indication shape of the light-emitting bulb; the indicating shape includes at least one of: straight, left turn, right turn.
Optionally, the projecting the three-dimensional point cloud to the image coordinate system, and the determining the point cloud matched with the temporary traffic signal lamp includes:
projecting the three-dimensional point cloud to the image coordinate system to obtain a point cloud to be filtered corresponding to the temporary traffic signal lamp;
and acquiring map data corresponding to the current position, filtering the point cloud to be filtered by using the map data, and taking the filtered point cloud as the point cloud matched with the temporary traffic signal lamp.
According to a second aspect of the present application, there is provided a temporary traffic signal detecting device, comprising:
the acquisition unit is used for acquiring a current image and a corresponding three-dimensional point cloud;
the determining unit is used for identifying a temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in an image coordinate system;
the matching unit is used for projecting the three-dimensional point cloud to the image coordinate system and determining the point cloud matched with the temporary traffic signal lamp;
and the position unit is used for determining the position of the temporary traffic signal lamp in the world coordinate system according to the point cloud matched with the temporary traffic signal lamp.
Optionally, the device is operated in a situation where a temporary traffic signal detection condition is met.
Optionally, the condition that the temporary traffic signal detection condition is satisfied includes at least one of: the current position is an intersection, and the intersection is not provided with a fixed traffic signal lamp; the current position is a crossing, and the crossing is provided with a fixed traffic signal lamp, but the fixed traffic signal lamp has a fault.
Optionally, the image is acquired by a camera of the autonomous device, and the three-dimensional point cloud is acquired by a laser radar of the autonomous device; the camera and the laser radar are subjected to pre-calibration treatment.
Optionally, the determining unit is configured to identify a temporary traffic signal from the image by using a traffic signal detection model including a lamp frame detection network, a bulb detection network, and an overall detection network, specifically: identifying a plurality of light-emitting bulbs from the image by using the bulb detection network, and identifying a plurality of temporary traffic signal lamp integers from the image by using the integral detection network; matching the identified lamp frame with the identified light-emitting lamp bulb to obtain the lamp frame containing the light-emitting lamp bulb; and matching the lamp frame containing the light-emitting bulb with the temporary traffic signal lamp integrally to obtain the temporary traffic signal lamp.
Optionally, the traffic signal light detection model further includes an indication shape detection network, and the determining unit is configured to identify the identified light-emitting bulb by using the indication shape detection network, and determine an indication shape of the light-emitting bulb; the indicating shape includes at least one of: straight, left turn, right turn.
Optionally, the matching unit is configured to project the three-dimensional point cloud to the image coordinate system to obtain a point cloud to be filtered corresponding to the temporary traffic signal lamp; and acquiring map data corresponding to the current position, filtering the point cloud to be filtered by using the map data, and taking the filtered point cloud as the point cloud matched with the temporary traffic signal lamp.
According to a third aspect of the present application, there is provided an automatic driving apparatus including: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to perform a method as any one of the above.
According to a fourth aspect of the application, there is provided a computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement a method as in any above.
According to the technical scheme, the current image and the corresponding three-dimensional point cloud are obtained; identifying a temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in an image coordinate system; projecting the three-dimensional point cloud to the image coordinate system, and determining the point cloud matched with the temporary traffic signal lamp; and determining the position of the temporary traffic signal lamp in a world coordinate system according to the point cloud matched with the temporary traffic signal lamp. The traffic signal lamp detection method has the advantages that on the basis that road facilities are not required to be improved, based on the existing road infrastructure conditions, the temporary traffic signal lamp can be automatically identified and detected, the spatial position of the temporary traffic signal lamp is accurately determined, a good auxiliary effect is achieved on road driving, particularly automatic driving of unmanned equipment, and driving safety and automation degree are effectively improved.
The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a schematic flow diagram of a temporary traffic signal detection method according to one embodiment of the present application;
FIG. 2 illustrates a schematic structural diagram of a temporary traffic signal detection device according to one embodiment of the present application;
FIG. 3 shows a schematic structural diagram of an autopilot device according to an embodiment of the present application;
FIG. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present application.
Fig. 5 shows a schematic structural diagram of a temporary traffic signal.
Detailed Description
Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The method for detecting the traffic lights through the map mainly comprises the steps of drawing the positions of the traffic lights and the mapping relation between the traffic lights and lanes into the map, and then realizing the detection of the traffic lights through a traffic light detection module. The traffic signal lamp detection module can be realized through the following main processes: 1) obtaining position information through a GNSS (global navigation satellite system) sensor, and searching a traffic signal lamp associated with a current lane; 2) projecting the position of the traffic signal lamp to a camera image coordinate system to obtain an interested area; 3) inputting the image interesting area into a traffic signal lamp detection model to obtain a detection result; 4) and matching the output of the model with a traffic signal lamp in the map, and outputting a final result. However, as in the background art, in the map-based solution, since the traffic light position needs to be drawn in advance as a main element into the map, the temporary traffic light position is not fixed, and thus cannot be solved by the map-based solution.
The V2X (vehicle to outside information exchange) technology is to implement exchange and sharing of vehicle and X (including people, vehicles, drive test infrastructure, background cloud, etc.) intelligent information by carrying advanced vehicle-mounted sensors, controllers, actuators, etc. and integrating modern communication and network technologies, and to sense the surrounding conditions of the vehicle in real time, including the state of traffic lights. However, in the solutions based on the V2X technology, on the one hand, the V2X technology itself still needs to be further developed, and on the other hand, the road test infrastructure needs to be modified.
Therefore, the technical scheme of the application provides a new feasible temporary traffic signal lamp detection and identification method, and the temporary traffic signal lamp can be detected based on the existing road infrastructure conditions.
Fig. 1 shows a schematic flow diagram of a temporary traffic signal detection method according to an embodiment of the present application, and as shown in fig. 1, the method includes:
step S110, acquiring a current image and a corresponding three-dimensional point cloud. For example, images and three-dimensional point clouds may be acquired by sensors of the autopilot device.
And step S120, identifying the temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in the image coordinate system.
The temporary traffic signal lamp can be identified from the image according to the characteristics of the temporary traffic signal lamp, the state of the temporary traffic signal lamp can be further determined, and the state information can comprise the lighting condition of the traffic signal lamp, the indication shape type and other information. And establishing an image coordinate system and determining the position of the temporary traffic signal lamp in the image coordinate system. Thus, the information analysis of the temporary traffic signal lamp in the image is realized.
And step S130, projecting the three-dimensional point cloud to an image coordinate system, and determining the point cloud matched with the temporary traffic signal lamp.
The camera and the laser radar can be calibrated in advance, so that the three-dimensional point cloud data and the image data have a one-to-one correspondence relationship, the three-dimensional point cloud is conveniently projected onto an image coordinate system according to the corresponding mapping relationship subsequently, and the point cloud matched with the temporary traffic signal lamp can be obtained. Therefore, the point cloud data matched with the temporary traffic signal lamp can be obtained according to the image data.
And step S140, determining the position of the temporary traffic signal lamp in the world coordinate system according to the point cloud matched with the temporary traffic signal lamp.
The point cloud data matched with the temporary traffic signal lamp can reflect the spatial position relation of the temporary traffic signal lamp, in order to obtain accurate spatial position information of the temporary traffic signal lamp, the point cloud data matched with the temporary traffic signal lamp can be analyzed, a world coordinate system can be established, and coordinate information of the temporary traffic signal lamp in the world coordinate system can be determined in the world coordinate system. Therefore, the spatial position can be accurately determined according to the point cloud data matched with the temporary traffic signal lamp and is represented by the position information in the world coordinate system.
Therefore, the method shown in fig. 1 can automatically identify and detect the temporary traffic signal lamp based on the existing road infrastructure conditions without improving the road infrastructure, accurately determine the spatial position of the temporary traffic signal lamp, has a good auxiliary effect on road driving, particularly automatic driving of unmanned equipment, effectively improves driving safety and automation degree, and can be further applied to the fields of logistics, takeaway distribution and the like.
In one embodiment of the present application, among the above-described methods, the temporary traffic signal detection method is performed in a case where a temporary traffic signal detection condition is satisfied.
The fixed traffic signal lamp belongs to traffic infrastructure and is usually arranged at positions such as a traffic road intersection, while the temporary traffic signal lamp is not usually arranged at the traffic road intersection, and the temporary traffic signal lamp is generally arranged under specific conditions that the fixed traffic signal lamp breaks down or the road is a temporary road and the like. Therefore, when the temporary traffic signal lamp is detected, the detection condition of the temporary traffic signal lamp can be preset, and when the detection condition of the temporary traffic signal lamp is met, the detection process of the temporary traffic signal lamp is started again, the current image and the corresponding three-dimensional point cloud are obtained, and the subsequent steps are executed.
Certainly, in practical application, the current image and the corresponding three-dimensional point cloud not only can be used for identifying the temporary traffic signal lamp, but also can be used for identifying information such as obstacles, for example, a temporary traffic signal lamp identification sub-module and an obstacle sub-identification module are arranged and both belong to the identification module, the identification module acquires the image and the three-dimensional point cloud from the vehicle-mounted camera and the vehicle-mounted laser radar at a preset frequency, and the image and the three-dimensional point cloud are redistributed to the corresponding sub-modules to perform corresponding identification or detection processes.
In one embodiment of the present application, the condition that the temporary traffic signal detection condition is satisfied in the above method includes at least one of: the current position is an intersection, and the intersection is not provided with a fixed traffic signal lamp; the current position is a crossing, and the crossing is provided with a fixed traffic signal lamp, but the fixed traffic signal lamp has a fault.
Since both the fixed traffic lights and the temporary traffic lights are generally disposed at intersections of roads, it can be determined whether the current position is an intersection based on sensor information, and the sensors may include GNSS sensors, cameras, and the like, and can be implemented in the manner based on maps described above. If the intersection is the intersection, whether the intersection is provided with a fixed traffic signal lamp can be further judged; if the fixed traffic signal lamp is not arranged at the road junction, a temporary traffic signal lamp possibly exists at the time to replace the fixed traffic signal lamp as a temporary traffic facility, so that the detection condition of the temporary traffic signal lamp is met, and the detection flow of whether the temporary traffic signal lamp exists can be started; if the detection result shows that the fixed traffic signal lamp is arranged at the road intersection, whether the fixed traffic signal lamp has a fault or not needs to be determined, if the fixed traffic signal lamp has the fault, a temporary traffic signal lamp possibly exists at the moment and serves as a temporary substitute traffic facility, and a detection flow of whether the temporary traffic signal lamp exists or not can be started. Therefore, the detection conditions of the corresponding temporary traffic signal lamps are determined according to different conditions of the road junctions.
For example, for an intersection where a fixed traffic signal exists, multiple frames of images can be continuously shot, and whether the fixed traffic signal fails is determined by identifying whether bright red, green, and yellow bulbs exist, whether the color of the bulbs changes, and the like.
In one embodiment of the present application, in the above method, the image is collected by a camera of the automatic driving device, and the three-dimensional point cloud is collected by a laser radar of the automatic driving device; the camera and the laser radar are calibrated in advance.
The image may be collected by a camera of the autonomous device and the three-dimensional point cloud may be collected by a lidar of the autonomous device. In order to accurately ensure the corresponding relationship between the image data and the three-dimensional point cloud data and facilitate subsequent data mapping processing, calibration processing can be performed on the camera and the laser radar in advance. The parameters to be calibrated may include: the method comprises the following steps of measuring parameters such as camera internal parameter, camera-to-laser radar external parameter, and laser radar-to-IMU external parameter, wherein an IMU (inertial measurement unit) is a device for measuring the three-axis attitude angle and acceleration of an object. Therefore, the calibration processing of the camera and the laser radar is realized, so that the image data and the three-dimensional point cloud data with strict corresponding relation can be obtained in the subsequent acquisition, and the specific projection can be realized by matrix transformation according to the matrix obtained by calibration.
In one embodiment of the present application, the method, wherein identifying the temporary traffic signal from the image, comprises: identifying a temporary traffic signal from the image using a traffic signal detection model comprising a bezel detection network, a bulb detection network, and an integral detection network, specifically: identifying a plurality of lamp frames from the image by using a lamp frame detection network, identifying a plurality of light-emitting bulbs from the image by using a bulb detection network, and identifying a plurality of temporary traffic signal lamp integrals from the image by using an integral detection network; matching the identified lamp frame with the identified light-emitting lamp bulb to obtain the lamp frame containing the light-emitting lamp bulb; and matching the lamp frame containing the light-emitting bulb with the temporary traffic signal lamp integrally to obtain the temporary traffic signal lamp.
In order to quickly and efficiently identify the temporary traffic signal, the temporary traffic signal can be automatically identified from the image by using a traffic signal detection model. The traffic signal lamp detection model can be obtained by training based on a neural network, and the neural network can be obtained by adjusting and training based on R-CNN, Fast R-CNN and the like.
Fig. 5 shows a schematic structural diagram of a temporary traffic signal, and as shown in fig. 5, the temporary traffic signal is mainly composed of a solar panel 510, a lamp frame 520, a bulb 530, a support 540 and a mobile device 550. The light frame 520 surrounds the light bulb 530, and the light bulb 530 cannot show a specific color due to the form of the image, but generally, the light bulb 530 can show color signals of red light, yellow light, green light, and the like, and can also show indication signals with indication shapes such as left turn, straight going, and the like. When the temporary traffic signal is identified as a whole, the characteristics of the solar panel 510, the lamp frame 520, the bulb 530, the support 540, the mobile device 550 and other components are used as the characteristics for identifying the whole temporary traffic signal, so that the characteristics are identified.
Of course, the temporary traffic light shown in fig. 5 is only an example, the temporary traffic light may not include a solar panel (for example, power is supplied by a storage battery), but the appearances of different types of temporary traffic lights still have commonality, and the temporary traffic light may also be implemented by training a traffic light detection model, and only sufficient sample data needs to be obtained for training.
In order to accurately identify and detect the temporary traffic signal according to the traffic signal detection model, a lamp frame detection network, a bulb detection network and an overall detection network can be arranged in the traffic signal detection model. The lamp frame detection network can identify a plurality of lamp frames from the image; a bulb detection network capable of identifying a plurality of light emitting bulbs from the image; the integral detection network can identify a plurality of temporary traffic signal lamp integrals from the images.
After the lamp frame detection network, the bulb detection network and the overall detection network respectively realize the identification of the detection object, the lamp frame, the bulb and the overall detection network can be matched, so that the complete temporary traffic signal lamp is obtained. Specifically, the identified lamp frame and the identified light-emitting bulb can be used for matching, and the lamp frame comprising a plurality of light-emitting bulbs can be obtained; the temporary traffic signal lamp can be obtained by matching the lamp frame containing the light-emitting lamp bulb with the temporary traffic signal lamp as a whole, and the matching correspondence of the color state, the shape of the lamp bulb and the position of the temporary traffic signal lamp in an image coordinate system can be realized. Therefore, the respective identification detection and final matching of the lamp frame, the lamp bulb and the whole body by using the traffic signal lamp detection model are realized, and the temporary traffic signal lamp whole body is identified and detected.
In an embodiment of the application, the method wherein the traffic signal detection model further includes an indication shape detection network, and identifying a number of light-emitting bulbs from the image using the bulb detection network includes: identifying the identified light-emitting bulb by using an indication shape detection network, and determining the indication shape of the light-emitting bulb; the indicating shape includes at least one of: straight, left turn, right turn.
Traffic signal lights are often provided with various indicating shapes to provide richer traffic signal information for better and orderly guiding traffic operation. Therefore, an indication shape detection network may be provided in the traffic light detection model, the temporary traffic light bulb image area may be input to the shape detection network, the identified light-emitting bulb may be identified by the indication shape detection network, and the indication shape of the light-emitting bulb may be determined, which may include straight movement, left turn, right turn, etc., for example, the indication shape shown in fig. 5 includes straight movement and left turn. Therefore, the indication shape of the light-emitting bulb can be accurately determined by the indication shape detection network, and the indication information can be identified.
In one embodiment of the present application, in the method, projecting a three-dimensional point cloud to an image coordinate system, and determining the point cloud matched with the temporary traffic signal lamp includes: projecting the three-dimensional point cloud to an image coordinate system to obtain a point cloud to be filtered corresponding to the temporary traffic signal lamp; and acquiring map data corresponding to the current position, filtering the point cloud to be filtered by using the map data, and taking the filtered point cloud as the point cloud matched with the temporary traffic signal lamp.
When the three-dimensional point cloud is projected to the image coordinate system, the point cloud data of the temporary traffic signal lamp can be obtained, and one-dimensional information is lost in the process of projecting and converting the world coordinate system, so that the point cloud data obtained at the moment is not all points hitting the temporary traffic signal lamp. In addition, because the environment at the road junction is complex and there may be interference of various objects, the point cloud data may be filtered to reduce the interference of irrelevant data. Specifically, the map data corresponding to the current position can be acquired, the intersection structure is utilized to filter the three-dimensional point cloud, the points in the non-intersection are filtered, and the filtered point cloud is used as the point cloud matched with the temporary traffic signal lamp, so that the data interference is reduced. Therefore, the point cloud to be filtered can be filtered by utilizing the map data, and the aims of optimizing the data and reducing the interference are fulfilled. Preferably, the high-precision features of the high-precision map can be used for filtering to improve accuracy.
Fig. 2 is a schematic structural diagram illustrating a temporary traffic signal detecting apparatus according to an embodiment of the present application, and as shown in fig. 2, the temporary traffic signal detecting apparatus 200 includes:
the obtaining unit 210 obtains a current image and a corresponding three-dimensional point cloud. For example, images and three-dimensional point clouds may be acquired by sensors of the autopilot device.
The determining unit 220 is configured to identify the temporary traffic light from the image, and determine a state of the temporary traffic light and a position of the temporary traffic light in the image coordinate system.
The temporary traffic signal lamp can be identified from the image according to the characteristics of the temporary traffic signal lamp, the state of the temporary traffic signal lamp can be further determined, and the state information can comprise the lighting condition of the traffic signal lamp, the indication shape type and other information. And establishing an image coordinate system and determining the position of the temporary traffic signal lamp in the image coordinate system. Thus, the information analysis of the temporary traffic signal lamp in the image is realized.
And the matching unit 230 is used for projecting the three-dimensional point cloud to an image coordinate system and determining the point cloud matched with the temporary traffic signal lamp.
The camera and the laser radar can be calibrated in advance, so that the three-dimensional point cloud data and the image data have a one-to-one correspondence relationship, the three-dimensional point cloud is conveniently projected onto an image coordinate system according to the corresponding mapping relationship subsequently, and the point cloud matched with the temporary traffic signal lamp can be obtained. Therefore, the point cloud data matched with the temporary traffic signal lamp can be obtained according to the image data.
And the position unit 240 is used for determining the position of the temporary traffic signal lamp in the world coordinate system according to the point cloud matched with the temporary traffic signal lamp.
The point cloud data matched with the temporary traffic signal lamp can reflect the spatial position relation of the temporary traffic signal lamp, in order to obtain accurate spatial position information of the temporary traffic signal lamp, the point cloud data matched with the temporary traffic signal lamp can be analyzed, a world coordinate system can be established, and coordinate information of the temporary traffic signal lamp in the world coordinate system can be determined in the world coordinate system. Therefore, the spatial position can be accurately determined according to the point cloud data matched with the temporary traffic signal lamp and is represented by the position information in the world coordinate system.
Therefore, the device shown in fig. 2 can automatically identify and detect the temporary traffic signal lamp based on the existing road infrastructure conditions on the basis of not improving the road infrastructure, accurately determine the spatial position of the temporary traffic signal lamp, has a good auxiliary effect on road driving, particularly automatic driving of unmanned equipment, effectively improves the driving safety and the automation degree, and can be further applied to the fields of logistics, takeaway distribution and the like.
In one embodiment of the present application, in the above apparatus, the temporary traffic light detecting means is operated in a case where a temporary traffic light detecting condition is satisfied.
The fixed traffic signal lamp belongs to traffic infrastructure and is usually arranged at positions such as a traffic road intersection, while the temporary traffic signal lamp is not usually arranged at the traffic road intersection, and the temporary traffic signal lamp is generally arranged under specific conditions that the fixed traffic signal lamp breaks down or the road is a temporary road and the like. Therefore, when the temporary traffic signal lamp is detected, the detection condition of the temporary traffic signal lamp can be preset, and when the detection condition of the temporary traffic signal lamp is met, the detection process of the temporary traffic signal lamp is started again, the current image and the corresponding three-dimensional point cloud are obtained, and the subsequent steps are executed.
Certainly, in practical application, the current image and the corresponding three-dimensional point cloud not only can be used for identifying the temporary traffic signal lamp, but also can be used for identifying information such as obstacles, for example, a temporary traffic signal lamp identification sub-module and an obstacle sub-identification module are arranged and both belong to the identification module, the identification module acquires the image and the three-dimensional point cloud from the vehicle-mounted camera and the vehicle-mounted laser radar at a preset frequency, and the image and the three-dimensional point cloud are redistributed to the corresponding sub-modules to perform corresponding identification or detection processes.
In one embodiment of the present application, in the above apparatus, the case where the temporary traffic signal detection condition is satisfied includes at least one of: the current position is an intersection, and the intersection is not provided with a fixed traffic signal lamp; the current position is a crossing, and the crossing is provided with a fixed traffic signal lamp, but the fixed traffic signal lamp has a fault.
Since both the fixed traffic lights and the temporary traffic lights are generally disposed at intersections of roads, it can be determined whether the current position is an intersection based on sensor information, and the sensors may include GNSS sensors, cameras, and the like, and can be implemented in the manner based on maps described above. If the intersection is the intersection, whether the intersection is provided with a fixed traffic signal lamp can be further judged; if the fixed traffic signal lamp is not arranged at the road junction, a temporary traffic signal lamp possibly exists at the time to replace the fixed traffic signal lamp as a temporary traffic facility, so that the detection condition of the temporary traffic signal lamp is met, and the detection flow of whether the temporary traffic signal lamp exists can be started; if the detection result shows that the fixed traffic signal lamp is arranged at the road intersection, whether the fixed traffic signal lamp has a fault or not needs to be determined, if the fixed traffic signal lamp has the fault, a temporary traffic signal lamp possibly exists at the moment and serves as a temporary substitute traffic facility, and a detection flow of whether the temporary traffic signal lamp exists or not can be started. Therefore, the detection conditions of the corresponding temporary traffic signal lamps are determined according to different conditions of the road junctions.
For example, for an intersection where a fixed traffic signal exists, multiple frames of images can be continuously shot, and whether the fixed traffic signal fails is determined by identifying whether bright red, green, and yellow bulbs exist, whether the color of the bulbs changes, and the like.
In one embodiment of the application, in the device, the image is collected by a camera of the automatic driving equipment, and the three-dimensional point cloud is collected by a laser radar of the automatic driving equipment; the camera and the laser radar are calibrated in advance.
The image may be collected by a camera of the autonomous device and the three-dimensional point cloud may be collected by a lidar of the autonomous device. In order to accurately ensure the corresponding relationship between the image data and the three-dimensional point cloud data and facilitate subsequent data mapping processing, calibration processing can be performed on the camera and the laser radar in advance. The parameters to be calibrated may include: the method comprises the following steps of measuring parameters such as camera internal parameter, camera-to-laser radar external parameter, and laser radar-to-IMU external parameter, wherein an IMU (inertial measurement unit) is a device for measuring the three-axis attitude angle and acceleration of an object. Therefore, the calibration processing of the camera and the laser radar is realized, so that the image data and the three-dimensional point cloud data with strict corresponding relation can be obtained in the subsequent acquisition, and the specific projection can be realized by matrix transformation according to the matrix obtained by calibration.
In an embodiment of the present application, in the above apparatus, the determining unit 220 is configured to identify the temporary traffic signal from the image by using a traffic signal detection model including a lamp frame detection network, a bulb detection network, and an overall detection network, specifically: identifying a plurality of lamp frames from the image by using a lamp frame detection network, identifying a plurality of light-emitting bulbs from the image by using a bulb detection network, and identifying a plurality of temporary traffic signal lamp integrals from the image by using an integral detection network; matching the identified lamp frame with the identified light-emitting lamp bulb to obtain the lamp frame containing the light-emitting lamp bulb; and matching the lamp frame containing the light-emitting bulb with the temporary traffic signal lamp integrally to obtain the temporary traffic signal lamp.
In order to quickly and efficiently identify the temporary traffic signal, the temporary traffic signal can be automatically identified from the image by using a traffic signal detection model. The traffic signal lamp detection model can be obtained by training based on a neural network, and the neural network can be obtained by adjusting and training based on R-CNN, Fast R-CNN and the like.
Fig. 5 shows a schematic structural diagram of a temporary traffic signal, and as shown in fig. 5, the temporary traffic signal is mainly composed of a solar panel 510, a lamp frame 520, a bulb 530, a support 540 and a mobile device 550. The light frame 520 surrounds the light bulb 530, and the light bulb 530 cannot show a specific color due to the form of the image, but generally, the light bulb 530 can show color signals of red light, yellow light, green light, and the like, and can also show indication signals with indication shapes such as left turn, straight going, and the like. When the temporary traffic signal is identified as a whole, the characteristics of the solar panel 510, the lamp frame 520, the bulb 530, the support 540, the mobile device 550 and other components are used as the characteristics for identifying the whole temporary traffic signal, so that the characteristics are identified.
Of course, the temporary traffic light shown in fig. 5 is only an example, the temporary traffic light may not include a solar panel (for example, power is supplied by a storage battery), but the appearances of different types of temporary traffic lights still have commonality, and the temporary traffic light may also be implemented by training a traffic light detection model, and only sufficient sample data needs to be obtained for training.
In order to accurately identify and detect the temporary traffic signal according to the traffic signal detection model, a lamp frame detection network, a bulb detection network and an overall detection network can be arranged in the traffic signal detection model. The lamp frame detection network can identify a plurality of lamp frames from the image; a bulb detection network capable of identifying a plurality of light emitting bulbs from the image; the integral detection network can identify a plurality of temporary traffic signal lamp integrals from the images.
After the lamp frame detection network, the bulb detection network and the overall detection network respectively realize the identification of the detection object, the lamp frame, the bulb and the overall detection network can be matched, so that the complete temporary traffic signal lamp is obtained. Specifically, the identified lamp frame and the identified light-emitting bulb can be used for matching, and the lamp frame comprising a plurality of light-emitting bulbs can be obtained; the temporary traffic signal lamp can be obtained by matching the lamp frame containing the light-emitting lamp bulb with the temporary traffic signal lamp as a whole, and the matching correspondence of the color state, the shape of the lamp bulb and the position of the temporary traffic signal lamp in an image coordinate system can be realized. Therefore, the respective identification detection and final matching of the lamp frame, the lamp bulb and the whole body by using the traffic signal lamp detection model are realized, and the temporary traffic signal lamp whole body is identified and detected.
In an embodiment of the present application, in the above apparatus, the traffic signal detection model further includes an indication shape detection network, and the determining unit 220 is configured to identify the identified light-emitting bulb by using the indication shape detection network, and determine the indication shape of the light-emitting bulb; the indicating shape includes at least one of: straight, left turn, right turn.
Traffic signal lights are often provided with various indicating shapes to provide richer traffic signal information for better and orderly guiding traffic operation. Therefore, an indication shape detection network may be provided in the traffic light detection model, the temporary traffic light bulb image area may be input to the shape detection network, the identified light-emitting bulb may be identified by the indication shape detection network, and the indication shape of the light-emitting bulb may be determined, which may include straight movement, left turn, right turn, etc., for example, the indication shape shown in fig. 5 includes straight movement and left turn. Therefore, the indication shape of the light-emitting bulb can be accurately determined by the indication shape detection network, and the indication information can be identified.
In an embodiment of the present application, in the above apparatus, the matching unit 230 is configured to project the three-dimensional point cloud to an image coordinate system, so as to obtain a point cloud to be filtered corresponding to the temporary traffic signal lamp; and acquiring map data corresponding to the current position, filtering the point cloud to be filtered by using the map data, and taking the filtered point cloud as the point cloud matched with the temporary traffic signal lamp.
When the three-dimensional point cloud is projected to the image coordinate system, the point cloud data of the temporary traffic signal lamp can be obtained, and one-dimensional information is lost in the process of projecting and converting the world coordinate system, so that the point cloud data obtained at the moment is not all points hitting the temporary traffic signal lamp. In addition, because the environment at the road junction is complex and there may be interference of various objects, the point cloud data may be filtered to reduce the interference of irrelevant data. Specifically, the map data corresponding to the current position can be acquired, the intersection structure is utilized to filter the three-dimensional point cloud, the points in the non-intersection are filtered, and the filtered point cloud is used as the point cloud matched with the temporary traffic signal lamp, so that the data interference is reduced. Therefore, the point cloud to be filtered can be filtered by utilizing the map data, and the aims of optimizing the data and reducing the interference are fulfilled. Preferably, the high-precision features of the high-precision map can be used for filtering to improve accuracy.
In summary, according to the technical scheme of the application, the current image and the corresponding three-dimensional point cloud are obtained; identifying a temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in an image coordinate system; projecting the three-dimensional point cloud to the image coordinate system, and determining the point cloud matched with the temporary traffic signal lamp; and determining the position of the temporary traffic signal lamp in a world coordinate system according to the point cloud matched with the temporary traffic signal lamp. The traffic signal lamp detection method has the advantages that on the basis that road facilities are not required to be improved, based on the existing road infrastructure conditions, the temporary traffic signal lamp can be automatically identified and detected, the spatial position of the temporary traffic signal lamp is accurately determined, a good auxiliary effect is achieved on road driving, particularly automatic driving of unmanned equipment, and driving safety and automation degree are effectively improved.
It should be noted that:
the algorithms and displays presented herein are not inherently related to any particular computer, virtual machine, or other apparatus. Various general purpose devices may be used with the teachings herein. The required structure for constructing such a device will be apparent from the description above. In addition, this application is not directed to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the present application as described herein, and any descriptions of specific languages are provided above to disclose the best modes of the present application.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the application, various features of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the application and aiding in the understanding of one or more of the various inventive aspects. However, the disclosed method should not be interpreted as reflecting an intention that: this application is intended to cover such departures from the present disclosure as come within known or customary practice in the art to which this invention pertains. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this application.
Those skilled in the art will appreciate that the modules in the device in an embodiment may be adaptively changed and disposed in one or more devices different from the embodiment. The modules or units or components of the embodiments may be combined into one module or unit or component, and furthermore they may be divided into a plurality of sub-modules or sub-units or sub-components. All of the features disclosed in this specification (including any accompanying claims, abstract and drawings), and all of the processes or elements of any method or apparatus so disclosed, may be combined in any combination, except combinations where at least some of such features and/or processes or elements are mutually exclusive. Each feature disclosed in this specification (including any accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features included in other embodiments, rather than other features, combinations of features of different embodiments are meant to be within the scope of the application and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.
The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the temporary traffic signal detection apparatus according to embodiments of the present application. The present application may also be embodied as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
For example, fig. 3 shows a schematic structural diagram of an autopilot device according to an embodiment of the application. The autopilot device 300 includes a processor 310 and a memory 320 arranged to store computer executable instructions (computer readable program code). The memory 320 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. The memory 320 has a storage space 330 storing computer readable program code 331 for performing any of the method steps described above. For example, the storage space 330 for storing the computer readable program code may comprise respective computer readable program codes 331 for respectively implementing various steps in the above method. The computer readable program code 331 may be read from or written to one or more computer program products. These computer program products comprise a program code carrier such as a hard disk, a Compact Disc (CD), a memory card or a floppy disk. Such a computer program product is typically a computer readable storage medium such as described in fig. 4. FIG. 4 shows a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present application. The computer readable storage medium 400 has stored thereon computer readable program code 331 for performing the steps of the method according to the application, readable by a processor 310 of the autopilot device 300, which computer readable program code 331, when executed by the autopilot device 300, causes the autopilot device 300 to perform the steps of the method described above, in particular the computer readable program code 331 stored thereon may perform the method shown in any of the embodiments described above. The computer readable program code 331 may be compressed in a suitable form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
Claims (10)
1. A temporary traffic signal detection method, comprising:
acquiring a current image and a corresponding three-dimensional point cloud;
identifying a temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in an image coordinate system;
projecting the three-dimensional point cloud to the image coordinate system, and determining the point cloud matched with the temporary traffic signal lamp;
and determining the position of the temporary traffic signal lamp in a world coordinate system according to the point cloud matched with the temporary traffic signal lamp.
2. The method of claim 1, wherein the method is performed in a situation where a temporary traffic signal detection condition is satisfied.
3. The method of claim 2, wherein the condition that the temporary traffic signal detection condition is satisfied comprises at least one of: the current position is an intersection, and the intersection is not provided with a fixed traffic signal lamp; the current position is a crossing, and the crossing is provided with a fixed traffic signal lamp, but the fixed traffic signal lamp has a fault.
4. The method of claim 1, wherein the image is acquired by a camera of an autonomous device, the three-dimensional point cloud is acquired by a lidar of the autonomous device;
the camera and the laser radar are subjected to pre-calibration treatment.
5. The method of claim 1, wherein said identifying a temporary traffic signal from said image comprises identifying a temporary traffic signal from said image using a traffic signal detection model comprising a bezel detection network, a bulb detection network, and an ensemble detection network, in particular:
identifying a plurality of lamp frames from the image by using a lamp frame detection network, identifying a plurality of light-emitting lamps from the image by using a lamp bulb detection network, and identifying a plurality of temporary traffic signal lamp integrals from the image by using the integral detection network;
matching the identified lamp frame with the identified light-emitting lamp bulb to obtain the lamp frame containing the light-emitting lamp bulb;
and matching the lamp frame containing the light-emitting bulb with the temporary traffic signal lamp integrally to obtain the temporary traffic signal lamp.
6. The method of claim 5, wherein the traffic signal detection model further comprises an indication shape detection network, the identifying a number of light emitting bulbs from the image using the bulb detection network comprising:
identifying the identified light-emitting bulb by using an indication shape detection network, and determining the indication shape of the light-emitting bulb; the indicating shape includes at least one of: straight, left turn, right turn.
7. The method of any one of claims 1-6, wherein the projecting the three-dimensional point cloud to the image coordinate system, the determining the point cloud that matches the temporary traffic signal comprises:
projecting the three-dimensional point cloud to the image coordinate system to obtain a point cloud to be filtered corresponding to the temporary traffic signal lamp;
and acquiring map data corresponding to the current position, filtering the point cloud to be filtered by using the map data, and taking the filtered point cloud as the point cloud matched with the temporary traffic signal lamp.
8. A temporary traffic signal detection device, comprising:
the acquisition unit is used for acquiring a current image and a corresponding three-dimensional point cloud;
the determining unit is used for identifying a temporary traffic signal lamp from the image, and determining the state of the temporary traffic signal lamp and the position of the temporary traffic signal lamp in an image coordinate system;
the matching unit is used for projecting the three-dimensional point cloud to the image coordinate system and determining the point cloud matched with the temporary traffic signal lamp;
and the position unit is used for determining the position of the temporary traffic signal lamp in the world coordinate system according to the point cloud matched with the temporary traffic signal lamp.
9. An autopilot device, wherein the autopilot device comprises: a processor; and a memory arranged to store computer-executable instructions that, when executed, cause the processor to perform the method of any one of claims 1-7.
10. A computer readable storage medium, wherein the computer readable storage medium stores one or more programs which, when executed by a processor, implement the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911342645.2A CN111079680B (en) | 2019-12-23 | 2019-12-23 | Temporary traffic signal lamp detection method and device and automatic driving equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911342645.2A CN111079680B (en) | 2019-12-23 | 2019-12-23 | Temporary traffic signal lamp detection method and device and automatic driving equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111079680A true CN111079680A (en) | 2020-04-28 |
| CN111079680B CN111079680B (en) | 2023-09-19 |
Family
ID=70317011
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911342645.2A Active CN111079680B (en) | 2019-12-23 | 2019-12-23 | Temporary traffic signal lamp detection method and device and automatic driving equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111079680B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111661054A (en) * | 2020-05-08 | 2020-09-15 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle control method, device, electronic device and storage medium |
| CN111882899A (en) * | 2020-07-24 | 2020-11-03 | 北京小马慧行科技有限公司 | Method, device and system for determining position of traffic signal lamp |
| CN112101209A (en) * | 2020-09-15 | 2020-12-18 | 北京百度网讯科技有限公司 | Method and apparatus for determining a world coordinate point cloud for roadside computing devices |
| CN112580489A (en) * | 2020-12-15 | 2021-03-30 | 深兰人工智能(深圳)有限公司 | Traffic light detection method and device, electronic equipment and storage medium |
| CN112735253A (en) * | 2020-10-22 | 2021-04-30 | 深圳裹动智驾科技有限公司 | Traffic light automatic labeling method and computer equipment |
| CN113096395A (en) * | 2021-03-31 | 2021-07-09 | 武汉理工大学 | Road traffic safety evaluation system based on positioning and artificial intelligence recognition |
| CN114594600A (en) * | 2020-12-03 | 2022-06-07 | 中移(成都)信息通信科技有限公司 | Near-eye display system, fixing device, signal processing method, signal processing apparatus, and signal processing medium |
| CN114782924A (en) * | 2022-05-10 | 2022-07-22 | 智道网联科技(北京)有限公司 | Traffic light detection method and device for automatic driving and electronic equipment |
| CN114782924B (en) * | 2022-05-10 | 2024-07-12 | 智道网联科技(北京)有限公司 | Traffic light detection method and device for automatic driving and electronic equipment |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012043260A (en) * | 2010-08-20 | 2012-03-01 | Sanyo Electric Co Ltd | Mobile communication device |
| CN103337190A (en) * | 2013-06-11 | 2013-10-02 | 西安费斯达自动化工程有限公司 | Intelligent detection and alarming method for failure of traffic light |
| CN103489324A (en) * | 2013-09-22 | 2014-01-01 | 北京联合大学 | Real-time dynamic traffic light detection identification method based on unmanned driving |
| CN104411559A (en) * | 2012-03-26 | 2015-03-11 | 谷歌公司 | A robust method for detecting traffic signals and their associated states |
| KR101532420B1 (en) * | 2014-10-29 | 2015-06-29 | (주)씨앤에스아이 | Automated detection system of failed LED traffic lights |
| US20160035223A1 (en) * | 2014-07-31 | 2016-02-04 | Google Inc. | Traffic Signal Response for Autonomous Vehicles |
| CN106932780A (en) * | 2017-03-14 | 2017-07-07 | 北京京东尚科信息技术有限公司 | Object positioning method, device and system |
| US20170214843A1 (en) * | 2016-01-22 | 2017-07-27 | Coban Technologies, Inc. | Systems, apparatuses, and methods for controlling audiovisual apparatuses |
| CN107507432A (en) * | 2017-09-25 | 2017-12-22 | 南京律智诚专利技术开发有限公司 | A kind of big data analyzes dynamic traffic signal system |
| CN108734979A (en) * | 2017-04-20 | 2018-11-02 | 通用汽车环球科技运作有限责任公司 | Traffic lights detecting system and method |
| CN109035831A (en) * | 2018-09-07 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | Recognition methods, device, equipment, storage medium and the vehicle of traffic light |
| CN109583415A (en) * | 2018-12-11 | 2019-04-05 | 兰州大学 | A kind of traffic lights detection and recognition methods merged based on laser radar with video camera |
| CN109635782A (en) * | 2018-12-31 | 2019-04-16 | 天合光能股份有限公司 | A method of obtaining unmanned required static traffic information |
| JP2019120963A (en) * | 2017-12-28 | 2019-07-22 | 株式会社デンソー | Control device for vehicle and control method |
| CN110047285A (en) * | 2019-04-18 | 2019-07-23 | 河南勇威特种设备有限公司 | A kind of traffic signal lamp detector and detection method |
-
2019
- 2019-12-23 CN CN201911342645.2A patent/CN111079680B/en active Active
Patent Citations (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2012043260A (en) * | 2010-08-20 | 2012-03-01 | Sanyo Electric Co Ltd | Mobile communication device |
| CN104411559A (en) * | 2012-03-26 | 2015-03-11 | 谷歌公司 | A robust method for detecting traffic signals and their associated states |
| CN103337190A (en) * | 2013-06-11 | 2013-10-02 | 西安费斯达自动化工程有限公司 | Intelligent detection and alarming method for failure of traffic light |
| CN103489324A (en) * | 2013-09-22 | 2014-01-01 | 北京联合大学 | Real-time dynamic traffic light detection identification method based on unmanned driving |
| US20160035223A1 (en) * | 2014-07-31 | 2016-02-04 | Google Inc. | Traffic Signal Response for Autonomous Vehicles |
| CN106537271A (en) * | 2014-07-31 | 2017-03-22 | 谷歌公司 | Traffic signal response for autonomous vehicles |
| KR101532420B1 (en) * | 2014-10-29 | 2015-06-29 | (주)씨앤에스아이 | Automated detection system of failed LED traffic lights |
| US20170214843A1 (en) * | 2016-01-22 | 2017-07-27 | Coban Technologies, Inc. | Systems, apparatuses, and methods for controlling audiovisual apparatuses |
| CN106932780A (en) * | 2017-03-14 | 2017-07-07 | 北京京东尚科信息技术有限公司 | Object positioning method, device and system |
| CN108734979A (en) * | 2017-04-20 | 2018-11-02 | 通用汽车环球科技运作有限责任公司 | Traffic lights detecting system and method |
| CN107507432A (en) * | 2017-09-25 | 2017-12-22 | 南京律智诚专利技术开发有限公司 | A kind of big data analyzes dynamic traffic signal system |
| JP2019120963A (en) * | 2017-12-28 | 2019-07-22 | 株式会社デンソー | Control device for vehicle and control method |
| CN109035831A (en) * | 2018-09-07 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | Recognition methods, device, equipment, storage medium and the vehicle of traffic light |
| CN109583415A (en) * | 2018-12-11 | 2019-04-05 | 兰州大学 | A kind of traffic lights detection and recognition methods merged based on laser radar with video camera |
| CN109635782A (en) * | 2018-12-31 | 2019-04-16 | 天合光能股份有限公司 | A method of obtaining unmanned required static traffic information |
| CN110047285A (en) * | 2019-04-18 | 2019-07-23 | 河南勇威特种设备有限公司 | A kind of traffic signal lamp detector and detection method |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111661054A (en) * | 2020-05-08 | 2020-09-15 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle control method, device, electronic device and storage medium |
| CN111661054B (en) * | 2020-05-08 | 2022-03-04 | 东软睿驰汽车技术（沈阳）有限公司 | Vehicle control method, device, electronic device and storage medium |
| CN111882899A (en) * | 2020-07-24 | 2020-11-03 | 北京小马慧行科技有限公司 | Method, device and system for determining position of traffic signal lamp |
| CN111882899B (en) * | 2020-07-24 | 2021-10-22 | 北京小马慧行科技有限公司 | Method, device and system for determining position of traffic signal lamp |
| CN112101209A (en) * | 2020-09-15 | 2020-12-18 | 北京百度网讯科技有限公司 | Method and apparatus for determining a world coordinate point cloud for roadside computing devices |
| CN112101209B (en) * | 2020-09-15 | 2024-04-09 | 阿波罗智联(北京)科技有限公司 | Method and apparatus for determining world coordinate point cloud for roadside computing device |
| CN112735253A (en) * | 2020-10-22 | 2021-04-30 | 深圳裹动智驾科技有限公司 | Traffic light automatic labeling method and computer equipment |
| CN112735253B (en) * | 2020-10-22 | 2023-03-10 | 深圳安途智行科技有限公司 | Traffic light automatic labeling method and computer equipment |
| CN114594600A (en) * | 2020-12-03 | 2022-06-07 | 中移(成都)信息通信科技有限公司 | Near-eye display system, fixing device, signal processing method, signal processing apparatus, and signal processing medium |
| CN114594600B (en) * | 2020-12-03 | 2023-08-15 | 中移(成都)信息通信科技有限公司 | Near-eye display system, fixing device, signal processing method, device and medium thereof |
| CN112580489A (en) * | 2020-12-15 | 2021-03-30 | 深兰人工智能(深圳)有限公司 | Traffic light detection method and device, electronic equipment and storage medium |
| CN113096395B (en) * | 2021-03-31 | 2022-03-25 | 武汉理工大学 | Road traffic safety evaluation system based on positioning and artificial intelligence recognition |
| CN113096395A (en) * | 2021-03-31 | 2021-07-09 | 武汉理工大学 | Road traffic safety evaluation system based on positioning and artificial intelligence recognition |
| CN114782924A (en) * | 2022-05-10 | 2022-07-22 | 智道网联科技(北京)有限公司 | Traffic light detection method and device for automatic driving and electronic equipment |
| CN114782924B (en) * | 2022-05-10 | 2024-07-12 | 智道网联科技(北京)有限公司 | Traffic light detection method and device for automatic driving and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111079680B (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111079680B (en) | Temporary traffic signal lamp detection method and device and automatic driving equipment | |
| CN112069856B (en) | Map generation method, driving control device, electronic equipment and system | |
| CN108305475B (en) | Traffic light identification method and device | |
| CN112991791B (en) | Traffic information identification and intelligent driving method, device, equipment and storage medium | |
| JP6781711B2 (en) | Methods and systems for automatically recognizing parking zones | |
| CN110796007B (en) | Scene recognition method and computing device | |
| CN109767637A (en) | The method and apparatus of the identification of countdown signal lamp and processing | |
| CN112667837A (en) | Automatic image data labeling method and device | |
| CN106023622B (en) | A kind of method and apparatus of determining traffic lights identifying system recognition performance | |
| CN114332815B (en) | Traffic light state detection method and device, vehicle and storage medium | |
| WO2020007589A1 (en) | Training a deep convolutional neural network for individual routes | |
| US20220410900A1 (en) | Assertive vehicle detection model generation | |
| CN114764876A (en) | Evaluation method and evaluation device for perception fusion algorithm | |
| CN110083099A (en) | One kind meeting automobile function safety standard automatic Pilot architecture system and working method | |
| US20210300356A1 (en) | Vehicle uncertainty sharing | |
| CN115294544A (en) | Driving scene classification method, device, equipment and storage medium | |
| JP2021128705A (en) | Object state identification device | |
| CN115965682B (en) | Vehicle passable area determining method and device and computer equipment | |
| CN111126154A (en) | Method and device for identifying road surface element, unmanned equipment and storage medium | |
| CN114252859A (en) | Target area determination method and device, computer equipment and storage medium | |
| AU2020254128A1 (en) | System and method for determining location and orientation of an object in a space | |
| US20220221585A1 (en) | Systems and methods for monitoring lidar sensor health | |
| CN112880692A (en) | Map data annotation method and device and storage medium | |
| CN111661054B (en) | Vehicle control method, device, electronic device and storage medium | |
| CN113048988B (en) | Method and device for detecting change elements of scene corresponding to navigation map |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |