WO2024239605A1 - 自动驾驶系统的攻击检测方法、装置、设备及存储介质 - Google Patents

自动驾驶系统的攻击检测方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2024239605A1
WO2024239605A1 PCT/CN2023/137634 CN2023137634W WO2024239605A1 WO 2024239605 A1 WO2024239605 A1 WO 2024239605A1 CN 2023137634 W CN2023137634 W CN 2023137634W WO 2024239605 A1 WO2024239605 A1 WO 2024239605A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
detection frame
target
array
information data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/137634
Other languages
English (en)
French (fr)
Inventor
缪祖佳
邵翠萍
李慧云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2024239605A1 publication Critical patent/WO2024239605A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present application relates to the field of sensor detection technology, and in particular to an attack detection method, device, equipment and storage medium for an autonomous driving system.
  • Unmanned driving technology is a comprehensive technology involving many cutting-edge technologies such as artificial intelligence, sensor technology, map technology, and computers.
  • the function realization of unmanned driving technology depends on the vehicle positioning sensor system.
  • the positioning sensor system is mainly composed of laser radar, visual camera, millimeter wave radar, global positioning system (GPS) and other devices.
  • GPS global positioning system
  • the positioning sensor system provides rich positioning data for the planning and decision-making modules of unmanned vehicles, namely information such as vehicle speed, posture and position.
  • the safety of route planning and decision-making control of autonomous driving vehicles is based on the safety of the positioning sensor system. If the positioning sensor system of the autonomous driving vehicle is abnormal, it will cause the sensor to obtain incorrect positioning information, and then plan the wrong driving control strategy, posing a threat to the safety of other vehicles and the lives of drivers and pedestrians.
  • the sensor attack detection method of unmanned driving technology is mainly combined with a specific single-sensor target detection algorithm, and attack detection is performed on this basis.
  • this method is not universal.
  • the attack detection method combined with the target detection algorithm based on lidar cannot be applied to the attack detection scenario of the target detection algorithm based on camera, and the attack detection results of the detection method for a single type of sensor are not accurate.
  • the present application provides an attack detection method, device, equipment and storage medium for an autonomous driving system to solve the problem that the existing attack detection methods for autonomous driving systems are not universal and have low accuracy.
  • a technical solution adopted in the present application is: to provide an attack detection method for an autonomous driving system, comprising: respectively obtaining three-dimensional information data and two-dimensional information data of a target area using a three-dimensional sensor and a two-dimensional sensor; respectively inputting the three-dimensional information data and the two-dimensional information data into a pre-trained three-dimensional detection model and a two-dimensional detection model to obtain a first three-dimensional detection frame and a first two-dimensional detection frame of each target object in the target area; inputting the three-dimensional information data and the two-dimensional information data into a pre-trained fusion detection model to obtain a second three-dimensional detection frame of each target object in the target area, and performing a coordinate system change on the second three-dimensional detection frame to obtain a second two-dimensional detection frame; calculating a first IoU value between a first two-dimensional detection frame and a second two-dimensional detection frame corresponding to each target object to obtain a first array, and calculating a second IoU value between a
  • the present application before inputting the three-dimensional information data and the two-dimensional information data into a pre-trained fusion detection model, it also includes: performing a chi-square comparison on the two-dimensional information data at the previous moment and the two-dimensional information data at the current moment to obtain a chi-square comparison value; judging whether the chi-square comparison value exceeds a preset threshold; when the chi-square comparison value exceeds the preset threshold, performing a histogram matching on the two-dimensional information data at the current moment and the two-dimensional information data at the previous moment to obtain enhanced two-dimensional information data at the current moment, and replacing the two-dimensional information data at the current moment; when the chi-square comparison value does not exceed the preset threshold, maintaining the two-dimensional information data at the current moment.
  • the coordinate system of the second three-dimensional detection frame is changed to obtain the second two-dimensional detection frame, including: obtaining the coordinates of the three-dimensional center point of the second three-dimensional detection frame; using the coordinates of the three-dimensional center point to confirm that the second three-dimensional detection frame is in the same plane as the three-dimensional center point. Homogeneous coordinates of four points; four two-dimensional coordinate points are calculated using the pre-acquired camera projection matrix, camera rotation matrix, sensor-to-camera coordinate system rotation matrix and homogeneous coordinates; and a second two-dimensional detection frame is constructed using the four two-dimensional coordinate points.
  • a first IoU value of a first two-dimensional detection frame and a second two-dimensional detection frame corresponding to each target object is calculated to obtain a first array, including: confirming a target first two-dimensional detection frame and a target second two-dimensional detection frame corresponding to each target object; respectively calculating a first area, a second area, and a third area of an overlapping area of the target first two-dimensional detection frame and the target second two-dimensional detection frame; obtaining a first IoU value corresponding to each target object by using the first area, the second area, and the third area; and constructing a first array using the first IoU values corresponding to all target objects.
  • the second IoU values of the first three-dimensional detection frame and the second three-dimensional detection frame corresponding to each target object are calculated to obtain a second array, including: confirming the target first three-dimensional detection frame and the target second three-dimensional detection frame corresponding to each target object; respectively calculating the first volume and the second volume of the target first three-dimensional detection frame and the target second three-dimensional detection frame; calculating the bottom surface area of the overlapping area of the target first three-dimensional detection frame and the target second three-dimensional detection frame; confirming the height of the overlapping area of the target first three-dimensional detection frame and the target second three-dimensional detection frame; using the bottom surface area and the height to calculate the third volume of the overlapping area; using the first volume, the second volume, and the third volume to calculate the second IoU value corresponding to each target object; and constructing a second array using the second IoU values corresponding to all target objects.
  • attack detection is performed based on the first array and the second array and the attacked sensor is located, including: respectively determining whether there are outliers in the first array and the second array; when there are outliers in the first array and no outliers in the second array, confirming that the three-dimensional sensor is attacked; when there are no outliers in the first array and there are outliers in the second array, confirming that the two-dimensional sensor is attacked; when there are outliers in both the first array and the second array, confirming that the three-dimensional sensor and/or the two-dimensional sensor is attacked; when there are no outliers in both the first array and the second array, confirming that the three-dimensional sensor and/or the two-dimensional sensor is attacked; value, confirm that the 3D sensor and 2D sensor are not attacked.
  • the three-dimensional sensor includes a lidar
  • the two-dimensional sensor includes a camera
  • the three-dimensional detection model is constructed based on the PointPillar algorithm
  • the two-dimensional detection model is constructed based on the YOLOv3 algorithm
  • the fusion detection model is constructed based on the AVOD algorithm.
  • an attack detection device for an autonomous driving system comprising: an acquisition module, used to respectively use a three-dimensional sensor and a two-dimensional sensor to respectively acquire three-dimensional information data and two-dimensional information data of a target area; a first detection module, used to input the three-dimensional information data and the two-dimensional information data into a pre-trained three-dimensional detection model and a two-dimensional detection model, respectively, to obtain a first three-dimensional detection frame and a first two-dimensional detection frame of each target object in the target area; a second detection module, used to input the three-dimensional information data and the two-dimensional information data into a pre-trained fusion detection model, to obtain a second three-dimensional detection frame of each target object in the target area, and to perform a coordinate system change on the second three-dimensional detection frame to obtain a second two-dimensional detection frame; a calculation module, used to calculate a first IoU value between a first two-dimensional detection frame and a second two
  • another technical solution adopted in the present application is: to provide a computer device, wherein the computer device includes a processor and a memory coupled to the processor, wherein program instructions are stored in the memory, and when the program instructions are executed by the processor, the processor executes the steps of the attack detection method of the autonomous driving system as described in any one of the above-mentioned items.
  • another technical solution adopted in the present application is: to provide a storage medium storing program instructions of an attack detection method for an autonomous driving system that can implement any of the above-mentioned ones.
  • the attack detection method of the autonomous driving system of the present application performs target detection on the three-dimensional information data and the two-dimensional information data obtained by the three-dimensional sensor and the two-dimensional sensor respectively to obtain a first three-dimensional detection frame and a first two-dimensional detection frame, and performs fusion target detection on the three-dimensional information data and the two-dimensional information data to obtain a second three-dimensional detection frame, and performs coordinate transformation on the second three-dimensional detection frame to obtain a second two-dimensional detection frame, and then performs IoU numerical calculation on the first two-dimensional detection frame and the second two-dimensional detection frame corresponding to each object to obtain a first array, performs IoU numerical calculation on the first three-dimensional detection frame and the second three-dimensional detection frame corresponding to each object to obtain a second array, analyzes the first array and the second array to confirm whether the sensor is attacked, and uses the correlation between the three-dimensional sensor and the two-dimensional sensor in time and space to perform attack detection, thereby improving the accuracy of attack detection,
  • FIG1 is a schematic diagram of a flow chart of an attack detection method for an automatic driving system according to an embodiment of the present invention
  • FIG2 is a schematic diagram of a second three-dimensional detection frame according to an embodiment of the present invention.
  • FIG3 is a schematic diagram of overlapping areas of two-dimensional detection frames according to an embodiment of the present invention.
  • FIG4 is a schematic diagram of functional modules of an attack detection device for an automatic driving system according to an embodiment of the present invention.
  • FIG. 5 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention.
  • FIG. 6 is a schematic diagram of the structure of a storage medium according to an embodiment of the present invention.
  • first”, “second”, “third” in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.
  • the features defined as “first”, “second”, “third” can expressly or implicitly include at least one of the features.
  • the meaning of “multiple” is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
  • all directional indications (such as up, down, left, right, front, back%) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly.
  • FIG1 is a flow chart of an attack detection method for an autonomous driving system according to an embodiment of the present invention. It should be noted that the method of the present invention is not limited to the flow sequence shown in FIG1 if substantially the same results are achieved. As shown in FIG1 , the attack detection method for an autonomous driving system includes the following steps:
  • Step S101 using a three-dimensional sensor and a two-dimensional sensor to obtain three-dimensional information data and two-dimensional information data of a target area respectively.
  • the three-dimensional sensor is preferably a laser radar. It should be noted that the three-dimensional sensor of the embodiment of the present invention is not limited to the laser radar, and other sensor devices that can be used to obtain three-dimensional information data are also included in the protection scope of the present invention.
  • the two-dimensional sensor is preferably a camera. It should be noted that the two-dimensional sensor of the embodiment of the present invention is not limited to the camera, and other sensor devices that can be used to obtain two-dimensional information data are also included in the protection scope of the present invention.
  • the present embodiment uses the three-dimensional sensor and the two-dimensional sensor equipped on the vehicle to simultaneously collect the three-dimensional information data and the two-dimensional information data of the target area.
  • the detection range is limited to a target area, which is the overlapping part of the field of view of the three-dimensional sensor and the two-dimensional sensor.
  • the overlapping area is the FOV (Field of view, FOV) field of view of the laser radar.
  • Step S102 input the three-dimensional information data and the two-dimensional information data into a pre-trained three-dimensional detection model and a two-dimensional detection model respectively, to obtain a first three-dimensional detection frame and a first two-dimensional detection frame of each target object in the target area.
  • the 3D detection model is built based on the PointPillar algorithm. Specifically, the 3D sensor uses the PointPillars (3D Object Proposal Generation and Detection from Point Cloud) algorithm to perform 3D target detection and obtain the first 3D target detection frame.
  • the input data is the original data point cloud data of the 3D sensor, that is, the 3D information data, in the format of [x, y, z, intensity].
  • the original feature of the 3D sensor is a point cloud set, which can be expressed as a vector (64-line laser radar, 1800 point clouds per line scan) where X represents the horizontal coordinate of the point cloud, Y represents the vertical coordinate of the point cloud, Z represents the height of the point cloud, and intensity represents the reflection intensity of the point cloud.
  • the two-stage method uses PointNet++ as the backbone network to complete the segmentation task first, determine the label of each 3D point, and use the feature to generate a box for each point that is divided into the foreground. Then the box is optimized.
  • the target object detection result is expressed as a vector Indicated by .
  • x lidar [X v Y v Z v ] T represents the center point value of the target object in the 3D sensor coordinate system
  • [L, W, H] T represents the length, width and height of the first 3D detection frame.
  • the two-dimensional detection model is built based on the YOLOv3 algorithm.
  • the YOLOv3 algorithm is used to perform two-dimensional target detection on the two-dimensional information data obtained by the two-dimensional sensor to obtain the target center point.
  • the collected picture is input into the two-dimensional detection model in RGB format.
  • the two-dimensional detection model calls the deep convolutional neural network algorithm to extract features from the RGB image.
  • Common structures in the neural network algorithm include convolution layer, pooling layer, activation layer, dropout layer, BN (batch normalization) layer, fully connected layer, etc.
  • the features extracted from the picture can effectively describe the information of the target object.
  • the input data format is the RGB image of the camera, and the output is the center point coordinates and the detection box width (width) and height (height) in the image coordinate system, in the format of Where X camera represents the value in the x direction on the image coordinate system, Y camera represents the value in the y direction, w represents the width of the first two-dimensional detection frame, and h represents the height of the first two-dimensional detection frame.
  • Step S103 input the three-dimensional information data and the two-dimensional information data into a pre-trained fusion detection model to obtain a second three-dimensional detection frame of each target object in the target area, and perform a coordinate system transformation on the second three-dimensional detection frame to obtain a second two-dimensional detection frame.
  • the fusion detection model is built based on the AVOD algorithm (Joint 3D Proposal Generation and Object Detection from View Aggregation).
  • the three-dimensional information data and the two-dimensional information data are input into the AVOD algorithm for target detection to obtain the second three-dimensional detection frame.
  • the AVOD algorithm fuses the three-dimensional information data and the two-dimensional information data.
  • the three-dimensional information data is the point cloud data obtained by the laser radar
  • the AVOD algorithm only uses the top view and the front view of the point cloud, which can reduce the amount of calculation without losing too much information.
  • generate a three-dimensional candidate area fuse the features and the candidate area, and output the final second three-dimensional detection frame.
  • step S103 the following steps are further included:
  • the two-dimensional information data at the current moment Histogram matching is performed with the two-dimensional information data of the previous moment to obtain enhanced two-dimensional information data of the current moment, and replace the two-dimensional information data of the current moment.
  • the purpose of image enhancement is to improve the defense capability of the fusion detection model against two-dimensional sensor attacks.
  • the two-dimensional information data at the previous moment and the two-dimensional information data at the current moment are compared by chi-square to obtain a chi-square comparison value, and then it is determined whether the chi-square comparison value exceeds a preset threshold.
  • the chi-square comparison value exceeds the preset threshold the two-dimensional information data at the current moment is matched with the two-dimensional information data at the previous moment by histogram matching to obtain enhanced two-dimensional information data at the current moment, and the two-dimensional information data at the current moment is replaced.
  • the chi-square comparison value does not exceed the preset threshold, the two-dimensional information data at the current moment is maintained.
  • step S103 the step of performing coordinate system transformation on the second three-dimensional detection frame to obtain the second two-dimensional detection frame specifically includes:
  • the second three-dimensional detection frame is projected to the image coordinate system to obtain the second two-dimensional detection frame.
  • xci represents the x-axis coordinate of point ci
  • yci represents the y-axis coordinate of point ci
  • i 1 , 2, 3, 4
  • [XvYvZv ] T represents the coordinates of the center point of the second three-dimensional detection box.
  • Y represents the coordinates of the second two-dimensional detection box
  • the homogeneous coordinates of point Y represent the homogeneous vector of the target object
  • P is the camera projection matrix
  • R is the camera rotation matrix
  • Tr velocity matrix is the rotation matrix from the 3x4 three-dimensional coordinate system to the camera coordinate system.
  • Step S104 Calculate the first two-dimensional detection frame and the second two-dimensional detection frame corresponding to each target object.
  • the first IoU value of the detection box is obtained to obtain a first array
  • the second IoU value of the first three-dimensional detection box and the second three-dimensional detection box corresponding to each target object is calculated to obtain a second array.
  • the IoU value can be used to characterize the inconsistency between the two groups of detection frames.
  • the IoU value is calculated using the first two-dimensional detection frame and the second two-dimensional detection frame to obtain a first IoU value that characterizes the inconsistency between the first two-dimensional detection frame and the second two-dimensional detection frame.
  • Multiple target objects correspond to multiple first IoU values, which are constructed into a first array.
  • the IoU value is calculated using the first three-dimensional detection frame and the second three-dimensional detection frame to obtain a second IoU value that characterizes the inconsistency between the first three-dimensional detection frame and the second three-dimensional detection frame.
  • Multiple target objects correspond to multiple second IoU values, which are constructed into a second array.
  • the two groups of detection frames correspond to the same target area, and the target objects in the target area are constant. Therefore, the number of elements in the first array and the second array is equal, and each target object has a corresponding element in the first array or the second array.
  • step S104 the step of calculating the first IoU value between the first two-dimensional detection frame and the second two-dimensional detection frame corresponding to each target object to obtain the first array specifically includes:
  • the first area and the second area of the target first two-dimensional detection frame and the target second two-dimensional detection frame are calculated by the coordinates of the four vertices.
  • ; S B
  • SA represents the first area, A1(x a1 ,y a1 ), B1(x a1 ,y a2 ), C1(x a2 ,y a1 ), and D1(x a2 ,y a2 ) represent the coordinates of the four vertices of the first two-dimensional detection box of the target.
  • SB represents the second area, A2(x b1 ,y b1 ), B2(x b1 ,y b2 ), C2(x b2 ,y b1 ), and D2(x b2 ,y b2 ) represent the coordinates of the four vertices of the second two-dimensional detection box of the target.
  • the overlapping area is also a rectangular frame.
  • the four vertices of the rectangular frame can be obtained according to the four vertex coordinates of the overlapping target first two-dimensional detection frame and the target second two-dimensional detection frame.
  • the coordinates of the upper left corner are A2 (x b1 , y b1 )
  • the coordinates of the lower left corner are E (x b1 , y a2 )
  • the coordinates of the upper right corner are F (x a2 , y b1 )
  • the coordinates of the lower right corner are D1 (x a2 , y a2 ).
  • the third area of the overlapping area is then calculated using the coordinates of the four vertices of the overlapping area.
  • the first IoU value the third area/(the first area+the second area-the third area).
  • step S104 the step of calculating the second IoU value of the first three-dimensional detection frame and the second three-dimensional detection frame corresponding to each target object to obtain a second array specifically includes:
  • the target first three-dimensional detection frame and the target second three-dimensional detection frame are also confirmed by using the Euclidean distance between the center point coordinates of the first three-dimensional detection frame and the second three-dimensional detection frame.
  • the length, width and height of the three-dimensional detection frame can be confirmed according to the coordinates of the eight vertices of the three-dimensional detection frame, and the volume of the three-dimensional detection frame can be calculated using the length, width and height.
  • the overlapping area of the first three-dimensional detection frame and the second three-dimensional detection frame is also a cubic structure, and its volume can be obtained by multiplying the bottom area by the height, and the target objects corresponding to the first three-dimensional detection frame and the second three-dimensional detection frame are both on the ground, that is, the bottom surfaces of the first three-dimensional detection frame and the second three-dimensional detection frame are in the same plane, so the vertex coordinates of the bottom surface of the overlapping area of the first three-dimensional detection frame and the second three-dimensional detection frame can be confirmed according to the four vertex coordinates of the bottom surface of the first three-dimensional detection frame and the four vertex coordinates of the second three-dimensional detection frame, and then the bottom area of the overlapping area can be obtained according to the vertex coordinates of the bottom surface of the overlapping area.
  • This method is the same as the method of obtaining the overlapping area in the two-dimensional detection frame, and will not be repeated here.
  • the height of the overlapping area since the target objects corresponding to the first three-dimensional detection frame and the second three-dimensional detection frame are both on the ground, that is, the bottom surfaces of the first three-dimensional detection frame and the second three-dimensional detection frame are in the same plane, therefore, the height of the overlapping area of the first three-dimensional detection frame and the second three-dimensional detection frame is the smaller height value of the first three-dimensional detection frame and the second three-dimensional detection frame.
  • the second IoU value the third volume / (the first volume + the second volume - the third volume)
  • Step S105 Perform attack detection based on the first array and the second array and locate the attacked sensor.
  • an inconsistency detection is performed on the two arrays to confirm whether the sensor is attacked and locate whether the attacked three-dimensional sensor or two-dimensional sensor.
  • step S105 specifically includes:
  • the attack detection method of the automatic driving system of the embodiment of the present invention performs target detection on the three-dimensional information data and the two-dimensional information data obtained by the three-dimensional sensor and the two-dimensional sensor respectively to obtain a first three-dimensional detection frame and a first two-dimensional detection frame, and performs fusion target detection on the three-dimensional information data and the two-dimensional information data to obtain a second three-dimensional detection frame, and performs coordinate transformation on the second three-dimensional detection frame to obtain a second two-dimensional detection frame, and then performs coordinate transformation on the first two-dimensional detection frame corresponding to each object.
  • the IoU value calculation is performed on the first three-dimensional detection frame and the second three-dimensional detection frame to obtain a first array
  • the IoU value calculation is performed on the first three-dimensional detection frame and the second three-dimensional detection frame corresponding to each object to obtain a second array
  • the first array and the second array are analyzed to confirm whether the sensor is attacked. It uses the correlation between the three-dimensional sensor and the two-dimensional sensor in time and space to perform attack detection, thereby improving the accuracy of attack detection, and can realize attack detection by using the three-dimensional sensor and the two-dimensional sensor as the basic configuration of the autonomous driving car. It is no longer limited to the target detection algorithm using one of the sensors as the main one for attack detection, and has higher universality.
  • Fig. 4 is a functional module diagram of an attack detection device for an autonomous driving system according to an embodiment of the present invention.
  • the attack detection device 20 for the autonomous driving system includes an acquisition module 21 , a first detection module 22 , a second detection module 23 , a calculation module 24 and an analysis module 25 .
  • An acquisition module 21 is used to respectively acquire three-dimensional information data and two-dimensional information data of a target area using a three-dimensional sensor and a two-dimensional sensor;
  • a first detection module 22 used to input the three-dimensional information data and the two-dimensional information data into a pre-trained three-dimensional detection model and a two-dimensional detection model, respectively, to obtain a first three-dimensional detection frame and a first two-dimensional detection frame of each target object in the target area;
  • the second detection module 23 is used to input the three-dimensional information data and the two-dimensional information data into a pre-trained fusion detection model to obtain a second three-dimensional detection frame of each target object in the target area, and perform a coordinate system change on the second three-dimensional detection frame to obtain a second two-dimensional detection frame;
  • the calculation module 24 is used to calculate a first IoU value between the first two-dimensional detection frame and the second two-dimensional detection frame corresponding to each target object to obtain a first array, and calculate a second IoU value between the first three-dimensional detection frame and the second three-dimensional detection frame corresponding to each target object to obtain a second array;
  • the analysis module 25 is used to perform attack detection based on the first array and the second array and locate the attacked sensor.
  • the second detection module 23 before the second detection module 23 performs the operation of inputting the three-dimensional information data and the two-dimensional information data into the pre-trained fusion detection model, it is also used to:
  • the information data is compared with the two-dimensional information data at the current moment by chi-square to obtain a chi-square comparison value; whether the chi-square comparison value exceeds a preset threshold is determined; when the chi-square comparison value exceeds the preset threshold, the two-dimensional information data at the current moment is matched with the two-dimensional information data at the previous moment by histogram to obtain enhanced two-dimensional information data at the current moment, and the two-dimensional information data at the current moment is replaced; when the chi-square comparison value does not exceed the preset threshold, the two-dimensional information data at the current moment is maintained.
  • the second detection module 23 performs a coordinate system change on the second three-dimensional detection frame to obtain an operation of a second two-dimensional detection frame, specifically including: obtaining the three-dimensional center point coordinates of the second three-dimensional detection frame; using the three-dimensional center point coordinates to confirm the homogeneous coordinates of four points on the second three-dimensional detection frame that are in the same plane as the three-dimensional center point; using the pre-acquired camera projection matrix, camera rotation matrix, sensor-to-camera coordinate system rotation matrix and homogeneous coordinate calculations to obtain four two-dimensional coordinate points; and using the four two-dimensional coordinate points to construct a second two-dimensional detection frame.
  • the calculation module 24 performs an operation of calculating the first IoU value of the first two-dimensional detection frame and the second two-dimensional detection frame corresponding to each target object to obtain the first array, specifically including: confirming the target first two-dimensional detection frame and the target second two-dimensional detection frame corresponding to each target object; respectively calculating the first area and the second area of the target first two-dimensional detection frame and the target second two-dimensional detection frame, and the third area of the overlapping area of the target first two-dimensional detection frame and the target second two-dimensional detection frame; using the first area, the second area, and the third area to calculate the first IoU value corresponding to each target object; and constructing the first array using the first IoU values corresponding to all target objects.
  • the calculation module 24 performs the operation of calculating the second IoU value of the first three-dimensional detection frame and the second three-dimensional detection frame corresponding to each target object to obtain the second array, specifically including: confirming the target first three-dimensional detection frame and the target second three-dimensional detection frame corresponding to each target object; calculating the first volume and the second volume of the target first three-dimensional detection frame and the target second three-dimensional detection frame respectively; calculating the bottom of the overlapping area of the target first three-dimensional detection frame and the target second three-dimensional detection frame.
  • the analysis module 25 performs operations of attack detection and locating the attacked sensor based on the first array and the second array, specifically including: respectively determining whether there are outliers in the first array and the second array; when there are outliers in the first array and no outliers in the second array, confirming that the three-dimensional sensor is attacked; when there are no outliers in the first array and there are outliers in the second array, confirming that the two-dimensional sensor is attacked; when there are outliers in both the first array and the second array, confirming that the three-dimensional sensor and/or the two-dimensional sensor are attacked; when there are no outliers in both the first array and the second array, confirming that the three-dimensional sensor and the two-dimensional sensor are not attacked.
  • the three-dimensional sensor includes a lidar
  • the two-dimensional sensor includes a camera
  • the three-dimensional detection model is constructed based on a PointPillar algorithm
  • the two-dimensional detection model is constructed based on a YOLOv3 algorithm
  • the fusion detection model is constructed based on an AVOD algorithm.
  • each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.
  • the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
  • FIG5 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention.
  • the computer device 30 includes a processor 31 and a memory 32 coupled to the processor 31.
  • the memory 32 stores program instructions.
  • the program instructions are executed by the processor 31, the computer device 30 performs the following operations:
  • the processor 31 executes the steps of the attack detection method for the autonomous driving system described in any of the above embodiments.
  • the processor 31 may also be referred to as a CPU (Central Processing Unit).
  • the processor 31 may be an integrated circuit chip having the ability to process signals.
  • the processor 31 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP digital signal processor
  • ASIC application-specific integrated circuit
  • FPGA field programmable gate array
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
  • the storage medium of the embodiment of the present invention stores program instructions 41 that can implement the attack detection method of the above-mentioned autonomous driving system, wherein the program instructions 41 can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application.
  • a computer device which can be a personal computer, server, or network device, etc.
  • processor processor
  • the aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer device such as a computer, a server, a mobile phone, or a tablet.
  • program codes such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer device such as a computer, a server, a mobile phone, or a tablet.
  • the disclosed computer equipment, devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种自动驾驶系统的攻击检测方法、装置、设备及存储介质，其中方法包括分别对三维传感器、二维传感器获取的三维信息数据、二维信息数据进行目标检测，得到第一三维检测框、第一二维检测框，且对三维信息数据、二维信息数据进行融合目标检测，得到第二三维检测框，并对第二三维检测框进行坐标变换，得到第二二维检测框，再对每个物体对应的第一二维检测框和第二二维检测框进行IoU数值计算，得到第一数组，对每个物体对应的第一三维检测框和第二三维检测框进行IoU数值计算，得到第二数组，分析第一数组和和第二数组确认是否传感器是否被攻击。本发明能够基于多模态的数据进行攻击检测并能够定位到被攻击的传感器。

Description

自动驾驶系统的攻击检测方法、装置、设备及存储介质 技术领域
本申请涉及传感器检测技术领域，特别是涉及一种自动驾驶系统的攻击检测方法、装置、设备及存储介质。
背景技术
无人驾驶技术是一个涉及人工智能、传感技术、地图技术以及计算机等诸多前沿科技的综合技术。无人驾驶技术的功能实现依赖于车载定位传感器系统。定位传感器系统主要由激光雷达、视觉摄像头、毫米波雷达、全球定位系统(GPS)等设备组成。定位传感器系统为无人驾驶汽车的规划与决策模块提供丰富的定位数据，即车辆速度、姿态和位置等信息。自动驾驶汽车路线规划与决策控制的安全性以定位传感器系统的安全为前提，如果自动驾驶汽车的定位传感器系统出现异常，将会导致传感器获取到错误的定位信息，进而规划错误的驾驶控制策略，对其他车辆安全及驾驶员、行人生命造成威胁。
目前，无人驾驶技术的传感器攻击检测的方式主要是与具体的单传感器目标检测算法结合，并在此基础上进行攻击检测，但是，该种方式不具有普适性，例如，以激光雷达为主的目标检测算法搭配的攻击检测方法不能适用于以摄像头为主的目标检测算法的攻击检测场景，并且针对单一类型传感器的检测方式的攻击检测结果准确性不高。
发明内容
有鉴于此，本申请提供一种自动驾驶系统的攻击检测方法、装置、设备及存储介质，以解决现有自动驾驶系统攻击检测方式不具备普适性且准确性不高的问题。
为解决上述技术问题，本申请采用的一个技术方案是：提供一种自动驾驶系统的攻击检测方法，其包括：分别利用三维传感器、二维传感器分别获取目标区域的三维信息数据、二维信息数据；将三维信息数据、二维信息数据分别输入预先训练好的三维检测模型、二维检测模型，得到目标区域中的每个目标物体的第一三维检测框、第一二维检测框；将三维信息数据、二维信息数据输入预先训练好的融合检测模型，得到目标区域中的每个目标物体的第二三维检测框，并对第二三维检测框进行坐标系变化，得到第二二维检测框；计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组，且计算每个目标物体对应的第一三维检测框和第二三维检测框的第二IoU数值，得到第二数组；基于第一数组和第二数组进行攻击检测并定位被攻击的传感器。
作为本申请的进一步改进，将三维信息数据、二维信息数据输入预先训练好的融合检测模型之前，还包括：将上一时刻的二维信息数据和当前时刻的二维信息数据进行卡方比较，得到卡方比较数值；判断卡方比较数值是否超过预设阈值；当卡方比较数值超过预设阈值时，将当前时刻的二维信息数据与上一时刻的二维信息数据进行直方图匹配，得到当前时刻的增强二维信息数据，并替换掉当前时刻的二维信息数据；当卡方比较数值未超过预设阈值时，维持当前时刻的二维信息数据。
作为本申请的进一步改进，对第二三维检测框进行坐标系变化，得到第二二维检测框，包括：获取第二三维检测框的三维中心点坐标；利用三维中心点坐标确认第二三维检测框上与三维中心点在同一平面的 四个点的齐次坐标；利用预先获取的相机投影矩阵、相机旋转矩阵、传感器到相机坐标系的旋转矩阵和齐次坐标计算得到四个二维坐标点；利用四个二维坐标点构建第二二维检测框。
作为本申请的进一步改进，计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组，包括：确认每个目标物体对应的目标第一二维检测框和目标第二二维检测框；分别计算目标第一二维检测框、目标第二二维检测框的第一面积、第二面积，以及目标第一二维检测框与目标第二二维检测框的重叠区域的第三面积；利用第一面积、第二面积、第三面积计算得到每个目标物体对应的第一IoU数值；利用所有目标物体对应的第一IoU数值构建第一数组。
作为本申请的进一步改进，计算每个目标物体对应的第一三维检测框和第二三维检测框的第二IoU数值，得到第二数组，包括：确认每个目标物体对应的目标第一三维检测框和目标第二三维检测框；分别计算目标第一三维检测框、目标第二三维检测框的第一体积、第二体积；计算目标第一三维检测框与目标第二三维检测框的重叠区域的底面面积；确认目标第一三维检测框与目标第二三维检测框的重叠区域的高度；利用底面面积和高度计算得到重叠区域的第三体积；利用第一体积、第二体积、第三体积计算得到每个目标物体对应的第二IoU数值；利用所有目标物体对应的第二IoU数值构建第二数组。
作为本申请的进一步改进，基于第一数组和第二数组进行攻击检测并定位被攻击的传感器，包括：分别判断第一数组、第二数组是否存在离群值；当第一数组存在离群值、第二数组不存在离群值时，确认三维传感器被攻击；当第一数组不存在离群值、第二数组存在离群值时，确认二维传感器被攻击；当第一数组、第二数组均存在离群值时，确认三维传感器和/或二维传感器被攻击；当第一数组、第二数组均不存在离群 值时，确认三维传感器和二维传感器未受到攻击。
作为本申请的进一步改进，三维传感器包括激光雷达，二维传感器包括摄像头，三维检测模型基于PointPillar算法构建，二维检测模型基于YOLOv3算法构建，融合检测模型基于AVOD算法构建。
为解决上述技术问题，本申请采用的又一个技术方案是：提供一种自动驾驶系统的攻击检测装置，其包括：获取模块，用于分别利用三维传感器、二维传感器分别获取目标区域的三维信息数据、二维信息数据；第一检测模块，用于将三维信息数据、二维信息数据分别输入预先训练好的三维检测模型、二维检测模型，得到目标区域中的每个目标物体的第一三维检测框、第一二维检测框；第二检测模块，用于将三维信息数据、二维信息数据输入预先训练好的融合检测模型，得到目标区域中的每个目标物体的第二三维检测框，并对第二三维检测框进行坐标系变化，得到第二二维检测框；计算模块，用于计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组，且计算每个目标物体对应的第一三维检测框和第二三维检测框的第二IoU数值，得到第二数组；分析模块，用于基于第一数组和第二数组进行攻击检测并定位被攻击的传感器。
为解决上述技术问题，本申请采用的再一个技术方案是：提供一种计算机设备，所述计算机设备包括处理器、与所述处理器耦接的存储器，所述存储器中存储有程序指令，所述程序指令被所述处理器执行时，使得所述处理器执行如上述任一项的自动驾驶系统的攻击检测方法的步骤。
为解决上述技术问题，本申请采用的再一个技术方案是：提供一种存储介质，存储有能够实现上述任一项的自动驾驶系统的攻击检测方法的程序指令。
本申请的有益效果是：本申请的自动驾驶系统的攻击检测方法通过分别对三维传感器、二维传感器获取的三维信息数据、二维信息数据进行目标检测，得到第一三维检测框、第一二维检测框，且对三维信息数据、二维信息数据进行融合目标检测，得到第二三维检测框，并对第二三维检测框进行坐标变换，得到第二二维检测框，再对每个物体对应的第一二维检测框和第二二维检测框进行IoU数值计算，得到第一数组，对每个物体对应的第一三维检测框和第二三维检测框进行IoU数值计算，得到第二数组，分析第一数组和和第二数组确认是否传感器是否被攻击，其利用三维传感器和二维传感器在时间和空间上的相关性进行攻击检测，从而提高攻击检测的准确性，并且利用作为自动驾驶汽车的基本配置的三维传感器和二维传感器即能够实现攻击检测，不再局限于利用其中一种传感器为主的目标检测算法进行攻击检测，具有更高的普适性。
附图说明
图1是本发明实施例的自动驾驶系统的攻击检测方法的一流程示意图；
图2是本发明实施例的第二三维检测框的示意图；
图3是本发明实施例的二维检测框重叠区域的示意图；
图4是本发明实施例的自动驾驶系统的攻击检测装置的功能模块示意图；
图5是本发明实施例的计算机设备的结构示意图；
图6是本发明实施例的存储介质的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请的一部分实 施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请中的术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”、“第三”的特征可以明示或者隐含地包括至少一个该特征。本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。本申请实施例中所有方向性指示(诸如上、下、左、右、前、后……)仅用于解释在某一特定姿态(如附图所示)下各部件之间的相对位置关系、运动情况等，如果该特定姿态发生改变时，则该方向性指示也相应地随之改变。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
图1是本发明实施例的自动驾驶系统的攻击检测方法的流程示意图。需注意的是，若有实质上相同的结果，本发明的方法并不以图1所示的流程顺序为限。如图1所示，该自动驾驶系统的攻击检测方法包括步骤：
步骤S101：分别利用三维传感器、二维传感器分别获取目标区域的三维信息数据、二维信息数据。
本实施例中，三维传感器优选为激光雷达，需要说明的是，本发明实施例的三维传感器不局限于激光雷达，其他可用于获取三维信息数据的传感器设备同样包括在本发明的保护范围之内。二维传感器优选为摄像头，需要说明的是，本发明实施例的二维传感器不局限于摄像头，其他可用于获取二维信息数据的传感器设备同样包括在本发明的保护范围之内。
具体地，本实施例在使用自动驾驶系统驾驶车辆时，利用车辆上配备的三维传感器和二维传感器同时采集目标区域的三维信息数据和二维信息数据。需要说明的是，在进行攻击检测时，其检测范围局限在一个目标区域内，该目标区域为三维传感器和二维传感器视野交叠的部分，例如，对于激光雷达和摄像头而言，该交叠区域为激光雷达的FOV(Field of view,FOV)视域，通过将摄像头拍摄的图像尺寸投影到激光雷达的视觉区域之内，从而获取激光雷达采集的三维信息数据和摄像头采集的二维信息数据。
步骤S102：将三维信息数据、二维信息数据分别输入预先训练好的三维检测模型、二维检测模型，得到目标区域中的每个目标物体的第一三维检测框、第一二维检测框。
其中，该三维检测模型基于PointPillar算法构建。具体地，三维传感器使用PointPillars(3D Object Proposal Generation and Detection from Point Cloud)算法进行三维目标检测得到第一三维目标检测框，输入数据为三维传感器的原始数据点云数据即三维信息数据，格式为[x,y,z,intensity]。三维传感器的原始特征为点云集合，可表示为矢量(64线激光雷达，每线扫描一周1800个点云)其中X表示点云在水平面的横坐标，Y表示点云在水平面的纵坐标，Z表示点云的高度，intensity表示点云的反射强度。使用 two-stage方式，利用PointNet++作为主干网络，先完成segmentation任务，判断每个三维点的label，对分为前景的每个点，使用feature生成框。然后对框进行优化。目标物体检测结果用向量 表示。其中xlidar＝[XvYvZv]T表示目标物体在三维传感器坐标系上的中心点值，[L,W,H]T表示第一三维检测框的长宽高。
其中，该二维检测模型基于YOLOv3算法构建。具体地，使用YOLOv3算法对二维传感器获取的二维信息数据进行二维目标检测得到目标中心点。例如，以摄像头为例，将采集到的图片以RGB格式输入二维检测模型，二维检测模型调用深度卷积神经网络算法对RGB图像进行特征提取，神经网络算法中常见的结构有卷积层，池化层，激活层，dropout层，BN(batch normalization)层，全连接层等，最终从图片中提取的特征能有效描述目标物体的信息。输入的数据格式为摄像头的RGB图像，输出的为图像坐标系上的中心点坐标及检测框宽(width)和高(height)，格式为其中Xcamera表示图像坐标系上x方向的数值，Ycamera表示y方向数值，w表示第一二维检测框的宽度，h表示第一二维检测框的高度。得到第一二维检测框后可以进一步得到第一二维检测框四个点的坐标分别如下所示：




Xcamera，1,＝Xcamera，3,＝Xcamera-w/2；
Xcamera，2,＝Xcamera，4,＝Xcamera+w/2；
Ycamera，1,＝Ycamera，2,＝Ycamera+h/2；
Ycamera，3,＝Ycamera，4,＝Ycamera-h/2；
其中Xcamera，i,i＝1，2，3，4分别表示第一二维检测框左上角，右上角，左下角，右下角的图片坐标系的x方向的坐标值。Ycamera，i,i＝1，2，3，4分别表示第一二维检测框左上角，右上角，左下角，右下角的图片坐标系的y方向的坐标值。
步骤S103：将三维信息数据、二维信息数据输入预先训练好的融合检测模型，得到目标区域中的每个目标物体的第二三维检测框，并对第二三维检测框进行坐标系变化，得到第二二维检测框。
具体地，该融合检测模型基于AVOD算法(Joint 3D Proposal Generation and Object Detection from ViewAggregation)构建，在得到三维信息数据和二维信息数据后，将三维信息数据和二维信息数据输入至AVOD算法进行目标检测，得到第二三维检测框。AVOD算法融合了三维信息数据和二维信息数据，当三维信息数据为激光雷达获取的点云数据信息时，AVOD算法只使用了点云的俯视图和前视图，这样既能减少计算量，又不至于丧失过多的信息。随后生成三维候选区域，把特征和候选区域融合后输出最终的第二三维检测框。
进一步的，为了提高二维传感器的防御能力，在一些实施例中，步骤S103之前还包括：
1、将上一时刻的二维信息数据和当前时刻的二维信息数据进行卡方比较，得到卡方比较数值。
2、判断卡方比较数值是否超过预设阈值。
3、当卡方比较数值超过预设阈值时，将当前时刻的二维信息数据 与上一时刻的二维信息数据进行直方图匹配，得到当前时刻的增强二维信息数据，并替换掉当前时刻的二维信息数据。
4、当卡方比较数值未超过预设阈值时，维持当前时刻的二维信息数据。
本实施例中，图像增强的目的是为了提高融合检测模型对二维传感器攻击的防御能力。通过将上一时刻的二维信息数据和当前时刻的二维信息数据进行卡方比较，得到卡方比较数值，再判断卡方比较数值是否超过预设阈值。当卡方比较数值超过预设阈值时，将当前时刻的二维信息数据与上一时刻的二维信息数据进行直方图匹配，得到当前时刻的增强二维信息数据，并替换掉当前时刻的二维信息数据。当卡方比较数值未超过预设阈值时，维持当前时刻的二维信息数据。
进一步的，步骤S103中对第二三维检测框进行坐标系变化，得到第二二维检测框的步骤，具体包括：
1、获取第二三维检测框的三维中心点坐标。
2、利用三维中心点坐标确认第二三维检测框上与三维中心点在同一平面的四个点的齐次坐标。
3、利用预先获取的相机投影矩阵、相机旋转矩阵、传感器到相机坐标系的旋转矩阵和齐次坐标计算得到四个二维坐标点。
4、利用四个二维坐标点构建第二二维检测框。
具体地，本实施例中，使用4个角点(只包含x坐标,y坐标)和2个高度来描述一个第二三维检测框，如图2所示，第二三维检测框以向量表示，其中ci＝[xciyci]T，i＝1，2，3，4表示第二三维检测框4个顶点的x、y方向的坐标，h1表示第二三维检测框底面距离三维坐标系中x轴与y轴构成的平面的高度，h2表示第二三维检测框顶面距离三维坐标系中x轴与y轴构成的平 面的高度。
具体地，在得到第二三维检测框后，将第二三维检测框投影至图像坐标系，得到第二二维检测框。请一并参阅图2，首先将第二三维检测框的向量形式转换为常规的三维检测框格式Y3D融合检测＝[XvYvZv]T，其中：
Zv＝(h2-h1)/2；
Xv＝xc1-xc2；
Yv＝yc1-yc2；
其中，xci表示ci点的x轴坐标，yci表示ci点的y轴坐标，i＝1，2，3，4，[XvYvZv]T表示第二三维检测框中心点的坐标。
然后，对第二三维检测框中心点附近的四个点(图2中A、B、C、D四个点)依次进行坐标系的转换，得到四个在图像坐标系的点，具体转换过程如下：
Y点A齐次坐标＝[Xv，1,Yv，1,Zv,1]T＝[Xv-(xc2-xc1)/2,Yv,h2]；
Y点B齐次坐标＝[Xv，2,Yv，2,Zv,2]T＝[Xv+(xc2-xc1)/2,Yv,h2]；
Y点c齐次坐标＝[Xv，3,Yv，3,Zv,3]T＝[Xv-(xc2-xc1)/2,Yv,h1]；
Y点D齐次坐标＝[Xv，4,Yv，4,Zv,4]T＝[Xv+(xc2-xc1)/2,Yv,h1]；
Y＝P*R*Trvelotocam*Y点n齐次坐标，n＝A，B，C，D；
其中，Y表示第二二维检测框的坐标，Y点n齐次坐标表示目标物体的齐次向量，P为相机投影矩阵，R为相机旋转矩阵，Trvelotocam为3x4的三维坐标系到相机坐标系的旋转矩阵。
步骤S104：计算每个目标物体对应的第一二维检测框与第二二维检 测框的第一IoU数值，得到第一数组，且计算每个目标物体对应的第一三维检测框和第二三维检测框的第二IoU数值，得到第二数组。
具体地，IoU数值可以用于表征两组检测框之间的不一致性。利用第一二维检测框和第二二维检测框进行IoU数值计算，得到表征第一二维检测框和第二二维检测框不一致性的第一IoU数值，多个目标物体则对应有多个第一IoU数值，将其构建为第一数组。利用第一三维检测框和第二三维检测框进行IoU数值计算，得到表征第一三维检测框和第二三维检测框不一致性的第二IoU数值，多个目标物体则对应有多个第二IoU数值，将其构建为第二数组。需要说明的是，两组检测框均对应的同一目标区域，该目标区域的目标物体是恒定的，因此，第一数组和第二数组中元素个数相等，每个目标物体在第一数组或第二数组中均存在对应的元素。
进一步的，步骤S104中计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组的步骤，具体包括：
1、确认每个目标物体对应的目标第一二维检测框和目标第二二维检测框。
具体地，由于场景中往往存在多个目标物体，会有多个检测框，因此，在确认每个目标物体对应的目标第一二维检测框和目标第二二维检测框时，先选取一个第一二维检测框作为目标第一二维检测框，再计算该目标第一二维检测框的中心点与每个第二二维检测框的中心点的欧式距离，选取欧式距离最小的第二二维检测框作为目标第二二维检测框。
2、分别计算目标第一二维检测框、目标第二二维检测框的第一面积、第二面积，以及目标第一二维检测框与目标第二二维检测框的重叠区域的第三面积。
具体地，通过获取二维检测框四个顶点的坐标，通过四个顶点的坐标计算得到目标第一二维检测框和目标第二二维检测框的第一面积和第二面积。计算公式如下：
SA＝|xa2-xa1|×|ya2-ya1|；
SB＝|xb2-xb1|×|yb2-yb1|；
其中，SA表示第一面积，A1(xa1,ya1)、B1(xa1,ya2)、C1(xa2,ya1)、D1(xa2,ya2)表示目标第一二维检测框四个顶点的坐标。SB表示第二面积，A2(xb1,yb1)、B2(xb1,yb2)、C2(xb2,yb1)、D2(xb2,yb2)表示目标第二二维检测框四个顶点的坐标。
以图3所示为例进行说明，两个检测框存在重叠区域时，该重叠区域同样为一个矩形框，该矩形框的四个顶点可根据重叠的目标第一二维检测框和目标第二二维检测框的四个顶点坐标得到，其左上角坐标为A2(xb1，yb1)，左下角坐标为E(xb1,ya2)，右上角坐标为F(xa2,yb1)，右下角坐标为D1(xa2,ya2)。再利用重叠区域的四个顶点坐标计算得到重叠区域的第三面积。
3、利用第一面积、第二面积、第三面积计算得到每个目标物体对应的第一IoU数值。
具体地，第一IoU数值＝第三面积/(第一面积+第二面积-第三面积)。
4、利用所有目标物体对应的第一IoU数值构建第一数组。
具体地，第一数组表示为：I＝[i1,i2,…,in]，n为第一IoU数值的个数。
进一步的，步骤S104中计算每个目标物体对应的第一三维检测框和第二三维检测框的第二IoU数值，得到第二数组的步骤，具体包括：
1、确认每个目标物体对应的目标第一三维检测框和目标第二三维检测框。
具体地，目标第一三维检测框和目标第二三维检测框同样利用第一三维检测框和第二三维检测框的中心点坐标之间的欧式距离确认。
2、分别计算目标第一三维检测框、目标第二三维检测框的第一体积、第二体积。
具体地，根据三维检测框的8个顶点的坐标即可确认三维检测框的长宽高，利用长宽高计算得到三维检测框的体积。
3、计算目标第一三维检测框与目标第二三维检测框的重叠区域的底面面积。
4、确认目标第一三维检测框与目标第二三维检测框的重叠区域的高度。
5、利用底面面积和高度计算得到重叠区域的第三体积。
需要理解的是，第一三维检测框和第二三维检测框的重叠区域同样为一方体结构，其体积可由底面积乘以高求得，而第一三维检测框和第二三维检测框对应的目标物体均处于地面上，即第一三维检测框和第二三维检测框的底面处于同一平面，因此第一三维检测框和第二三维检测框的重叠区域的底面的顶点坐标可根据第一三维检测框的底面的四个顶点坐标和第二三维检测框的四个顶点坐标确认，再根据重叠区域的底面的顶点坐标可求得重叠区域的底面积，该方式与二维检测框中求取重叠面积的方式相同，此处不再赘述。而对于重叠区域的高，因第一三维检测框和第二三维检测框对应的目标物体均处于地面上，即第一三维检测框和第二三维检测框的底面处于同一平面，因此，第一三维检测框和第二三维检测框重叠区域的高为第一三维检测框和第二三维检测框中较小的高度值。在得到第一三维检测框和第二三维检测框的重叠区域的底面积和高之后，即可计算得到该重叠区域的第三体积。
6、利用第一体积、第二体积、第三体积计算得到每个目标物体对 应的第二IoU数值。
具体地，第二IoU数值＝第三体积/(第一体积+第二体积-第三体积)
7、利用所有目标物体对应的第二IoU数值构建第二数组。
具体地，第一数组表示为：U＝[u1,u2,…,un]，n为第二IoU数值的个数。
步骤S105：基于第一数组和第二数组进行攻击检测并定位被攻击的传感器。
具体地，在得到第一数组和第二数组后，对两个数组进行不一致性检测，从而确认是否传感器是否被攻击，并定位被攻击的三维传感器还是二维传感器。
进一步的，步骤S105具体包括：
1、分别判断第一数组、第二数组是否存在离群值。
2、当第一数组存在离群值、第二数组不存在离群值时，确认三维传感器被攻击。
3、当第一数组不存在离群值、第二数组存在离群值时，确认二维传感器被攻击。
4、当第一数组、第二数组均存在离群值时，确认三维传感器和/或二维传感器被攻击。
5、当第一数组、第二数组均不存在离群值时，确认三维传感器和二维传感器未受到攻击。
本发明实施例的自动驾驶系统的攻击检测方法通过分别对三维传感器、二维传感器获取的三维信息数据、二维信息数据进行目标检测，得到第一三维检测框、第一二维检测框，且对三维信息数据、二维信息数据进行融合目标检测，得到第二三维检测框，并对第二三维检测框进行坐标变换，得到第二二维检测框，再对每个物体对应的第一二维检测 框和第二二维检测框进行IoU数值计算，得到第一数组，对每个物体对应的第一三维检测框和第二三维检测框进行IoU数值计算，得到第二数组，分析第一数组和和第二数组确认是否传感器是否被攻击，其利用三维传感器和二维传感器在时间和空间上的相关性进行攻击检测，从而提高攻击检测的准确性，并且利用作为自动驾驶汽车的基本配置的三维传感器和二维传感器即能够实现攻击检测，不再局限于利用其中一种传感器为主的目标检测算法进行攻击检测，具有更高的普适性。
图4是本发明实施例的自动驾驶系统的攻击检测装置的功能模块示意图。如图4所示，该自动驾驶系统的攻击检测装置20包括获取模块21、第一检测模块22、第二检测模块23、计算模块24和分析模块25。
获取模块21，用于分别利用三维传感器、二维传感器分别获取目标区域的三维信息数据、二维信息数据；
第一检测模块22，用于将三维信息数据、二维信息数据分别输入预先训练好的三维检测模型、二维检测模型，得到目标区域中的每个目标物体的第一三维检测框、第一二维检测框；
第二检测模块23，用于将三维信息数据、二维信息数据输入预先训练好的融合检测模型，得到目标区域中的每个目标物体的第二三维检测框，并对第二三维检测框进行坐标系变化，得到第二二维检测框；
计算模块24，用于计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组，且计算每个目标物体对应的第一三维检测框和第二三维检测框的第二IoU数值，得到第二数组；
分析模块25，用于基于第一数组和第二数组进行攻击检测并定位被攻击的传感器。
可选地，第二检测模块23执行将三维信息数据、二维信息数据输入预先训练好的融合检测模型的操作之前，还用于：将上一时刻的二维 信息数据和当前时刻的二维信息数据进行卡方比较，得到卡方比较数值；判断卡方比较数值是否超过预设阈值；当卡方比较数值超过预设阈值时，将当前时刻的二维信息数据与上一时刻的二维信息数据进行直方图匹配，得到当前时刻的增强二维信息数据，并替换掉当前时刻的二维信息数据；当卡方比较数值未超过预设阈值时，维持当前时刻的二维信息数据。
可选地，第二检测模块23执行对第二三维检测框进行坐标系变化，得到第二二维检测框的操作，具体包括：获取第二三维检测框的三维中心点坐标；利用三维中心点坐标确认第二三维检测框上与三维中心点在同一平面的四个点的齐次坐标；利用预先获取的相机投影矩阵、相机旋转矩阵、传感器到相机坐标系的旋转矩阵和齐次坐标计算得到四个二维坐标点；利用四个二维坐标点构建第二二维检测框。
可选地，计算模块24执行计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组的操作，具体包括：确认每个目标物体对应的目标第一二维检测框和目标第二二维检测框；分别计算目标第一二维检测框、目标第二二维检测框的第一面积、第二面积，以及目标第一二维检测框与目标第二二维检测框的重叠区域的第三面积；利用第一面积、第二面积、第三面积计算得到每个目标物体对应的第一IoU数值；利用所有目标物体对应的第一IoU数值构建第一数组。
可选地，计算模块24执行计算每个目标物体对应的第一三维检测框和第二三维检测框的第二IoU数值，得到第二数组的操作，具体包括：确认每个目标物体对应的目标第一三维检测框和目标第二三维检测框；分别计算目标第一三维检测框、目标第二三维检测框的第一体积、第二体积；计算目标第一三维检测框与目标第二三维检测框的重叠区域的底 面面积；确认目标第一三维检测框与目标第二三维检测框的重叠区域的高度；利用底面面积和高度计算得到重叠区域的第三体积；利用第一体积、第二体积、第三体积计算得到每个目标物体对应的第二IoU数值；利用所有目标物体对应的第二IoU数值构建第二数组。
可选地，分析模块25执行基于第一数组和第二数组进行攻击检测并定位被攻击的传感器的操作，具体包括：分别判断第一数组、第二数组是否存在离群值；当第一数组存在离群值、第二数组不存在离群值时，确认三维传感器被攻击；当第一数组不存在离群值、第二数组存在离群值时，确认二维传感器被攻击；当第一数组、第二数组均存在离群值时，确认三维传感器和/或二维传感器被攻击；当第一数组、第二数组均不存在离群值时，确认三维传感器和二维传感器未受到攻击。
可选地，三维传感器包括激光雷达，二维传感器包括摄像头，三维检测模型基于PointPillar算法构建，二维检测模型基于YOLOv3算法构建，融合检测模型基于AVOD算法构建。
关于上述实施例自动驾驶系统的攻击检测装置中各模块实现技术方案的其他细节，可参见上述实施例中的自动驾驶系统的攻击检测方法中的描述，此处不再赘述。
需要说明的是，本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。对于装置类实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
请参阅图5，图5为本发明实施例的计算机设备的结构示意图。如图5所示，该计算机设备30包括处理器31及和处理器31耦接的存储器32，存储器32中存储有程序指令，程序指令被处理器31执行时，使 得处理器31执行上述任一实施例所述的自动驾驶系统的攻击检测方法步骤。
其中，处理器31还可以称为CPU(Central Processing Unit，中央处理单元)。处理器31可能是一种集成电路芯片，具有信号的处理能力。处理器31还可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现场可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
参阅图6，图6为本发明实施例的存储介质的结构示意图。本发明实施例的存储介质存储有能够实现上述自动驾驶系统的攻击检测方法的程序指令41，其中，该程序指令41可以以软件产品的形式存储在上述存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施方式所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质，或者是计算机、服务器、手机、平板等计算机设备。
在本申请所提供的几个实施例中，应该理解到，所揭露的计算机设备，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。以上仅为本申请的实施方式，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (10)

  1. 一种自动驾驶系统的攻击检测方法，其特征在于，其包括：
    分别利用三维传感器、二维传感器分别获取所述目标区域的三维信息数据、二维信息数据；
    将所述三维信息数据、所述二维信息数据分别输入预先训练好的三维检测模型、二维检测模型，得到所述目标区域中的每个目标物体的第一三维检测框、第一二维检测框；
    将所述三维信息数据、所述二维信息数据输入预先训练好的融合检测模型，得到所述目标区域中的每个目标物体的第二三维检测框，并对所述第二三维检测框进行坐标系变化，得到第二二维检测框；
    计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组，且计算每个目标物体对应的第一三维检测框和所述第二三维检测框的第二IoU数值，得到第二数组；
    基于所述第一数组和所述第二数组进行攻击检测并定位被攻击的传感器。
  2. 根据权利要求1所述的自动驾驶系统的攻击检测方法，其特征在于，所述将所述三维信息数据、所述二维信息数据输入预先训练好的融合检测模型之前，还包括：
    将上一时刻的二维信息数据和当前时刻的二维信息数据进行卡方比较，得到卡方比较数值；
    判断所述卡方比较数值是否超过预设阈值；
    当所述卡方比较数值超过预设阈值时，将当前时刻的二维信息数据与上一时刻的二维信息数据进行直方图匹配，得到当前时刻的增强二维信息数据，并替换掉当前时刻的二维信息数据；
    当所述卡方比较数值未超过预设阈值时，维持当前时刻的二维信息数据。
  3. 根据权利要求1所述的自动驾驶系统的攻击检测方法，其特征在于，所述对所述第二三维检测框进行坐标系变化，得到第二二维检测框，包括：
    获取所述第二三维检测框的三维中心点坐标；
    利用所述三维中心点坐标确认所述第二三维检测框上与所述三维中心点在同一平面的四个点的齐次坐标；
    利用预先获取的相机投影矩阵、相机旋转矩阵、传感器到相机坐标系的旋转矩阵和所述齐次坐标计算得到四个二维坐标点；
    利用所述四个二维坐标点构建所述第二二维检测框。
  4. 根据权利要求1所述的自动驾驶系统的攻击检测方法，其特征在于，所述计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组，包括：
    确认每个目标物体对应的目标第一二维检测框和目标第二二维检测框；
    分别计算所述目标第一二维检测框、所述目标第二二维检测框的第一面积、第二面积，以及所述目标第一二维检测框与所述目标第二二维检测框的重叠区域的第三面积；
    利用所述第一面积、所述第二面积、所述第三面积计算得到每个目标物体对应的第一IoU数值；
    利用所有目标物体对应的第一IoU数值构建所述第一数组。
  5. 根据权利要求1所述的自动驾驶系统的攻击检测方法，其特征在于，所述计算每个目标物体对应的第一三维检测框和所述第二三维检测框的第二IoU数值，得到第二数组，包括：
    确认每个目标物体对应的目标第一三维检测框和目标第二三维检测框；
    分别计算所述目标第一三维检测框、所述目标第二三维检测框的第一体积、第二体积；
    计算所述目标第一三维检测框与所述目标第二三维检测框的重叠区域的底面面积；
    确认所述目标第一三维检测框与所述目标第二三维检测框的重叠区域的高度；
    利用所述底面面积和所述高度计算得到重叠区域的第三体积；
    利用所述第一体积、所述第二体积、所述第三体积计算得到每个目标物体对应的第二IoU数值；
    利用所有目标物体对应的第二IoU数值构建所述第二数组。
  6. 根据权利要求1所述的自动驾驶系统的攻击检测方法，其特征在于，所述基于所述第一数组和所述第二数组进行攻击检测并定位被攻击的传感器，包括：
    分别判断所述第一数组、所述第二数组是否存在离群值；
    当所述第一数组存在离群值、所述第二数组不存在离群值时，确认所述三维传感器被攻击；
    当所述第一数组不存在离群值、所述第二数组存在离群值时，确认所述二维传感器被攻击；
    当所述第一数组、所述第二数组均存在离群值时，确认所述三维传感器和/或所述二维传感器被攻击；
    当所述第一数组、所述第二数组均不存在离群值时，确认所述三维传感器和所述二维传感器未受到攻击。
  7. 根据权利要求1所述的自动驾驶系统的攻击检测方法，其特征 在于，所述三维传感器包括激光雷达，所述二维传感器包括摄像头，所述三维检测模型基于PointPillar算法构建，所述二维检测模型基于YOLOv3算法构建，所述融合检测模型基于AVOD算法构建。
  8. 一种自动驾驶系统的攻击检测装置，其特征在于，其包括：
    获取模块，用于分别利用三维传感器、二维传感器分别获取所述目标区域的三维信息数据、二维信息数据；
    第一检测模块，用于将所述三维信息数据、所述二维信息数据分别输入预先训练好的三维检测模型、二维检测模型，得到所述目标区域中的每个目标物体的第一三维检测框、第一二维检测框；
    第二检测模块，用于将所述三维信息数据、所述二维信息数据输入预先训练好的融合检测模型，得到所述目标区域中的每个目标物体的第二三维检测框，并对所述第二三维检测框进行坐标系变化，得到第二二维检测框；
    计算模块，用于计算每个目标物体对应的第一二维检测框与第二二维检测框的第一IoU数值，得到第一数组，且计算每个目标物体对应的第一三维检测框和所述第二三维检测框的第二IoU数值，得到第二数组；
    分析模块，用于基于所述第一数组和所述第二数组进行攻击检测并定位被攻击的传感器。
  9. 一种计算机设备，其特征在于，所述计算机设备包括处理器、与所述处理器耦接的存储器，所述存储器中存储有程序指令，所述程序指令被所述处理器执行时，使得所述处理器执行如权利要求1-7中任一项权利要求所述的自动驾驶系统的攻击检测方法的步骤。
  10. 一种存储介质，其特征在于，存储有能够实现如权利要求1-7中任一项所述的自动驾驶系统的攻击检测方法的程序指令。
PCT/CN2023/137634 2023-05-23 2023-12-08 自动驾驶系统的攻击检测方法、装置、设备及存储介质 Ceased WO2024239605A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310584658.0A CN116912788B (zh) 2023-05-23 2023-05-23 自动驾驶系统的攻击检测方法、装置、设备及存储介质
CN202310584658.0 2023-05-23

Publications (1)

Publication Number Publication Date
WO2024239605A1 true WO2024239605A1 (zh) 2024-11-28

Family

ID=88357100

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/137634 Ceased WO2024239605A1 (zh) 2023-05-23 2023-12-08 自动驾驶系统的攻击检测方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN116912788B (zh)
WO (1) WO2024239605A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120808296A (zh) * 2025-09-08 2025-10-17 罗博网联(杭州)信息技术有限公司 基于多模态融合的目标检测方法、装置、设备及存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116912788B (zh) * 2023-05-23 2026-04-14 深圳先进技术研究院 自动驾驶系统的攻击检测方法、装置、设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626217A (zh) * 2020-05-28 2020-09-04 宁波博登智能科技有限责任公司 一种基于二维图片和三维点云融合的目标检测和追踪方法
CN113111978A (zh) * 2021-06-11 2021-07-13 之江实验室 一种基于点云和图像数据的三维目标检测系统和方法
CN114187579A (zh) * 2021-12-14 2022-03-15 智道网联科技(北京)有限公司 自动驾驶的目标检测方法、装置和计算机可读存储介质
CN114596358A (zh) * 2022-03-03 2022-06-07 深圳一清创新科技有限公司 一种物体检测方法、装置及电子设备
CN115453589A (zh) * 2022-08-19 2022-12-09 中国科学院深圳先进技术研究院 基于自动驾驶的攻击检测方法、终端设备以及存储介质
WO2023277998A2 (en) * 2021-04-26 2023-01-05 Neural Propulsion Systems, Inc. Moving aperture lidar
CN116912788A (zh) * 2023-05-23 2023-10-20 深圳先进技术研究院 自动驾驶系统的攻击检测方法、装置、设备及存储介质

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111369589B (zh) * 2020-02-26 2022-04-22 桂林电子科技大学 一种基于多策略融合的无人机跟踪方法
CN112927211B (zh) * 2021-03-09 2023-08-25 电子科技大学 一种基于深度三维检测器的通用对抗攻击方法、存储介质和终端
CN113156440B (zh) * 2021-04-27 2024-03-26 浙江工业大学 一种基于雷达和图像数据融合检测的防御方法及系统
CN113532499B (zh) * 2021-07-15 2022-08-30 中国科学院深圳先进技术研究院 无人系统的传感器安全性检测方法、设备及存储介质
CN114359891B (zh) * 2021-12-08 2024-05-28 华南理工大学 一种三维车辆检测方法、系统、装置及介质
CN114463736B (zh) * 2021-12-28 2024-11-05 天津大学 一种基于多模态信息融合的多目标检测方法及装置
CN115527074B (zh) * 2022-11-29 2023-03-07 深圳依时货拉拉科技有限公司 一种车辆检测框的生成方法、生成装置及计算机设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111626217A (zh) * 2020-05-28 2020-09-04 宁波博登智能科技有限责任公司 一种基于二维图片和三维点云融合的目标检测和追踪方法
WO2023277998A2 (en) * 2021-04-26 2023-01-05 Neural Propulsion Systems, Inc. Moving aperture lidar
CN113111978A (zh) * 2021-06-11 2021-07-13 之江实验室 一种基于点云和图像数据的三维目标检测系统和方法
CN114187579A (zh) * 2021-12-14 2022-03-15 智道网联科技(北京)有限公司 自动驾驶的目标检测方法、装置和计算机可读存储介质
CN114596358A (zh) * 2022-03-03 2022-06-07 深圳一清创新科技有限公司 一种物体检测方法、装置及电子设备
CN115453589A (zh) * 2022-08-19 2022-12-09 中国科学院深圳先进技术研究院 基于自动驾驶的攻击检测方法、终端设备以及存储介质
CN116912788A (zh) * 2023-05-23 2023-10-20 深圳先进技术研究院 自动驾驶系统的攻击检测方法、装置、设备及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120808296A (zh) * 2025-09-08 2025-10-17 罗博网联(杭州)信息技术有限公司 基于多模态融合的目标检测方法、装置、设备及存储介质

Also Published As

Publication number Publication date
CN116912788B (zh) 2026-04-14
CN116912788A (zh) 2023-10-20

Similar Documents

Publication Publication Date Title
CN111563415B (zh) 一种基于双目视觉的三维目标检测系统及方法
CN112785702B (zh) 一种基于2d激光雷达和双目相机紧耦合的slam方法
US12094226B2 (en) Simultaneous localization and mapping method, device, system and storage medium
WO2022188663A1 (zh) 一种目标检测方法及装置
CN114898313B (zh) 驾驶场景的鸟瞰图生成方法、装置、设备及存储介质
CN111665842B (zh) 一种基于语义信息融合的室内slam建图方法及系统
US12315165B2 (en) Object detection method, object detection device, terminal device, and medium
JP5430456B2 (ja) 幾何特徴抽出装置、幾何特徴抽出方法、及びプログラム、三次元計測装置、物体認識装置
Saha et al. 3D LiDAR-based obstacle detection and tracking for autonomous navigation in dynamic environments: A. Saha, BC Dhara
CN114419098A (zh) 基于视觉变换的运动目标轨迹预测方法及装置
Song et al. End-to-end learning for inter-vehicle distance and relative velocity estimation in ADAS with a monocular camera
WO2024239605A1 (zh) 自动驾驶系统的攻击检测方法、装置、设备及存储介质
CN114088081A (zh) 一种基于多段联合优化的用于精确定位的地图构建方法
US20200226392A1 (en) Computer vision-based thin object detection
CN114648639B (zh) 一种目标车辆的检测方法、系统及装置
US12283120B2 (en) Method for detecting three-dimensional objects in relation to autonomous driving and electronic device
KR20190060679A (ko) 이동 객체의 자세 학습 방법 및 장치
Li et al. Indoor layout estimation by 2d lidar and camera fusion
JP2018073308A (ja) 認識装置、プログラム
Syntakas et al. Object detection and navigation of a mobile robot by fusing laser and camera information
Vatavu et al. Modeling and tracking of dynamic obstacles for logistic plants using omnidirectional stereo vision
Barth et al. Vehicle tracking at urban intersections using dense stereo
US12296488B2 (en) Method and system to detect and estimate dynamic obstacles using RGB-D sensors for robot navigation
CN118644551A (zh) 一种基于多模态的无人车跟踪运动目标的位姿估计方法
CN117789193A (zh) 基于二次增强的多模态数据融合3d目标检测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23938272

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE