WO2023220977A1 - Method and device for detecting data - Google Patents
Method and device for detecting data Download PDFInfo
- Publication number
- WO2023220977A1 WO2023220977A1 PCT/CN2022/093636 CN2022093636W WO2023220977A1 WO 2023220977 A1 WO2023220977 A1 WO 2023220977A1 CN 2022093636 W CN2022093636 W CN 2022093636W WO 2023220977 A1 WO2023220977 A1 WO 2023220977A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- block
- blocks
- abnormal
- data
- module
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 51
- 230000002159 abnormal effect Effects 0.000 claims abstract description 58
- 230000015654 memory Effects 0.000 claims description 40
- 238000010801 machine learning Methods 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 15
- 238000013461 design Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 9
- 238000001514 detection method Methods 0.000 description 6
- 230000001360 synchronised effect Effects 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- GSFCOAGADOGIGE-UHFFFAOYSA-N 2-amino-2-methyl-3-phosphonooxypropanoic acid Chemical compound OC(=O)C(N)(C)COP(O)(O)=O GSFCOAGADOGIGE-UHFFFAOYSA-N 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/93—Lidar systems specially adapted for specific applications for anti-collision purposes
- G01S17/931—Lidar systems specially adapted for specific applications for anti-collision purposes of land vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4808—Evaluating distance, position or velocity data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/497—Means for monitoring or calibrating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
Definitions
- Embodiments of the present application relate to the field of information technologies, and more specifically, to a methodand a device fordetecting data.
- LiDAR light detection and ranging
- IR Infrared
- one of the anomaly detection methods is to compare information of a prior frame datawith that of current frame data. Since a difference of information between two pieces of consecutive frame data is large, there are limitations in the methods of anomaly detection.
- Embodiments of this application provide a method and a device fordetecting data, which can improve safety of transport.
- the transport may include one or more different types of conveyances or movable objects operating or moving on land (e.g., a highway, a road, a railway) , water surface (e.g., a waterway, a river, an ocean) or space.
- land e.g., a highway, a road, a railway
- water surface e.g., a waterway, a river, an ocean
- the transport mayinclude vehicles, bicycles, motorcycles, trains, subways, planes, ships, aircraft, robots or other types of conveyances or movable objects.
- an embodiment of this application provides a method for detecting data, characterized in including: obtaining, frame data, where the frame data includes a snapshot of point cloud data during a preset duration; dividing, the frame data into a plurality of blocks; extracting, feature value of each block of the plurality of blocks, wherein the feature value represents statistical characteristics of points in the each block; and determining, whether the each block is abnormal or normal according to changes of feature values of the plurality of blocks.
- the above-mentioned technical solution provides a method to detect abnormaldata generated by a forgery attack on point cloud data collected from LiDAR sensor.
- the method can detect anomalies, such as inserting a fake object into the point cloud data or deleting real objects in the point cloud data. By detecting abnormal data, the method can help transport make a safe decision.
- Embodiments of the present application provide a reasonable area where the forgery occurred by analyzing the point cloud into blocks.
- the frame data includes N point clouds, where N is greater than or equal to 1 and N is a positive integer.
- the determining whether the each block is abnormal or normal according to the changes of the feature values of the plurality of blocks includes: inputting, the changes of the feature value into a machine learning model for obtaining a prediction result which indicates whether the each block is abnormal or normal, wherein the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
- the above-mentioned technical solution provides a machine learning model that learns a difference of block-wise time series of statistical values between a normal state and an abnormalstate.
- the machine learning model also determines whether the each block is abnormal or normal, which will help to improve intelligence of the transport and also help to improve accuracy of determination of abnormal blocks.
- the feature value comprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
- the method further includes: prompting, a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality of blocks includes the first block; and/or, executing, safety instructions if a second block is abnormal, wherein the plurality of blocks includes the second block.
- the user can be prompted and/or safety instructions can be executed so that the user can be aware of the abnormal data in time, which will help to improve driving safety of the user.
- the method is performed by a device located in a cloud server.
- the obtaining frame data includes: receiving, the frame data of a vehicle; andthe method further includes: sending an indication to the vehicle, wherein the indication is used to indicate whether the each block is abnormal.
- an embodiment of this application provides a devicefor detecting data, characterized in including: an obtaining module, configured to obtain a frame data, where the frame data contains snapshot of point cloud data at a preset duration; a dividing module, configured to divide the frame data into plurality blocks; an extracting module, configured to extract feature value of each block of the plurality blocks, wherein the feature value represents statistical characteristics of points in the each block; and a determining module, configured to determine whether the each block is abnormal or normal according to change of the feature values of the plurality blocks.
- the determining module is specifically configured to input the changes of the feature value into a machine learning model for obtaining a prediction result which indicates whether the each block is abnormal or not, where the machine learning model is trained by training samples, and the training samples include differences of feature values of abnormal blocks and change of feature values of normal blocks.
- the feature value comprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
- the device further includes: a prompting module, configured to prompt a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality blocks includes the first block; and/or, an executing module, configured to execute safety instructions if a second block is abnormal, wherein the plurality blocks includes the second block.
- a prompting module configured to prompt a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality blocks includes the first block
- an executing module configured to execute safety instructions if a second block is abnormal, wherein the plurality blocks includes the second block.
- the device is located in a cloud server.
- the obtaining module is specifically configured to receive the frame data of a vehicle; and the device further includes: a sending module, configured to send an indication to the vehicle, wherein the indication is used to indicate whether the each block is abnormal.
- an embodiment of this application provides transport, and the transport has a function of implementing the method in the first aspect.
- the function may be implemented by hardware, or may be implemented by hardware executing corresponding software.
- the hardware executing the corresponding software includes one or more modules corresponding to the function.
- the transport is a vehicle.
- adevice including a processor and a memory.
- the processor is connected to the memory.
- the memory is configured to store instructions, and the processor is configured to execute the instructions.
- the processor executes the instructions stored in the memory, the processor is enabled to perform the method in the first aspector any possible implementation of the first aspect.
- a chip system includes a memory and a processor, the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that a server on which the chip system is disposed performs the method in the first aspect or any possible implementation of the first aspect.
- a computer program product is provided, where when the computer program product is run on adevice, the deviceis enabled to perform the method in the first aspect or any possible implementation of the first aspect.
- FIG. 1 is a schematic block diagram of a vehicle.
- FIG. 2 is a schematic block diagram of anin-vehicle network topology.
- FIG. 3 is a structure of aminiature small outline package (MSOP) .
- FIG. 4 is a schematic block diagram of anin-vehicle network topology with an attacker.
- FIG. 5 is a schematic block diagram of anin-vehicle network topology with an attacker.
- FIG. 6 is a schematic block diagram of a system.
- FIG. 7 shows a point cloud
- FIG. 8 showsthat a point cloud is divided into 5 blocks.
- FIG. 9 shows a graphical user interface (GUI) .
- FIG. 10 is a schematic block diagram of a system.
- FIG. 11 isa flowchart of a method for detecting data.
- FIG. 12 is a schematic block diagram of a device for detecting data.
- FIG. 13 is a schematic block diagram of a device for detecting data.
- FIG. 1 is a schematic block diagram of a vehicle 100.
- the vehicle 100 may include a sensing system 110, a display device 120, and a computing platform 130, where the sensing system 110 may include several kinds of sensors that sense information about the environment around the vehicle 100.
- the sensing system 110 may include a positioning system, which may be a global positioning system (GPS) , a Beidou system or other positioning systems.
- the sensing system 110 further includes an inertial measurement unit (IMU) , LiDARsensor, a millimeter wave radar, anultrasonic radar or imaging devices.
- IMU inertial measurement unit
- the computing platform 130 may include processors 131 to 13n (n is a positive integer) .
- the processor is a circuit with the capability of signal processing.
- the processor may be a circuit with the capability of reading and running instructions, such as a central processing unit (CPU) , a microprocessor, a graphics processing unit (GPU) , or a digital signal processor (DSP) .
- CPU central processing unit
- microprocessor a graphics processing unit
- DSP digital signal processor
- the processor can realize certain functions through a logical relationship of a hardware circuit, which is fixed or reconfigurable.
- the processor is a hardware circuit realized by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) , such as a field programmable gate array (FPGA) .
- ASIC application-specific integrated circuit
- PLD programmable logic device
- FPGA field programmable gate array
- the process, during which the processor loads a configuration document to realize configuration of the hardware circuit can be understood as a process during which the processor loads instructions to realize the functions of some or all of the above units.
- the processor can also be a hardware circuit designed for artificial intelligence (AI) , which can be understood as an ASIC, such as a neural network processing unit (NPU) , a tensor processing unit (TPU) , a deep learning processing unit (DPU) .
- AI artificial intelligence
- the computing platform 130 may further include a memory for storing instructions, and some or all of the processors 131 to 13n may call the instructions in the memory and execute the instructions to realize the corresponding functions.
- LiDARsensor acts as an eye of self-driving vehicles providing them a 360-degree view.
- Autonomous vehicles may use LiDARsensor for obstacle detection and avoidance to navigate safely through environments.
- FIG. 2 is a schematic block diagram of anin-vehicle network topology 200 with a LiDAR sensor.
- the in-vehicle network topology 200 includes four nodes: LiDAR sensors 210, an in-vehicle network switch 220, a preprocessing module 230, and a perception module 240.
- the function of the preprocessing module 230 can be realized by a CPU and an AI chip.
- the CPU may segment the collected point cloud data, extract features, and obtain time sequence change features of a plurality of blocks. Finally, the time sequence change features are input to the AI chip, and the AI chip outputs a result of whether each block is abnormal.
- the LiDAR sensors 210 generate and transmit raw sensor data to the preprocessing module 230 via the in-vehicle network switch 220.
- the raw sensor data consists of R, ⁇ values, which means the distance between the point and the LiDAR sensor, and the azimuth angle of the point, respectively.
- a vertical angle ( ⁇ ) and a horizontal angle offset ( ⁇ ) are pre-defined constant values set by a user.
- FIG. 3 is a structure of aminiature small outline package (MSOP) , which is one of LiDAR packets. As the location of each data is determined as shown in FIG. 3, an attacker can easily change the raw information by using a point address value.
- MSOP aminiature small outline package
- FIG. 4 is a schematic block diagram of anin-vehicle network topology with an attacker.
- an attacker is positioned between the in-vehicle network switch 220 and the LiDAR sensors 210, he can easily modify points. It can be assumed that the attacker is a man in the middle attack (MITM) using address resolution protocol (ARP) spoofing or network tapping that can physically access the network cable through attacker’s device.
- MITM man in the middle attack
- ARP address resolution protocol
- object insertion can be performed by modifying the raw sensor data with the values of the object pointsthe attacker has. Then, as shown in FIG. 5, an object (vehicle) that was not in the actual situation comes out as an object (vehicle) on the perception module240.
- Embodiments of this application provide a method for detecting data and related devices.
- the technical solution can identify point cloud modifying attacks in order to avoid transport to make a wrong decision.
- FIG. 6 is a schematic block diagram of a system 600.
- the system 600 includes a dividing module 610, a feature extracting module 620, a time series managing module 630, amachine learning model 640 and an attack detecting module 650.
- system 600 is included in the above preprocessing module 230.
- the system 600 detects abnormal changes of statistical values of points in a series of point clouds. To calculate statistical values of points, the system divides a point cloud into blocks with the same sizes. When an attack occurs, statistical values of points in the attacked block change abnormally with time.
- the dividing module 610 receives point cloud data from LiDAR sensors.
- the LiDAR sensors scan a direction of and a distance to an object around an automotive vehicle to obtain LiDAR data, and the LiDAR data is formatted to obtain point cloud data. Also, the LiDAR data includes laser pulse reflection intensity from the object.
- FIG. 7 shows a point cloud.
- the dividing module 610 divides a point cloud into blocks. There are numerous points in one point cloud (usually, 10 to 100 thousand points) . Therefore, it is more effective to perform the processing after the point cloud is divided into blocks.
- FIG. 8 showsthat a point cloud is divided into 5 blocks.
- FIG. 8 is for illustration only to show how it looks when the point cloud is divided into 5 blocks.
- the size of each block is depending on a developer. A reasonable size of the each block is 2m x 2m x 1m, where m means meter.
- the raw data of LiDAR which consists of thousands of LiDAR’s laser light points is called as point cloud data.
- a frame is nothing but a snapshot of point cloud data during a preset duration. As the vehicle moves, the point cloud data changes and data in frames also changes. Each piece of frame data can be divided into a total of 3600 blocks.
- the feature extracting module 620 extracts features of points in the each block.
- the features of the points in the each block are values that can represents statistical characteristics of the points.
- the feature valueofthe each block comprises an average of coordinate values or a standard deviation of coordinate values.
- the average of coordinate valuesor the standard deviation of coordinate values shows the distribution of the points in the each block.
- the feature valueofthe each block comprisesan average of intensityor a standard deviation of intensity.
- the average of intensityor the standard deviation of intensity shows how many various surfaces of objects are in the each block.
- the time series managing module 630 generates time series of features of blocks in a length of a certain time window (e.g., 1 sec) .
- the time series contain feature values of the each block in recent N point clouds.
- the whole raw data of LiDAR is called as point cloud data which consists of thousands of Laser light points.
- the point cloud data is continuously updated every time the LiDAR scans its surrounding environment (for example once in every T milliseconds) .
- the snapshot of point cloud data during a preset duration is called as frame data.
- Each frame data can be divided into multiple blocks (3D) with the same size. Then, the data in the each block is monitored over a period of time. If any anomaly is detected within a block, then it is considered as the abnormal data.
- Consecutive feature values can represent changes of distribution and characteristics of points. If there is an attack, a suddenly appeared (or disappeared) object will significantly change the feature values of points in related blocks. Hence, the time series-based approach helps to detect anomalies in the point cloud
- the machine learning model 640 analyzes time series of features of blocks and automatically classifies the each block into a normal or attack class.
- the machine learning model 640 learns normal and abnormal states from a pre-collected dataset (which may contain normal and attacked sensor data or may contain only normal sensor data) . After training, the machine learning model 640 can determine whether the each block in the point cloud is attacked or not.
- the machine learning model 640 is included in an advanced driving assistant system (ADAS) .
- ADAS advanced driving assistant system
- the attack detecting module 650 receives inference results from the machine learning model 640. If there is an attacked block, the module alerts the autonomous driving system, driver, or system manager.
- FIG. 9 shows a graphical user interface (GUI) .
- a prompt message "the data collected by LiDAR is abnormal, please take over the vehicle" can be displayed on a display screen.
- the user can be prompted with a change of the color of an atmosphere lamp.
- the system 600 can be located in a server.
- FIG. 10 is a schematic block diagram of a system 1000.
- the system 1000 includes a vehicle 1010 and a server 1020.
- the vehicle 1010 includes LiDARsensors and the server 1020 includes the above system 600.
- the vehicle 1010 can send point cloud data collected by LiDARsensor to the server 1020, and the system 600 in the server 1020 can determining the each block is abnormal or normal by analyzing the point cloud data.
- the server 1020 can send a result of whether the each block is abnormal to the vehicle 1010. Then, the vehicle 1010 can perform corresponding operations according to the result.
- the proposed system 600 can identify point cloud modifying attacks, especially removing or adding an object in the heading direction of the autonomous vehicle in order to avoid the perception module to make a wrong decision.
- system 600 can detect anomalies raised by network attacks and the system 600 does not requires data from other sources.
- the system 600 works on block-wise regions instead of analyzing the entire point cloud. It is faster than analyzing point by point in the scene.
- FIG. 11 isa flowchart of a method 1100 for detecting data.
- the method can be executed by a device.
- the device is the above-mentioned computing platform, the transport or the server.
- a device obtains frame data.
- the frame data includes a snapshot of point cloud data during a preset duration.
- the frame data includes N point clouds, where N is greater than or equal to 1 and N is a positive integer.
- the preset duration is 1 second.
- the device obtains the frame data, whichincludes: the device obtains a frame data from LiDAR sensors.
- the device dividesthe frame data into a plurality of blocks.
- the device dividesthe frame data into a plurality of blocks, which includes: the device dividesthe frame data into a plurality of blocks with a preset size.
- the preset size is 2m x 2m x 1m.
- the device extracts a feature value of each block of the plurality of blocks.
- the feature value represents statistical characteristics of points in the each block.
- the feature value comprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
- the average or the standard deviation of coordinate values shows the distribution of points in the block.
- the average or the standard deviation of intensity shows how many various surfaces of objects are in the block.
- the device determines whether the each block is abnormal or normal according to changes of feature values of the plurality of blocks.
- the determination of whether the each block is abnormal or normal according to the changes of the feature values of the plurality of blocks includes: inputting, by the device, the changes of the feature values into a machine learning model for obtaining a prediction result that indicates whether the each block is abnormal or normal, where the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
- the plurality of blocks are normal blocks. If thechanges of the feature values of the plurality of blocks match changes of feature values between abnormal blocks and normal blocks, the plurality of blocks includes abnormal blocks.
- FIG. 12 is a schematic block diagram of a device 1200 for detecting data according to an embodiment of this application.
- the device 1200 includes: an obtaining module 1210, a dividing module 1220, an extracting module 1230 and a determining module 1240.
- the obtaining module 1210 is configured to obtain frame data, where the frame data contains snapshot of point cloud data at a preset duration.
- the dividing module 1220 is configured to divide the frame data into a plurality of blocks.
- the extracting module 1230 is configured to extract feature value of each block of the plurality of blocks, wherein the feature value represents statistical characteristics of points in the each block.
- the determining module 1240 isconfigured to determine whether the each block is abnormal or normal according to changes of the feature values of the plurality of blocks.
- the determining module 1240 is specifically configured to input the changes of the feature value into a machine learning model for obtaining a prediction result that indicates whether the each block is abnormal or not, wherein the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
- the feature value comprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
- the device 1200 further includes: a prompting module, configured to prompt a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality of blocks includes the first block; and/or, an executing module, configured to execute safety instructions if a second block is abnormal, wherein the plurality of blocks includes the second block.
- a prompting module configured to prompt a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality of blocks includes the first block
- an executing module configured to execute safety instructions if a second block is abnormal, wherein the plurality of blocks includes the second block.
- the device 1200 is located in a cloud server.
- the obtaining module 1210 is specifically configured to receive frame data of a vehicle.
- the device 1200 further includes a sending moduleconfigured to send an indication to the vehicle, where the indication is used to indicate whether the each block is abnormal.
- a device 1300 may include a transceiver 1301, a processor 1302, and a memory 1303.
- the memory 1303 may be configured to store codes, instructions, and the like executed by the processor 1302.
- the device 1300 may be a computing platform and may execute the above method executed by the transport illustrated in FIG. 10.
- the device 1300 may be transport and may execute the above method executed by the transport illustrated in FIG. 10.
- the device 1300 may be a server and may execute the above method executed by the transport illustrated in FIG. 10.
- the processor 1302 may be an integrated circuit chip and has a signal processing capability. In an implementation process, steps of the foregoing method embodiments may be completed by using a hardware integrated logic circuit in the processor, or by using instructions in a form of software.
- the processor may be a general purpose processor, a digital signal processor (DSP) , an application-specific integrated circuit (ASIC) , a field programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component.
- DSP digital signal processor
- ASIC application-specific integrated circuit
- FPGA field programmable gate array
- the processor may implement or perform the methods, the steps, and the logical block diagrams that are disclosed in the embodiments of the present application.
- the general purpose processor may be a microprocessor, or the processor may be any conventional processor or the like.
- the steps of the methods disclosed with reference to the embodiments of the present application may be directly performed and completed by a hardware decoding processor, or may be performed and completed by using a combination of hardware in the decoding processor and a software module.
- the software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register.
- the storage medium is located in the memory, and the processor reads information in the memory and completes the steps of the foregoing methods in combination with hardware in the processor.
- the memory 1303 in the embodiments of the present application may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory.
- the nonvolatile memory may be a read-only memory (ROM) , a programmable read-only memory (PROM) , an erasable programmable read-only memory (Erasable PROM, EPROM) , an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) , or a flash memory.
- the volatile memory may be a random access memory (RAM) and is used as an external cache.
- RAMs may be used, and are, for example, a static random access memory (Static RAM, SRAM) , a dynamic random access memory (Dynamic RAM, DRAM) , a synchronous dynamic random access memory (Synchronous DRAM, SDRAM) , a double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM) , an enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM) , a synchronous link dynamic random access memory (Synchronous link DRAM, SLDRAM) , and a direct rambus random access memory (Direct Rambus RAM, DR RAM) .
- Static RAM Static RAM
- DRAM dynamic random access memory
- DRAM synchronous dynamic random access memory
- DDR SDRAM double data rate synchronous dynamic random access memory
- Enhanced SDRAM, ESDRAM enhanced synchronous dynamic random access memory
- SLDRAM synchronous link dynamic random access memory
- Direct Rambus RAM Direct Rambus RAM
- the memory in the systems and the methods described in this specification includes but is not limited to these memories and a memory of any other appropriate type.
- An embodiment of this application further provides a system chip, where the system chip includes an input/output interface, at least one processor, at least one memory, and a bus.
- the at least one memory is configured to store instructions
- the at least one processor is configured to invoke the instructions of the at least one memory to perform operations in the methods in the foregoing embodiments.
- An embodiment of this application further provides a computer storage medium, where the computer storage medium may store a program instruction for performing any of the foregoing methods.
- the storage medium may be specifically the memory 1303.
- the disclosed system, apparatus, and method may be implemented in other manners.
- the described apparatus embodiment is merely an example.
- the unit division is merely logical function division and may be other division in actual implementation.
- a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed.
- the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces.
- the indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.
- the units described as separate parts may be or may not be physically separate, and parts displayed as units may be or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.
- the functions When the functions are implemented in a form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer readable storage medium.
- the computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of this application.
- the foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM) , a random access memory (RAM) , a magnetic disk, or an optical disc.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Traffic Control Systems (AREA)
Abstract
A method and a device for detecting data, the method including: obtaining frame data, wherein the frame data includes a snapshot of point cloud data during a preset duration; dividing the frame data into a plurality of blocks; extracting feature value of each block of the plurality of blocks, wherein the feature value represents statistical characteristics of points in the each block; and determining whether the each block is abnormal or normal according to changes of feature values of the plurality of blocks. The embodiments of the present application can be applied to an intelligent vehicle or an electric vehicle, and can help the vehicle make a safe decision.
Description
Embodiments of the present application relate to the field of information technologies, and more specifically, to a methodand a device fordetecting data.
In modern vehicles, a light detection and ranging (LiDAR) sensor plays a crucial role in autonomous driving technologies to measure various information of other vehicles and objects. The LiDAR is a technology to measure a distance, a speed, a direction, and other characteristics of a target between the LiDAR sensor and an object using Infrared (IR) pulse signals. Moreover, while a vehicle equipped with the LiDAR sensor is being driven, the LiDAR sensor recognizes surrounding vehicles to secure safety of the vehicle. However, the LiDAR sensor has vulnerabilities that could be performable physically due to the technical characteristics.
Various anomaly detection methods of the LiDAR sensor have been proposed. For example, one of the anomaly detection methodsis to compare information of a prior frame datawith that of current frame data. Since a difference of information between two pieces of consecutive frame data is large, there are limitations in the methods of anomaly detection.
SUMMARY
Embodiments of this application provide a method and a device fordetecting data, which can improve safety of transport.
In a possible design, the transport may include one or more different types of conveyances or movable objects operating or moving on land (e.g., a highway, a road, a railway) , water surface (e.g., a waterway, a river, an ocean) or space. For example, the transport mayinclude vehicles, bicycles, motorcycles, trains, subways, planes, ships, aircraft, robots or other types of conveyances or movable objects.
According to a first aspect, an embodiment of this application provides a method for detecting data, characterized in including: obtaining, frame data, where the frame data includes a snapshot of point cloud data during a preset duration; dividing, the frame data into a plurality of blocks; extracting, feature value of each block of the plurality of blocks, wherein the feature value represents statistical characteristics of points in the each block; and determining, whether the each block is abnormal or normal according to changes of feature values of the plurality of blocks.
The above-mentioned technical solution provides a method to detect abnormaldata generated by a forgery attack on point cloud data collected from LiDAR sensor. The method can detect anomalies, such as inserting a fake object into the point cloud data or deleting real objects in the point cloud data. By detecting abnormal data, the method can help transport make a safe decision.
Embodiments of the present application provide a reasonable area where the forgery occurred by analyzing the point cloud into blocks.
In a possible design, the frame data includes N point clouds, where N is greater than or equal to 1 and N is a positive integer.
In a possible design, the determining whether the each block is abnormal or normal according to the changes of the feature values of the plurality of blocks, includes: inputting, the changes of the feature value into a machine learning model for obtaining a prediction result which indicates whether the each block is abnormal or normal, wherein the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
The above-mentioned technical solution providesa machine learning model that learns a difference of block-wise time series of statistical values between a normal state and an abnormalstate. The machine learning model also determines whether the each block is abnormal or normal, which will help to improve intelligence of the transport and also help to improve accuracy of determination of abnormal blocks.
In a possible design, the feature value comprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
In a possible design, the method further includes: prompting, a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality of blocks includes the first block; and/or, executing, safety instructions if a second block is abnormal, wherein the plurality of blocks includes the second block.
When the plurality of blocks include an abnormal block, the user can be prompted and/or safety instructions can be executed so that the user can be aware of the abnormal data in time, which will help to improve driving safety of the user.
In a possible design, the method is performed by a device located in a cloud server.
Since the computing power of the cloud server is stronger than that of the transport, a rate of abnormal data detection can be improved.
In a possible design, the obtaining frame data, includes: receiving, the frame data of a vehicle; andthe method further includes: sending an indication to the vehicle, wherein the indication is used to indicate whether the each block is abnormal.
According to a second aspect, an embodiment of this application provides a devicefor detecting data, characterized in including: an obtaining module, configured to obtain a frame data, where the frame data contains snapshot of point cloud data at a preset duration; a dividing module, configured to divide the frame data into plurality blocks; an extracting module, configured to extract feature value of each block of the plurality blocks, wherein the feature value represents statistical characteristics of points in the each block; anda determining module, configured to determine whether the each block is abnormal or normal according to change of the feature values of the plurality blocks.
In a possible design, the determining module is specifically configured to input the changes of the feature value into a machine learning model for obtaining a prediction result which indicates whether the each block is abnormal or not, where the machine learning model is trained by training samples, and the training samples include differences of feature values of abnormal blocks and change of feature values of normal blocks.
In a possible design, the feature valuecomprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
In a possible design, the device further includes: a prompting module, configured to prompt a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality blocks includes the first block; and/or, an executing module, configured to execute safety instructions if a second block is abnormal, wherein the plurality blocks includes the second block.
In a possible design, the device is located in a cloud server.
In a possible design, the obtaining module is specifically configured to receive the frame data of a vehicle; and the device further includes: a sending module, configured to send an indication to the vehicle, wherein the indication is used to indicate whether the each block is abnormal.
According to a thirdaspect, an embodiment of this application provides transport, and the transport has a function of implementing the method in the first aspect. The function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware executing the corresponding software includes one or more modules corresponding to the function.
In a possible design, the transport is a vehicle.
According to a fourth aspect, adevice is provided, including a processor and a memory. The processor is connected to the memory. The memory is configured to store instructions, and the processor is configured to execute the instructions. When the processor executes the instructions stored in the memory, the processor is enabled to perform the method in the first aspector any possible implementation of the first aspect.
According to anfifth aspect, a chip system is provided, where the chip system includes a memory and a processor, the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that a server on which the chip system is disposed performs the method in the first aspect or any possible implementation of the first aspect.
According to a sixth aspect, a computer program product is provided, where when the computer program product is run on adevice, the deviceis enabled to perform the method in the first aspect or any possible implementation of the first aspect.
DESCRIPTION OF DRAWINGS
FIG. 1 is a schematic block diagram of a vehicle.
FIG. 2is a schematic block diagram of anin-vehicle network topology.
FIG. 3is a structure of aminiature small outline package (MSOP) .
FIG. 4is a schematic block diagram of anin-vehicle network topology with an attacker.
FIG. 5is a schematic block diagram of anin-vehicle network topology with an attacker.
FIG. 6 is a schematic block diagram of a system.
FIG. 7 shows a point cloud.
FIG. 8 showsthat a point cloud is divided into 5 blocks.
FIG. 9 shows a graphical user interface (GUI) .
FIG. 10is a schematic block diagram of a system.
FIG. 11isa flowchart of a method for detecting data.
FIG. 12is a schematic block diagram of a device for detecting data.
FIG. 13is a schematic block diagram of a device for detecting data.
DESCRIPTION OF EMBODIMENTS
The following describes the technical solutions in this application with reference to the accompanying drawings.
FIG. 1 is a schematic block diagram of a vehicle 100.
As shown in FIG. 1, the vehicle 100 may include a sensing system 110, a display device 120, and a computing platform 130, where the sensing system 110 may include several kinds of sensors that sense information about the environment around the vehicle 100. For example, the sensing system 110 may include a positioning system, which may be a global positioning system (GPS) , a Beidou system or other positioning systems. The sensing system 110 further includes an inertial measurement unit (IMU) , LiDARsensor, a millimeter wave radar, anultrasonic radar or imaging devices.
Some or all of the functions of the vehicle 100 may be controlled by the computing platform 130. The computing platform 130 may include processors 131 to 13n (n is a positive integer) . The processor is a circuit with the capability of signal processing.
In some embodiments of the present application, the processor may be a circuit with the capability of reading and running instructions, such as a central processing unit (CPU) , a microprocessor, a graphics processing unit (GPU) , or a digital signal processor (DSP) .
In some embodiments of the present application, the processor can realize certain functions through a logical relationship of a hardware circuit, which is fixed or reconfigurable. For example, the processor is a hardware circuit realized by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) , such as a field programmable gate array (FPGA) . In the reconfigurable hardware circuit, the process, during which the processor loads a configuration document to realize configuration of the hardware circuit, can be understood as a process during which the processor loads instructions to realize the functions of some or all of the above units. In addition, the processor can also be a hardware circuit designed for artificial intelligence (AI) , which can be understood as an ASIC, such as a neural network processing unit (NPU) , a tensor processing unit (TPU) , a deep learning processing unit (DPU) . The computing platform 130 may further include a memory for storing instructions, and some or all of the processors 131 to 13n may call the instructions in the memory and execute the instructions to realize the corresponding functions.
LiDARsensor acts as an eye of self-driving vehicles providing them a 360-degree view. Autonomous vehicles may use LiDARsensor for obstacle detection and avoidance to navigate safely through environments.
FIG. 2is a schematic block diagram of anin-vehicle network topology 200 with a LiDAR sensor.
As shown in FIG. 2, the in-vehicle network topology 200includes four nodes: LiDAR sensors 210, an in-vehicle network switch 220, a preprocessing module 230, and a perception module 240.
In some embodiments of the present application, the function of the preprocessing module 230 can be realized by a CPU and an AI chip. The CPU may segment the collected point cloud data, extract features, and obtain time sequence change features of a plurality of blocks. Finally, the time sequence change features are input to the AI chip, and the AI chip outputs a result of whether each block is abnormal.
The LiDAR sensors 210 generate and transmit raw sensor data to the preprocessing module 230 via the in-vehicle network switch 220. The raw sensor data consists of R, α values, which means the distance between the point and the LiDAR sensor, and the azimuth angle of the point, respectively. A vertical angle (ω) and a horizontal angle offset (δ) are pre-defined constant values set by a user.
FIG. 3is a structure of aminiature small outline package (MSOP) , which is one of LiDAR packets. As the location of each data is determined as shown in FIG. 3, an attacker can easily change the raw information by using a point address value.
FIG. 4is a schematic block diagram of anin-vehicle network topology with an attacker.
As shown in FIG. 4, if an attacker is positioned between the in-vehicle network switch 220 and the LiDAR sensors 210, he can easily modify points. It can be assumed that the attacker is a man in the middle attack (MITM) using address resolution protocol (ARP) spoofing or network tapping that can physically access the network cable through attacker’s device.
If an attacker exists at the location as shown in FIG. 5, object insertion can be performed by modifying the raw sensor data with the values of the object pointsthe attacker has. Then, as shown in FIG. 5, an object (vehicle) that was not in the actual situation comes out as an object (vehicle) on the perception module240.
Embodiments of this application provide a method for detecting data and related devices. The technical solution can identify point cloud modifying attacks in order to avoid transport to make a wrong decision.
FIG. 6 is a schematic block diagram of a system 600. The system 600 includes a dividing module 610, a feature extracting module 620, a time series managing module 630, amachine learning model 640 and an attack detecting module 650.
In some embodiments of the present application, the system 600 is included in the above preprocessing module 230.
The system 600detects abnormal changes of statistical values of points in a series of point clouds. To calculate statistical values of points, the system divides a point cloud into blocks with the same sizes. When an attack occurs, statistical values of points in the attacked block change abnormally with time.
The dividing module 610receives point cloud data from LiDAR sensors. The LiDAR sensors scan a direction of and a distance to an object around an automotive vehicle to obtain LiDAR data, and the LiDAR data is formatted to obtain point cloud data. Also, the LiDAR data includes laser pulse reflection intensity from the object.
FIG. 7shows a point cloud. The dividing module 610 divides a point cloud into blocks. There are numerous points in one point cloud (usually, 10 to 100 thousand points) . Therefore, it is more effective to perform the processing after the point cloud is divided into blocks.
FIG. 8 showsthat a point cloud is divided into 5 blocks.
FIG. 8 is for illustration only to show how it looks when the point cloud is divided into 5 blocks. The size of each block is depending on a developer. A reasonable size of the each block is 2m x 2m x 1m, where m means meter. The raw data of LiDAR which consists of thousands of LiDAR’s laser light points is called as point cloud data. A frame is nothing but a snapshot of point cloud data during a preset duration. As the vehicle moves, the point cloud data changes and data in frames also changes. Each piece of frame data can be divided into a total of 3600 blocks.
The feature extracting module 620 extracts features of points in the each block. The features of the points in the each block are values that can represents statistical characteristics of the points.
In some embodiments of the present application, the feature valueofthe each block comprises an average of coordinate values ora standard deviation of coordinate values. The average of coordinate valuesor the standard deviation of coordinate values shows the distribution of the points in the each block.
In some embodiments of the present application, the feature valueofthe each block comprisesan average of intensityor a standard deviation of intensity. The average of intensityor the standard deviation of intensity shows how many various surfaces of objects are in the each block.
The time series managing module 630 generates time series of features of blocks in a length of a certain time window (e.g., 1 sec) . The time series contain feature values of the each block in recent N point clouds. The whole raw data of LiDAR is called as point cloud data which consists of thousands of Laser light points. The point cloud data is continuously updated every time the LiDAR scans its surrounding environment (for example once in every T milliseconds) . The snapshot of point cloud data during a preset duration is called as frame data. Each frame data can be divided into multiple blocks (3D) with the same size. Then, the data in the each block is monitored over a period of time. If any anomaly is detected within a block, then it is considered as the abnormal data. Consecutive feature values can represent changes of distribution and characteristics of points. If there is an attack, a suddenly appeared (or disappeared) object will significantly change the feature values of points in related blocks. Hence, the time series-based approach helps to detect anomalies in the point cloud.
The machine learning model 640 analyzes time series of features of blocks and automatically classifies the each block into a normal or attack class. The machine learning model 640 learns normal and abnormal states from a pre-collected dataset (which may contain normal and attacked sensor data or may contain only normal sensor data) . After training, the machine learning model 640 can determine whether the each block in the point cloud is attacked or not.
In some embodiments of the present application, the machine learning model 640 is included in an advanced driving assistant system (ADAS) .
The attack detecting module 650 receives inference results from the machine learning model 640. If there is an attacked block, the module alerts the autonomous driving system, driver, or system manager.
FIG. 9 shows a graphical user interface (GUI) .
As shown in FIG. 9, when the vehicle is in an automatic driving state, if the system 600 determines that a point cloud is abnormal, a prompt message "the data collected by LiDAR is abnormal, please take over the vehicle" can be displayed on a display screen.
In some embodiments of the present application, the user can be prompted with a change of the color of an atmosphere lamp.
In some embodiments of the present application, the system 600 can be located in a server.
FIG. 10is a schematic block diagram of a system 1000.
As shown in FIG. 10, the system 1000 includesa vehicle 1010 and a server 1020. The vehicle 1010 includes LiDARsensors and the server 1020 includes the above system 600. The vehicle 1010 can send point cloud data collected by LiDARsensor to the server 1020, and the system 600 in the server 1020 can determining the each block is abnormal or normal by analyzing the point cloud data. The server 1020 can send a result of whether the each block is abnormal to the vehicle 1010. Then, the vehicle 1010 can perform corresponding operations according to the result.
The proposed system 600 can identify point cloud modifying attacks, especially removing or adding an object in the heading direction of the autonomous vehicle in order to avoid the perception module to make a wrong decision.
In addition, the system 600 can detect anomalies raised by network attacks and the system 600 does not requires data from other sources.
From a practical perspective, the system 600 works on block-wise regions instead of analyzing the entire point cloud. It is faster than analyzing point by point in the scene.
FIG. 11isa flowchart of a method 1100 for detecting data. The method can be executed by a device. For instance, the device is the above-mentioned computing platform, the transport or the server.
1101, a device obtains frame data. The frame data includes a snapshot of point cloud data during a preset duration.
Optionally, the frame data includes N point clouds, where N is greater than or equal to 1 and N is a positive integer.
For instance, the preset duration is 1 second.
Optionally, the device obtains the frame data, whichincludes: the device obtains a frame data from LiDAR sensors.
1102, the device dividesthe frame data into a plurality of blocks.
Optionally, the device dividesthe frame data into a plurality of blocks, which includes: the device dividesthe frame data into a plurality of blocks with a preset size.
For instance, the preset size is 2m x 2m x 1m.
1103, the device extracts a feature value of each block of the plurality of blocks. The feature value represents statistical characteristics of points in the each block.
Optionally, the feature valuecomprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
For instance, the average or the standard deviation of coordinate values shows the distribution of points in the block. The average or the standard deviation of intensity shows how many various surfaces of objects are in the block.
1104, the devicedetermines whether the each block is abnormal or normal according to changes of feature values of the plurality of blocks.
Optionally, the determination of whether the each block is abnormal or normal according to the changes of the feature values of the plurality of blocks, includes: inputting, by the device, the changes of the feature values into a machine learning model for obtaining a prediction result that indicates whether the each block is abnormal or normal, where the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
Optionally, if the changes of the feature values of the plurality of blocks match changes of feature values of normal blocks, the plurality of blocks are normal blocks. If thechanges of the feature values of the plurality of blocks match changes of feature values between abnormal blocks and normal blocks, the plurality of blocks includes abnormal blocks.
FIG. 12is a schematic block diagram of a device 1200 for detecting data according to an embodiment of this application. As shown in FIG. 12, the device 1200 includes: an obtaining module 1210, a dividing module 1220, an extracting module 1230 and a determining module 1240.
The obtaining module 1210 is configured to obtain frame data, where the frame data contains snapshot of point cloud data at a preset duration.
The dividing module 1220 is configured to divide the frame data into a plurality of blocks.
The extracting module 1230 is configured to extract feature value of each block of the plurality of blocks, wherein the feature value represents statistical characteristics of points in the each block.
The determining module 1240 isconfigured to determine whether the each block is abnormal or normal according to changes of the feature values of the plurality of blocks.
Optionally, the determining module 1240 is specifically configured to input the changes of the feature value into a machine learning model for obtaining a prediction result that indicates whether the each block is abnormal or not, wherein the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
Optionally, the feature valuecomprises one or more of: an average of coordinate values; a standard deviation of coordinate values; an averageof intensity; and a standard deviation of intensity.
Optionally, the device 1200 further includes: a prompting module, configured to prompt a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality of blocks includes the first block; and/or, an executing module, configured to execute safety instructions if a second block is abnormal, wherein the plurality of blocks includes the second block.
Optionally, the device 1200 is located in a cloud server.
Optionally, the obtaining module 1210 is specifically configured to receive frame data of a vehicle.
The device 1200 further includesa sending moduleconfigured to send an indication to the vehicle, where the indication is used to indicate whether the each block is abnormal.
As shown in FIG. 13, a device 1300 may include a transceiver 1301, a processor 1302, and a memory 1303. The memory 1303 may be configured to store codes, instructions, and the like executed by the processor 1302.
In some embodiments, the device 1300 may be a computing platform and may execute the above method executed by the transport illustrated in FIG. 10.
In some embodiments, the device 1300 may be transport and may execute the above method executed by the transport illustrated in FIG. 10.
In some embodiments, the device 1300 may be a server and may execute the above method executed by the transport illustrated in FIG. 10.
It should be understood that the processor 1302 may be an integrated circuit chip and has a signal processing capability. In an implementation process, steps of the foregoing method embodiments may be completed by using a hardware integrated logic circuit in the processor, or by using instructions in a form of software. The processor may be a general purpose processor, a digital signal processor (DSP) , an application-specific integrated circuit (ASIC) , a field programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The processor may implement or perform the methods, the steps, and the logical block diagrams that are disclosed in the embodiments of the present application. The general purpose processor may be a microprocessor, or the processor may be any conventional processor or the like. The steps of the methods disclosed with reference to the embodiments of the present application may be directly performed and completed by a hardware decoding processor, or may be performed and completed by using a combination of hardware in the decoding processor and a software module. The software module may be located in a mature storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is located in the memory, and the processor reads information in the memory and completes the steps of the foregoing methods in combination with hardware in the processor.
It may be understood that the memory 1303 in the embodiments of the present application may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The nonvolatile memory may be a read-only memory (ROM) , a programmable read-only memory (PROM) , an erasable programmable read-only memory (Erasable PROM, EPROM) , an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) , or a flash memory. The volatile memory may be a random access memory (RAM) and is used as an external cache. By way of example rather than limitation, many forms of RAMs may be used, and are, for example, a static random access memory (Static RAM, SRAM) , a dynamic random access memory (Dynamic RAM, DRAM) , a synchronous dynamic random access memory (Synchronous DRAM, SDRAM) , a double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM) , an enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM) , a synchronous link dynamic random access memory (Synchronous link DRAM, SLDRAM) , and a direct rambus random access memory (Direct Rambus RAM, DR RAM) .
It should be noted that the memory in the systems and the methods described in this specification includes but is not limited to these memories and a memory of any other appropriate type.
An embodiment of this application further provides a system chip, where the system chip includes an input/output interface, at least one processor, at least one memory, and a bus. The at least one memory is configured to store instructions, and the at least one processor is configured to invoke the instructions of the at least one memory to perform operations in the methods in the foregoing embodiments.
An embodiment of this application further provides a computer storage medium, where the computer storage medium may store a program instruction for performing any of the foregoing methods.
Optionally, the storage medium may be specifically the memory 1303.
A person of ordinary skill in the art may be aware that, in combination with the examples described in the embodiments disclosed in this specification, units and algorithm steps can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed by hardware or software depends on particular applications and design constraints of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application, but it should not be considered that the implementation goes beyond the scope of this application.
It may be clearly understood by a person skilled in the art that, for the purpose of convenient and brief description, for a detailed working process of the foregoing system, apparatus, and unit, refer to a corresponding process in the foregoing method embodiment. Details are not described herein again.
In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the described apparatus embodiment is merely an example. For example, the unit division is merely logical function division and may be other division in actual implementation. For example, a plurality of units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.
The units described as separate parts may be or may not be physically separate, and parts displayed as units may be or may not be physical units, may be located in one position, or may be distributed on a plurality of network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.
In addition, functional units in the embodiments of this application may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit.
When the functions are implemented in a form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer readable storage medium. Based on such an understanding, the technical solutions in this application essentially, or the part contributing to the prior art, or some of the technical solutions may be implemented in a form of a software product. The computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in the embodiments of this application. The foregoing storage medium includes: any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM) , a random access memory (RAM) , a magnetic disk, or an optical disc.
The foregoing descriptions are merely specific implementations of this application, but are not intended to limit the protection scope of this application. Any variation or replacement readily figured out by a person skilled in the art within the technical scope disclosed in this application shall fall within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.
Claims (15)
- A method for detecting data, characterized in comprising:obtainingframe data, wherein the frame data comprisesa snapshot of point cloud data during a preset duration;dividingthe frame data into apluralityof blocks;extractinga feature value of each block of the plurality of blocks, wherein the feature valuerepresents statistical characteristics of points in the each block; anddeterminingwhether the each block is abnormal or normal according to changes of feature values of the plurality of blocks.
- The method according to claim 1, whereinthe determining whether the each block is abnormal or normal according to the changes of the feature values of the plurality of blocks, comprises:inputting the changes of the feature values into a machine learning model for obtaining a prediction resultthat indicates whether the each block is abnormal or normal, wherein the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
- The method according to claim 1 or 2, wherein the feature value comprises one or more of:anaverage of coordinate values;a standard deviation of coordinate values;an averageof intensity; anda standard deviation of intensity.
- The method according to any one of claims 1 to 3, further comprising:promptinga user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality of blocks comprise the first block; and/orexecuting safety instructions if a second block is abnormal, wherein the plurality of blocks comprise the second block.
- The method according to any one of claims 1 to 4, wherein the method is performed by a device located in a cloud server.
- The method according to claim 5, wherein the obtaining the frame data, comprises:receivingframe data of a vehicle; andthe method further comprises:sending an indication to the vehicle, wherein the indication is used to indicate whether the each block is abnormal.
- A device for detecting data, characterized in comprising:an obtaining module, configured to obtain frame data, wherein the frame data comprises a snapshot of point cloud data during a preset duration;a dividing module, configured to divide the frame data into a plurality of blocks;an extracting module, configured to extract a feature value of each block of the plurality of blocks, wherein the feature value represents statistical characteristics of points in the each block; anda determining module, configured to determine whether theeach block is abnormal or normal according to changes of the feature values of the plurality of blocks.
- The device according to claim 7, wherein the determining module is specifically configured to input the changes of the feature values into a machine learning model for obtaining a prediction result that indicates whether the each block is abnormal or normal, wherein the machine learning model is trained by training samples, and the training samples include differences of feature values between abnormal blocks and normal blocks.
- The device according to claim 7 or 8, wherein the feature value comprises one or more of:anaverage of coordinate values;a standard deviation of coordinate values;an averageof intensity; anda standard deviation of intensity.
- The device according to any one of claims 7 to 9, wherein the device further comprises:a prompting module, configured to prompt a user that the point cloud data is abnormal if a first block is abnormal, wherein the plurality of blocks comprise the first block; and/or,an executing module, configured to execute safety instructions if a second block is abnormal, wherein the plurality of blocks comprise the second block.
- The device according to any one of claims 7 to 10, wherein the device is located in a cloud server.
- The device according to claim 11, wherein the obtaining module is specifically configured to receive frame data of a vehicle; andthe device further comprises:a sending module, configured to send an indication to the vehicle, wherein the indication is used to indicate whether the each block is abnormal.
- A device for detecting data, comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program from the memory and run the computer program, so that a computer on which a chip is disposed performs the method according to any one of claims 1 to 6.
- A computer readable storage medium, wherein the computer readable storage medium stores instructions, and when the instructions are run on a computer, the computer is enabled to perform the method according to any one of claims 1 to 6.
- A computer program product, wherein when the computer program product is run on a computer, the computer is enabled to perform the method according to any one of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2022/093636 WO2023220977A1 (en) | 2022-05-18 | 2022-05-18 | Method and device for detecting data |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/CN2022/093636 WO2023220977A1 (en) | 2022-05-18 | 2022-05-18 | Method and device for detecting data |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2023220977A1 true WO2023220977A1 (en) | 2023-11-23 |
Family
ID=88834179
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2022/093636 WO2023220977A1 (en) | 2022-05-18 | 2022-05-18 | Method and device for detecting data |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2023220977A1 (en) |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN108846374A (en) * | 2018-06-29 | 2018-11-20 | 山东鲁能智能技术有限公司 | Power equipment method for inspecting and device |
CN109581324A (en) * | 2018-10-31 | 2019-04-05 | 歌尔股份有限公司 | The processing method and processing device of abnormal frame data |
US20190156519A1 (en) * | 2017-11-22 | 2019-05-23 | Apple Inc. | Point cloud compression with multi-layer projection |
CN109859156A (en) * | 2018-10-31 | 2019-06-07 | 歌尔股份有限公司 | The processing method and processing device of abnormal frame data |
CN110009671A (en) * | 2019-02-22 | 2019-07-12 | 南京航空航天大学 | A kind of grid surface reconstructing system of scene understanding |
CN112258474A (en) * | 2020-10-22 | 2021-01-22 | 深圳集智数字科技有限公司 | Wall surface anomaly detection method and device |
CN112541525A (en) * | 2020-11-23 | 2021-03-23 | 歌尔股份有限公司 | Point cloud data processing method and device |
WO2021051281A1 (en) * | 2019-09-17 | 2021-03-25 | 深圳市大疆创新科技有限公司 | Point-cloud noise filtering method, distance measurement device, system, storage medium, and mobile platform |
-
2022
- 2022-05-18 WO PCT/CN2022/093636 patent/WO2023220977A1/en unknown
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20190156519A1 (en) * | 2017-11-22 | 2019-05-23 | Apple Inc. | Point cloud compression with multi-layer projection |
CN108846374A (en) * | 2018-06-29 | 2018-11-20 | 山东鲁能智能技术有限公司 | Power equipment method for inspecting and device |
CN109581324A (en) * | 2018-10-31 | 2019-04-05 | 歌尔股份有限公司 | The processing method and processing device of abnormal frame data |
CN109859156A (en) * | 2018-10-31 | 2019-06-07 | 歌尔股份有限公司 | The processing method and processing device of abnormal frame data |
CN110009671A (en) * | 2019-02-22 | 2019-07-12 | 南京航空航天大学 | A kind of grid surface reconstructing system of scene understanding |
WO2021051281A1 (en) * | 2019-09-17 | 2021-03-25 | 深圳市大疆创新科技有限公司 | Point-cloud noise filtering method, distance measurement device, system, storage medium, and mobile platform |
CN112258474A (en) * | 2020-10-22 | 2021-01-22 | 深圳集智数字科技有限公司 | Wall surface anomaly detection method and device |
CN112541525A (en) * | 2020-11-23 | 2021-03-23 | 歌尔股份有限公司 | Point cloud data processing method and device |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Reina et al. | Ambient awareness for agricultural robotic vehicles | |
CN110046640B (en) | Distributed representation learning for correlating observations from multiple vehicles | |
US20210326609A1 (en) | Object classification using extra-regional context | |
EP3137355B1 (en) | Device for designating objects to a navigation module of a vehicle equipped with said device | |
Liu et al. | “Seeing is not always believing”: detecting perception error attacks against autonomous vehicles | |
CN111291697B (en) | Method and device for detecting obstacles | |
Niranjan et al. | Deep learning based object detection model for autonomous driving research using carla simulator | |
US10860034B1 (en) | Barrier detection | |
CN110286389A (en) | A kind of grid management method for obstacle recognition | |
CN113378760A (en) | Training target detection model and method and device for detecting target | |
EP4307219A1 (en) | Three-dimensional target detection method and apparatus | |
US20220035003A1 (en) | Method and apparatus for high-confidence people classification, change detection, and nuisance alarm rejection based on shape classifier using 3d point cloud data | |
KR20230036960A (en) | Obstacle detection method and device, automatic driving vehicle, equipment and storage medium | |
CN111959526B (en) | Unmanned vehicle-based control method and device, unmanned vehicle and electronic equipment | |
WO2023220977A1 (en) | Method and device for detecting data | |
Katsamenis et al. | Real time road defect monitoring from UAV visual data sources | |
Sharma et al. | Towards an AI-based after-collision forensic analysis protocol for autonomous vehicles | |
WO2023023982A1 (en) | An intrusion detection method and apparatus | |
KR102267287B1 (en) | LSTM-based future threat prediction method and apparatus | |
Katare et al. | Autonomous embedded system enabled 3-D object detector:(With point cloud and camera) | |
Ravishankaran | Impact on how AI in automobile industry has affected the type approval process at RDW | |
WO2023023951A1 (en) | Method for detecting anomalies of lidar point cloud data and related device | |
CN117985053B (en) | Sensing capability detection method and device | |
Yang et al. | Adversarial robustness analysis of LiDAR-included models in autonomous driving | |
CN117893412B (en) | Point cloud data filtering method, device, equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 22942019 Country of ref document: EP Kind code of ref document: A1 |