WO2021072709A1 - 目标检测与跟踪方法、系统、设备及存储介质 - Google Patents

目标检测与跟踪方法、系统、设备及存储介质 Download PDF

Info

Publication number
WO2021072709A1
WO2021072709A1 PCT/CN2019/111730 CN2019111730W WO2021072709A1 WO 2021072709 A1 WO2021072709 A1 WO 2021072709A1 CN 2019111730 W CN2019111730 W CN 2019111730W WO 2021072709 A1 WO2021072709 A1 WO 2021072709A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
point cloud
dimensional point
information
target
Prior art date
Application number
PCT/CN2019/111730
Other languages
English (en)
French (fr)
Inventor
崔健
陈晓智
徐斌
蒋卓键
张磊杰
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to CN201980034227.9A priority Critical patent/CN112166458B/zh
Priority to PCT/CN2019/111730 priority patent/WO2021072709A1/zh
Publication of WO2021072709A1 publication Critical patent/WO2021072709A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/143—Sensing or illuminating at different wavelengths
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the embodiments of the present application relate to the field of movable platforms, and in particular, to a target detection and tracking method, system, device, and storage medium.
  • the target tracking algorithm provides a reliable basis for the estimation of the target state, for example, the estimation of the target motion trajectory, the estimation of the target behavior, etc., and the target The accuracy of the tracking algorithm directly affects the safety of autonomous driving.
  • the target detection algorithm can provide the self-driving vehicle with perceptual information about the surrounding environment. For example, the target detection algorithm can be used to detect the location, size, orientation, category and other information of the target.
  • the target tracking algorithm and the target detection algorithm are separated and independent of each other. If two sets of methods are used for target tracking and target detection, it will cause a large waste of resources.
  • the embodiments of the present application provide a target detection and tracking method, system, device, and storage medium to avoid resource waste in the target detection and target tracking process.
  • the first aspect of the embodiments of the present application is to provide a target detection and tracking method, which is applied to a movable platform, the movable platform is provided with a detection device, and the detection device is used to detect objects around the movable platform to obtain a three-dimensional Point cloud, the method includes:
  • Target tracking is performed on the target object according to the position change, the first detection information and the second detection information.
  • the second aspect of the embodiments of the present application is to provide a target detection and tracking system, including: a detection device, a memory, and a processor;
  • the detection device is used to detect objects around the movable platform to obtain a three-dimensional point cloud
  • the memory is used to store program codes
  • the processor calls the program code, and when the program code is executed, is used to perform the following operations:
  • Target tracking is performed on the target object according to the position change, the first detection information and the second detection information.
  • the third aspect of the embodiments of the present application is to provide a movable platform, including:
  • the power system is installed on the fuselage to provide mobile power
  • the fourth aspect of the embodiments of the present application is to provide a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the method described in the first aspect.
  • the target detection and tracking method, system, device, and storage medium provided in this embodiment use the three-dimensional point cloud detected by the detection device to perform target detection and target tracking on the target object in the three-dimensional point cloud at the same time, that is, target detection It uses the same input as target tracking. Therefore, the features extracted from the input are also similar or the same. These similar or identical features can be shared by target detection and target tracking. Therefore, these similar or identical features save money. Repeated calculations avoid waste of resources and effectively improve computing efficiency.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the application
  • FIG. 2 is a flowchart of a target detection and tracking method provided by an embodiment of the application
  • FIG. 3 is a flowchart of a target detection algorithm based on deep learning provided by an embodiment of the application
  • FIG. 4 is a schematic diagram of a target object provided by an embodiment of the application.
  • FIG. 5 is a schematic diagram of a representation form of a target object provided by an embodiment of this application.
  • FIG. 6 is a flowchart of a target detection and tracking method provided by an embodiment of the application.
  • FIG. 7 is a flowchart of a target detection and tracking method provided by another embodiment of this application.
  • FIG. 8 is a schematic diagram of a target tracking provided by an embodiment of the application.
  • FIG. 9 is a flowchart of a target detection and tracking method provided by another embodiment of this application.
  • FIG. 10 is a schematic diagram of another target tracking provided by an embodiment of this application.
  • FIG. 11 is a structural diagram of a target detection and tracking system provided by an embodiment of the application.
  • 345 target object; 351: target object; 362: target object;
  • 90 two-dimensional image
  • 92 three-dimensional circumscribed frame
  • 93 three-dimensional circumscribed frame
  • 112 memory; 113: processor.
  • a component when referred to as being "fixed to” another component, it can be directly on the other component or a central component may also exist. When a component is considered to be “connected” to another component, it can be directly connected to the other component or there may be a centered component at the same time.
  • the embodiment of the present application provides a target detection and tracking method.
  • the method can be applied to a movable platform, the movable platform is provided with a detection device, and the detection device is used to detect objects around the movable platform to obtain a three-dimensional point cloud.
  • the detection device includes but is not limited to lidar.
  • the movable platform may be a drone, a movable robot or a vehicle.
  • the movable platform is a vehicle as an example.
  • the vehicle may be an unmanned vehicle or a vehicle equipped with an Advanced Driver Assistance Systems (ADAS) system.
  • ADAS Advanced Driver Assistance Systems
  • the vehicle 11 is a carrier equipped with a detection device, and the detection device may specifically be a binocular stereo camera, a time of flight (TOF) camera, and/or a lidar.
  • TOF time of flight
  • the detection device detects objects around the vehicle 11 in real time to obtain a three-dimensional point cloud.
  • Objects around the vehicle 11 include trees, pedestrians, and other vehicles around the vehicle 11, for example, the vehicle 13 and the vehicle 14 and so on.
  • lidar Take lidar as an example.
  • a beam of laser light emitted by the lidar illuminates the surface of an object
  • the surface of the object will reflect the beam of laser light.
  • the lidar can determine the relative position of the object based on the laser light reflected from the surface of the object. Information such as the position and distance of the lidar. If the laser beam emitted by the laser radar scans according to a certain trajectory, such as a 360-degree rotating scan, a large number of laser points will be obtained, and thus the laser point cloud data of the object can be formed, that is, a three-dimensional point cloud.
  • the target detection and tracking method can be executed by the vehicle-mounted device in the vehicle, or it can be executed by other devices with data processing functions other than the vehicle-mounted device, such as As shown in the server 12 shown in FIG. 1, the vehicle 11 and the server 12 can perform wireless communication or wired communication.
  • the vehicle 11 can send the three-dimensional point cloud detected by the detection device to the server 12, and the server 12 executes the target detection and tracking method .
  • the following uses a vehicle-mounted device as an example to introduce the target detection and tracking method provided in the embodiment of the present application.
  • the vehicle-mounted device may be a device with a data processing function integrated in the vehicle center console, or may also be a tablet computer, a mobile phone, a notebook computer, etc. placed in the vehicle.
  • Fig. 2 is a flowchart of a target detection and tracking method provided by an embodiment of the application. As shown in Figure 2, the method in this embodiment may include:
  • Step S201 Obtain a three-dimensional point cloud of the previous frame and a three-dimensional point cloud of the current frame.
  • the detection device mounted on the vehicle 11 detects objects around the vehicle 11 in real time to obtain a three-dimensional point cloud.
  • the detection device can communicate with the on-board equipment on the vehicle 11, so that the vehicle
  • the vehicle-mounted device on 11 can obtain the three-dimensional point cloud detected by the detection device in real time.
  • the three-dimensional point cloud detected by the detection device at the previous moment is recorded as P0
  • the three-dimensional point cloud detected by the detection device at the current moment is recorded as P0.
  • P1 correspondingly, the three-dimensional point cloud P0 is recorded as the three-dimensional point cloud of the previous frame
  • the three-dimensional point cloud P1 is recorded as the three-dimensional point cloud of the current frame.
  • the last frame of 3D point cloud may also be the 3D point cloud accumulated and detected by the detection device in a short historical time period
  • the current frame of 3D point cloud may be the accumulated detection of the detection device in a short current time period. The obtained 3D point cloud.
  • Step S202 Detect the target object in the last frame of the three-dimensional point cloud, and obtain first detection information corresponding to the target object.
  • the three-dimensional point cloud detected by the detection device includes the three-dimensional point cloud of objects around the vehicle 11, for example, the objects around the vehicle 11 may include trees, pedestrians, and other vehicles such as vehicles 13, vehicles 14, etc. Therefore, the three-dimensional point cloud detected by the detection device The point cloud includes the three-dimensional point cloud of trees around the vehicle 11, the three-dimensional point cloud of pedestrians, and the three-dimensional point cloud of other vehicles such as the vehicle 13 and the vehicle 14.
  • the last frame of three-dimensional point cloud includes N points, and each point includes position information and reflectivity.
  • the position information of each point may be the three-dimensional coordinates (x, y, z) of the point in the three-dimensional coordinate system.
  • the three-dimensional coordinate system may specifically be a vehicle. Body coordinate system, earth coordinate system, or world coordinate system, etc.
  • a target detection algorithm can be used to detect the target object in the last frame of 3D point cloud, for example, a target detection algorithm based on deep learning can be used to detect the target object in the last frame of 3D point cloud. Obtain the first detection information corresponding to the target object.
  • the flowchart of the target detection algorithm based on deep learning is shown in Figure 3.
  • the previous frame of 3D point cloud is used as input, and the disordered previous frame of 3D point cloud is processed into the first convolutional neural network through input preprocessing.
  • the required orderly input for example, the last frame of three-dimensional point cloud is processed into a tensor of a certain size.
  • the tensor here can be understood as a high-dimensional matrix.
  • a high-dimensional matrix is a matrix larger than two-dimensional.
  • the three-dimensional matrix is specifically taken as an example.
  • the size of the tensor can be expressed as C*H*W, where C represents the number of channels input by the first convolutional neural network, H represents height, and W represents width.
  • the first convolutional neural network is used for target detection. Further, the first convolutional neural network processes a tensor of a certain size to detect the target object in the last frame of the three-dimensional point cloud, and obtains the target detection result, that is, the detection information corresponding to the target object, after outputting the post-processing.
  • the detection information of the target object in the three-dimensional point cloud of the current frame is obtained. Therefore, in order to distinguish the detection information corresponding to the target object in the 3D point cloud of the previous frame from the detection information corresponding to the target object in the 3D point cloud of the current frame, the detection information corresponding to the target object in the 3D point cloud of the previous frame is recorded as the first One detection information, the detection information corresponding to the target object in the three-dimensional point cloud of the previous frame is recorded as the second detection information.
  • the first detection information corresponding to the target object includes at least one of the following: a first position, a first size, a first direction, a category of the target object, and a first position of the target object belonging to the category.
  • a probability value is a probability value.
  • 40 represents the last frame of 3D point cloud detected by the detection device.
  • the target object in the last frame of 3D point cloud 40 can be detected, and the last frame The first detection information of the target object in the three-dimensional point cloud 40.
  • the target object may be a point cloud cluster composed of three-dimensional point clouds corresponding to objects around the vehicle 11.
  • the target object 30 is a point cloud cluster composed of ground point clouds around the vehicle 11
  • the target object 31 is a point cloud cluster composed of three-dimensional point clouds corresponding to the vehicles 14 around the vehicle 11
  • the target object 32 is a three-dimensional point cloud cluster corresponding to the vehicles 13 around the vehicle 11.
  • the number of target objects detected from the last frame of the three-dimensional point cloud is not limited here, and the several target objects shown in FIG. 4 are only a schematic illustration.
  • the detection information corresponding to the target object may have multiple representation forms, and the representation form shown in FIG. 5 is only a schematic illustration.
  • the front direction of the own vehicle which is the above-mentioned vehicle 11
  • the right side of the vehicle 11 is the Y axis
  • the direction from the bottom of the vehicle 11 to the ground is Z
  • the axis establishes a three-dimensional coordinate system
  • the three-dimensional coordinate system is the vehicle body coordinate system.
  • the first detection information corresponding to the target object detected based on the above-mentioned target detection algorithm may also include the identification information of the target object, such as the number 342 shown in FIG. 5 , 345, 351, 362, 376 are the identification information of multiple target objects in the last frame of 3D point cloud, that is, the last frame of 3D point cloud includes target object 342, target object 345, target object 351, target Object 362, target object 376.
  • the position, size, and direction of the target object can be represented by the three-dimensional circumscribed frame of the target object.
  • the position, size, and direction of the target object 342 shown in FIG. 5 can be represented by the three-dimensional circumscribed frame of the target object 342.
  • the three-dimensional circumscribed frame can be marked as box, and the coordinates of the three-dimensional circumscribed frame in the vehicle body coordinate system can be Denoted as [x0,x1,x2,x3,y0,y1,y2,y3,zmin,zmax].
  • (x0, y0), (x1, y1), (x2, y2), (x3, y3) are the 4 vertices of the three-dimensional circumscribed frame in the top view.
  • zmin is the minimum coordinate value of the three-dimensional circumscribed frame on the Z axis of the vehicle body coordinate system
  • zmax is the maximum coordinate value of the three-dimensional circumscribed frame on the Z axis of the vehicle body coordinate system.
  • the category to which the target object belongs can be recorded as class, and the probability value of the target object belonging to this category can be recorded as score.
  • This category can include: road markings, vehicles, pedestrians, trees, road signs, etc.
  • the categories of different target objects shown in FIG. 5 may be different.
  • the category to which the target object 342 belongs is a vehicle
  • the category to which the target object 376 belongs is a tree.
  • the probability value of the target object 342 belonging to the vehicle is score1
  • the probability value of the target object 376 belonging to the tree is score2.
  • Step S203 Detect the target object in the three-dimensional point cloud of the current frame, and obtain second detection information corresponding to the target object.
  • the process of detecting the target object in the three-dimensional point cloud of the current frame is similar to the process of detecting the target object in the previous frame of the three-dimensional point cloud as described above, and will not be repeated here.
  • the second detection information corresponding to the target object includes at least one of the following: a second position, a second size, a second direction, a category of the target object, and a category that the target object belongs to. Two probability value.
  • the second detection information corresponding to the target object detected at the current moment may be different from the first detection information corresponding to the target object detected at the previous moment.
  • the second position of the target object detected at the current moment may be different from the first position of the target object detected at the previous moment.
  • the second size of the target object detected at the current moment may be different from the first size of the target object detected at the previous moment.
  • the second direction of the target object detected at the current moment may be different from the first direction of the target object detected at the previous moment.
  • the category to which the target object detected at the current moment belongs may be different or the same as the category to which the target object detected at the previous moment belongs. Here, the same category is taken as an example for schematic illustration.
  • the second probability value of the target object detected at the current moment belonging to a certain category may be different from the first probability value of the target object detected at the previous moment belonging to the category.
  • Step S204 Determine the position change of the target object between the last frame of 3D point cloud and the current frame of 3D point cloud according to the last frame of 3D point cloud and the current frame of 3D point cloud.
  • the first convolutional neural network is used for target detection
  • the second convolutional neural network is used for target tracking.
  • the first convolutional neural network on the left is used to detect the target object in the three-dimensional point cloud of the previous frame
  • the first convolutional neural network on the right is used to detect the target object in the three-dimensional point cloud of the current frame.
  • the process of performing target detection on the last frame of three-dimensional point cloud through the first convolutional neural network is similar to the process of performing target detection on the current frame of three-dimensional point cloud through the first convolutional neural network, and will not be repeated here.
  • the first convolutional neural network may include n convolutional layers, and the processing and calculation processes performed by different convolutional layers may be different or the same.
  • the output of convolutional layer 1 may be the input of convolutional layer 2, and the output of convolutional layer 2 may be the input of convolutional layer 3, and so on.
  • the processing calculation process of convolutional layer 1, convolutional layer 2, ..., convolutional layer n on the same side may be different or may be the same.
  • the output of the convolutional layer 2 of the network, and the output of the middle layer of the first convolutional neural network on the left and right sides are feature fused to obtain the fused feature. It is understandable that only one intermediate layer on each of the left and right sides is used as an example for schematic illustration. In other embodiments, the outputs of multiple intermediate layers of the first convolutional neural network on the left and right sides can also be obtained separately.
  • the output of convolutional layer 2 and convolutional layer 3 of the first convolutional neural network on the left and the output of convolutional layer 2 and convolutional layer 3 of the first convolutional neural network on the right, and compare the left
  • the output of the side convolutional layer 2 and the convolutional layer 3, and the output of the right convolutional layer 2 and the convolutional layer 3 perform feature fusion to obtain the fused feature.
  • the output of the top convolutional layer 1 and/or the bottom convolutional layer n in the first convolutional neural network on the left and right sides may also be obtained.
  • the fused features are input into the second convolutional neural network, and the second convolutional neural network obtains the position change of the target object between the two frames, that is, the target object is in the last frame of the three-dimensional point cloud and the position change. Describes the position change between the three-dimensional point clouds of the current frame.
  • the target object here may be the target object generally referred to in the three-dimensional point cloud of the previous frame and the three-dimensional point cloud of the current frame.
  • Step S205 Perform target tracking on the target object according to the position change, the first detection information and the second detection information.
  • the target object in the three-dimensional point cloud of the previous frame and the target object in the three-dimensional point cloud of the current frame may be partially the same, for example, the target object 342, the target object 345, and the target object 351 shown in FIG.
  • the target object 362 and the target object 376 are the target objects in the last frame of the three-dimensional point cloud.
  • the target object 345, the target object 351, the target object 362, and the target object 376 may be detected, but the target object 342 is not detected.
  • a new target object in the three-dimensional point cloud of the current frame that is, a target object that has not appeared in the three-dimensional point cloud of the previous frame.
  • the target tracking described in this embodiment can not only track the common target object in the 3D point cloud of the previous frame and the 3D point cloud of the current frame, but can also track the 3D point cloud that only appears in the previous frame or the 3D point cloud of the current frame. The target object in the tracker.
  • target detection and target tracking are performed on the target objects in the three-dimensional point cloud at the same time, that is, the same input is used for target detection and target tracking, so the input is extracted.
  • the features of are also similar or identical. These similar or identical features can be shared by target detection and target tracking. Therefore, these similar or identical features save the amount of repetitive calculation and avoid waste of resources.
  • FIG. 7 is a flowchart of a target detection and tracking method provided by another embodiment of the application.
  • the target tracking of the target object according to the position change, the first detection information, and the second detection information may include:
  • Step S701 Determine the predicted position of the target object in the three-dimensional point cloud of the current frame according to the position change and the first position of the target object.
  • 80 represents the 2D image obtained by projecting the last frame of 3D point cloud along the Z axis of the 3D coordinate system
  • 90 represents the projection of the current frame of 3D point cloud along the Z axis of the 3D coordinate system
  • the target object 81, the target object 82, and the target object 83 represent the target objects in the last frame of the three-dimensional point cloud.
  • the target object 81 as an example, the three-dimensional circumscribed frame corresponding to the target object 81 is marked as box0, and the position change of the target object 81 between the previous frame of the three-dimensional point cloud and the current frame of the three-dimensional point cloud is recorded as ⁇ box.
  • the predicted position of the target object 81 in the 3D point cloud of the current frame can be predicted.
  • the predicted position can be understood as the position where the target object 81 is expected to appear in the 3D point cloud of the current frame after the position change ⁇ box.
  • the predicted positions of the target object 82 and the target object 83 in the three-dimensional point cloud of the previous frame in the current frame of the three-dimensional point cloud respectively, and the three-dimensional circumscribed frame at the corresponding predicted position can be determined.
  • the target object 82 is in the corresponding The three-dimensional circumscribed frame at the predicted position is the three-dimensional circumscribed frame 92, and the three-dimensional circumscribed frame of the target object 83 at the corresponding predicted position is the three-dimensional circumscribed frame 93.
  • each target object in the three-dimensional point cloud of the previous frame corresponds to a predicted position in the three-dimensional point cloud of the current frame.
  • the target object 81 corresponds to the three-dimensional circumscribed frame 91
  • the target object 82 corresponds to the three-dimensional circumscribed frame 92
  • the target object 83 corresponds to the three-dimensional circumscribed frame 93.
  • Step S702 Perform target tracking on the target object according to the predicted position and the second position of the target object.
  • the target object A, the target object B, the target object C, and the target object D are the target objects in the three-dimensional point cloud of the current frame detected by the above-mentioned target detection algorithm.
  • the positions of the target object A, the target object B, the target object C, and the target object D are respectively recorded as the second positions.
  • the predicted position of the target object in the three-dimensional point cloud of the current frame and the actually detected position of the target object in the three-dimensional point cloud of the current frame should be very close.
  • the target control 81 and the target control A are the same target object
  • the three-dimensional circumscribed frame 91 of the target control 81 at the corresponding predicted position and the target control A should be very close. Therefore, the distance between the three-dimensional circumscribed frame at the predicted position and the three-dimensional circumscribed frame corresponding to the target object actually detected in the three-dimensional point cloud of the current frame can be used to determine which two target objects are the same target object.
  • the performing target tracking on the target object according to the predicted position and the second position of the target object includes: if the distance between the predicted position and the second position of the target object is If the distance is less than the preset distance, the identification information of the target object in the three-dimensional point cloud of the previous frame is used as the identification information of the target object in the three-dimensional point cloud of the current frame.
  • the three-dimensional circumscribed frame corresponding to target object A as box2, and calculate the distance between box1 and box2.
  • the distance is less than the preset distance, it is determined that target object 81 and target object A are the same target object. Therefore, Taking the identification number 81 of the target object 81 as the identification number of the target object A in the three-dimensional point cloud of the current frame, the association between the target object 81 and the target object A is realized, that is, the target object A in the three-dimensional point cloud of the current frame It is the target object 81 in the three-dimensional point cloud of the previous frame, so that the target tracking of the target object A is realized.
  • the target object B associated with the target object 82 and the target object C associated with the target object 83 in the three-dimensional point cloud of the current frame can be determined, and the identification number 82 of the target object 82 is taken as the three-dimensional point of the target object B in the current frame.
  • the identification number 83 of the target object 83 is used as the identification number of the target object C in the three-dimensional point cloud of the current frame, thereby achieving target tracking of the target object B and the target object C.
  • the target object D is a newly appeared target object in the three-dimensional point cloud of the current frame, that is, there is no target object associated with the target object D in the three-dimensional point cloud of the previous frame. Therefore, Assign a new identification number to the target object D, such as 84.
  • the predicted position of the target object in the three-dimensional point cloud of the previous frame in the three-dimensional point cloud of the current frame is used to determine whether the distance between the predicted position and the position of the target object detected in the three-dimensional point cloud of the current frame is less than The preset distance, if yes, it is determined that the target object in the last frame of 3D point cloud and the target object detected in the current frame of 3D point cloud are the same target object, and the target object is identified in the previous frame of 3D point cloud
  • the information is used as the identification information of the target object in the three-dimensional point cloud of the current frame, thereby realizing the target tracking of the target object.
  • FIG. 9 is a flowchart of a target detection and tracking method provided by another embodiment of the application.
  • the target object is in the three-dimensional point cloud of the previous frame and the The position changes between the 3D point clouds of the current frame.
  • Step S901 Obtain the first feature information corresponding to the last frame of the three-dimensional point cloud and the second feature information corresponding to the current frame of the three-dimensional point cloud.
  • the first feature information is output information of at least one convolutional layer of the first convolutional neural network when the first convolutional neural network is used to detect the target object in the last frame of the three-dimensional point cloud
  • the second feature information is the output information of the at least one convolutional layer of the first convolutional neural network when the first convolutional neural network is used to detect the target object in the three-dimensional point cloud of the current frame .
  • the first feature information may be the output information of the convolutional layer 2 of the first convolutional neural network on the left as shown in FIG. 6, and the second feature information may be the first convolutional neural network on the right as shown in FIG.
  • the output information of the convolutional layer 2 of the network For example, the first feature information is recorded as feature0, and the dimension of feature0 is c1*H*W.
  • the second feature information is recorded as feature1, and the dimension of feature1 is c1*H*W.
  • Step S902 Perform feature fusion on the first feature information and the second feature information to obtain the fused third feature information.
  • feature fusion is performed on the first feature information feature0 and the second feature information feature1 to obtain the fused third feature information.
  • the performing feature fusion on the first feature information and the second feature information to obtain the fused third feature information includes: performing feature fusion on the first feature information and the second feature information
  • the correlation calculation obtains the correlation information of the first characteristic information and the second characteristic information; according to the correlation information, the first characteristic information, and the second characteristic information, the fused information is obtained The third feature information.
  • the correlation calculation is performed on the first feature information feature0 and the second feature information feature1 to obtain the correlation information between feature0 and feature1, and the dimension of the correlation information is denoted as c2*H*W.
  • the correlation information, feature0, and feature1 are spliced to obtain the fused third feature information.
  • the fused third feature information is recorded as fused_feature, and the dimension of fused_feature is (2*c1+c2)*H*W.
  • correlation may specifically be an operation to obtain information about the timing change of adjacent frames
  • the first feature information feature0 is expressed as Denote the second feature information feature1 as Among them, t represents the time corresponding to the previous frame, t+ ⁇ represents the time corresponding to the current frame, l represents the number of layers of the convolutional layer, (i,j) represents the position of the target object, (p,q) represents the target object at The position change between the 3D point cloud of the previous frame and the 3D point cloud of the current frame.
  • the correlation information between feature0 and feature1 is recorded as The relationship between can be expressed as the following formula (1):
  • (2d+1) 2 c2
  • d represents the size of the offset in the correlation calculation.
  • d may be a preset value.
  • Step S903 Determine the position change of the target object between the three-dimensional point cloud of the previous frame and the three-dimensional point cloud of the current frame according to the third characteristic information.
  • the determining the position change of the target object between the three-dimensional point cloud of the previous frame and the three-dimensional point cloud of the current frame according to the third feature information includes: combining the third feature The information is input into a second convolutional neural network, and the position change of the target object between the three-dimensional point cloud of the previous frame and the three-dimensional point cloud of the current frame is determined through the second convolutional neural network.
  • the fused third feature information fused_feature is input into the second convolutional neural network, and the second convolutional neural network obtains the position change of the target object between two frames.
  • the first feature information corresponding to the three-dimensional point cloud of the previous frame is fused with the second feature information corresponding to the three-dimensional point cloud of the current frame to obtain the fused third feature information, and the target is determined according to the third feature information
  • the position change of the object between the last frame of 3D point cloud and the current frame of 3D point cloud that is to say, according to the intermediate result of target detection, the position change of the target object during target tracking is determined, which realizes the effective target detection and target tracking.
  • the combination avoids the independent operation of target detection and target tracking, which improves the waste of resources.
  • the embodiment of the present application provides a target detection and tracking method.
  • the detecting the target object in the three-dimensional point cloud of the current frame to obtain the second detection information corresponding to the target object may include: detecting the current frame by using a first convolutional neural network The target object in the three-dimensional point cloud obtains the second detection information corresponding to the target object.
  • the first convolutional neural network on the right is used to detect the target object in the three-dimensional point cloud of the current frame, and the second detection information corresponding to the target object is obtained.
  • the method further includes: correcting the second detection information according to the tracking information obtained by the target tracking.
  • the tracking information includes the number of frames of the historical three-dimensional point cloud that can be tracked to the target object, and historical detection information of the target object in each historical three-dimensional point cloud.
  • the historical detection information includes at least one of the following: a historical position, a historical size, a historical direction, a category of the target object, and a historical probability value of the target object belonging to the category.
  • 100 represents the two-dimensional image obtained by projecting the three-dimensional point cloud detected by the detection device at historical time t0 along the Z axis of the three-dimensional coordinate system
  • 101 represents the three-dimensional point detected by the detection device at historical time t1
  • 102 represents the two-dimensional image obtained after the three-dimensional point cloud detected by the detection device at the current time t2 is projected along the Z axis of the three-dimensional coordinate system.
  • the target object 71, the target object 81 and the target object A are the same target object
  • the target object 72, the target object 82 and the target object B are the same target object
  • the target object 73, the target object 83 and the target object C are the same target.
  • the tracking information corresponding to each of the target object A, the target object B, and the target object C can be obtained, and the tracking information can be recorded as a tracklet.
  • the three-dimensional circumscribed frame of the target object 71 is marked as box01
  • the type to which the target object 71 belongs is a vehicle
  • the probability value of the target object 71 belonging to the vehicle is recorded as score01.
  • the three-dimensional circumscribed frame of the target object 81 is marked as box02, the type to which the target object 81 belongs is a vehicle, and the probability value of the target object 81 belonging to a vehicle is recorded as score02.
  • the three-dimensional circumscribed frame of target object A is marked as box03, the type of target object A belongs to is a vehicle, and the probability value of target object A belonging to a vehicle is recorded as score03.
  • the tracklet corresponding to the target object A may specifically be a sequence composed of box01 and score01, box02 and score02.
  • the tracklet corresponding to the target object A may specifically be the historical detection information of the target object A in the historical three-dimensional point cloud.
  • the tracklet corresponding to the target object A may also include the number of frames of the historical three-dimensional point cloud of the target object A that can be tracked in the historical time, and the number of frames is denoted as N.
  • the second detection information corresponding to the target object in the three-dimensional point cloud of the current frame may be further corrected according to the tracklet corresponding to each target object. For example, according to the tracklet corresponding to the target object A, the corresponding box03 and score03 of the target object A in the 3D point cloud of the current frame are corrected, and the corrected box03 and the corrected score03 are stored in the tracklet corresponding to the target object A.
  • the correcting the second detection information according to the tracking information obtained by the target tracking includes: according to the historical probability value of the target object belonging to the category, performing the correction on the The second probability value of the target object belonging to the category is corrected.
  • the tracklet corresponding to the target object A includes box01 and score01, box02 and score02. Further, calculate the sum of score01 and score02 to obtain the cumulative probability value SUM_SCORE.
  • the corrected value of score03 is recorded as score03', and score03' can be obtained by the following formula (2) Calculated:
  • the revised score03' needs to be recorded in the tracklet corresponding to the target object A.
  • the N value included in the tracklet corresponding to the target object A is added 1.
  • the updated cumulative probability value SUM_SCORE needs to add score03’.
  • score01 and score02 may also be the probability values corrected according to a method similar to the formula (2).
  • a tracklet corresponding to target object A needs to be established, and box03 and score03 are stored in the tracklet corresponding to target object A.
  • the update method of N and SUM_SCORE corresponding to the tracklet is as described above, and will not be repeated here.
  • the correcting the second detection information according to the tracking information obtained by the target tracking includes at least one of the following: correcting the target according to the historical position of the target object The second position of the object is corrected; the second size of the target object is corrected according to the historical size of the target object; the second direction of the target object is corrected according to the historical direction of the target object.
  • the tracklet corresponding to target object A includes box01 and score01, box02 and score02.
  • box03 can be modified according to box01 and box02.
  • box01 corresponds to the position, direction, and size of target object 71
  • box02 corresponds to target object 81
  • the position, direction, and size of the box03 corresponds to the position, direction, and size of the target object A.
  • the correction of box03 according to box01 and box02 may include at least one of the following: according to the position of the target object 71 and the position of the target object 81 Correct the position of the target object A; correct the direction of the target object A according to the direction of the target object 71 and the direction of the target object 81; correct the size of the target object A according to the size of the target object 71 and the size of the target object 81 , So that the corrected box03 is obtained.
  • the corrected box03 is recorded as box03', and further, box03' is stored in the tracklet corresponding to the target object A.
  • box01 and box02 may also be modified three-dimensional circumscribed frames.
  • the detection information of the target object is corrected by tracking information obtained by tracking the target object. Since the tracking information can include the timing information corresponding to the target object, for example, the historical three-dimensional history of the target object can be tracked. The number of frames of the point cloud and the historical detection information of the target object in each historical three-dimensional point cloud. Therefore, the detection information of the target object can be corrected by the timing information of the target object to improve the detection of the target object Accuracy. For example, if there is a vehicle far away from the vehicle, it is difficult to judge whether there are other vehicles far away from the vehicle through a single frame of three-dimensional point cloud. However, by tracking the distant vehicle, it is determined that it can be tracked.
  • the detection information in the cloud is corrected to accurately determine whether the distant vehicle exists in the three-dimensional point cloud of the current frame.
  • FIG. 11 is a structural diagram of a target detection and tracking system provided by an embodiment of the application.
  • the target detection and tracking system 110 includes a detection device 111, a memory 112, and a processor 113.
  • the detection device 111 is used to detect objects around the movable platform to obtain a three-dimensional point cloud.
  • the processor 113 may specifically be a component in the in-vehicle device in the foregoing embodiment, or other components, devices, or components with data processing functions carried in the vehicle.
  • the memory 112 is used to store program codes; the processor 113 calls the program codes, and when the program codes are executed, is used to perform the following operations: obtain the last frame of 3D point cloud and the current frame of 3D point cloud; According to the target object in the three-dimensional point cloud of the previous frame, the first detection information corresponding to the target object is obtained; the target object in the three-dimensional point cloud of the current frame is detected to obtain the second detection information corresponding to the target object; The last frame of 3D point cloud and the current frame of 3D point cloud determine the position change of the target object between the last frame of 3D point cloud and the current frame of 3D point cloud; according to the position change , The first detection information and the second detection information perform target tracking on the target object.
  • the first detection information corresponding to the target object includes at least one of the following: a first position, a first size, a first direction, a category of the target object, and a first position of the target object belonging to the category.
  • a probability value is a probability value.
  • the second detection information corresponding to the target object includes at least one of the following: a second position, a second size, a second direction, a category of the target object, and a category that the target object belongs to. Two probability value.
  • the processor 113 when the processor 113 performs target tracking on the target object according to the position change, the first detection information, and the second detection information, it is specifically configured to: according to the position change and the target The first position of the object determines the predicted position of the target object in the three-dimensional point cloud of the current frame; and the target tracking is performed on the target object according to the predicted position and the second position of the target object.
  • the processor 113 when the processor 113 performs target tracking on the target object according to the predicted position and the second position of the target object, it is specifically configured to: if the predicted position and the second position of the target object are If the distance is less than the preset distance, then the identification information of the target object in the three-dimensional point cloud of the previous frame is used as the identification information of the target object in the three-dimensional point cloud of the current frame.
  • the processor 113 determines whether the target object is between the last frame of 3D point cloud and the current frame of 3D point cloud according to the last frame of 3D point cloud and the current frame of 3D point cloud.
  • the position changes it is specifically used to: obtain the first feature information corresponding to the last frame of the three-dimensional point cloud and the second feature information corresponding to the current frame of the three-dimensional point cloud; Perform feature fusion on the two feature information to obtain the fused third feature information; according to the third feature information, determine the position of the target object between the previous frame of 3D point cloud and the current frame of 3D point cloud Variety.
  • the first feature information is output information of at least one convolutional layer of the first convolutional neural network when the first convolutional neural network is used to detect the target object in the last frame of the three-dimensional point cloud
  • the second feature information is the output information of the at least one convolutional layer of the first convolutional neural network when the first convolutional neural network is used to detect the target object in the three-dimensional point cloud of the current frame .
  • the processor 113 performs feature fusion on the first feature information and the second feature information, and when the fused third feature information is obtained, it is specifically configured to: compare the first feature information and the second feature information. Second, perform correlation calculation on the characteristic information to obtain the correlation information of the first characteristic information and the second characteristic information; obtain the correlation information, the first characteristic information, and the second characteristic information according to the correlation information, the first characteristic information, and the second characteristic information.
  • the third feature information after fusion is described.
  • the processor 113 determines that the position of the target object changes between the three-dimensional point cloud of the previous frame and the three-dimensional point cloud of the current frame according to the third characteristic information, it is specifically configured to:
  • the third feature information is input into a second convolutional neural network, and the position change of the target object between the last frame of three-dimensional point cloud and the current frame of three-dimensional point cloud is determined through the second convolutional neural network.
  • the processor 113 detects the target object in the three-dimensional point cloud of the current frame, and obtains the second detection information corresponding to the target object, it is specifically configured to: use the first convolutional neural network to detect the three-dimensional point cloud of the current frame. From the target object in the point cloud, the second detection information corresponding to the target object is obtained.
  • the processor 113 is further configured to: correct the second detection information according to the tracking information obtained by the target tracking.
  • the tracking information includes the number of frames of the historical three-dimensional point cloud that can be tracked to the target object, and historical detection information of the target object in each historical three-dimensional point cloud.
  • the historical detection information includes at least one of the following: a historical location, a historical size, a historical direction, a category of the target object, and a historical probability value of the target object belonging to the category.
  • the processor 113 corrects the second detection information according to the tracking information obtained by the target tracking, it is specifically configured to: according to the historical probability value of the target object belonging to the category, perform the correction on the target The second probability value of the object belonging to the category is corrected.
  • the processor 113 corrects the second detection information according to the tracking information obtained by the target tracking, it is specifically used for at least one of the following: The second position is corrected; the second size of the target object is corrected according to the historical size of the target object; the second direction of the target object is corrected according to the historical direction of the target object.
  • the detection device includes a lidar.
  • the embodiment of the application provides a movable platform.
  • the movable platform includes: a fuselage, a power system, and the target detection and tracking system described in the above embodiment.
  • the power system is installed on the fuselage to provide moving power.
  • the target detection and tracking system can implement the target detection and tracking method as described above, and the specific principles and implementation manners of the target detection and tracking method are similar to the foregoing embodiment, and will not be repeated here.
  • This embodiment does not limit the specific form of the movable platform.
  • the movable platform may be a drone, a movable robot, or a vehicle.
  • this embodiment also provides a computer-readable storage medium on which a computer program is stored, and the computer program is executed by a processor to implement the target detection and tracking method described in the foregoing embodiment.
  • the disclosed device and method can be implemented in other ways.
  • the device embodiments described above are merely illustrative, for example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components may be combined or It can be integrated into another system, or some features can be ignored or not implemented.
  • the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
  • the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware, or may be implemented in the form of hardware plus software functional units.
  • the above-mentioned integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium.
  • the above-mentioned software functional unit is stored in a storage medium, and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute the method described in each embodiment of the present application. Part of the steps.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program code .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Biology (AREA)
  • Health & Medical Sciences (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Image Analysis (AREA)

Abstract

一种目标检测与跟踪方法、系统、设备及存储介质。该方法包括：获取上一帧三维点云和当前帧三维点云(S201)；检测上一帧三维点云中的目标对象，得到目标对象对应的第一检测信息(S202)；检测当前帧三维点云中的目标对象，得到目标对象对应的第二检测信息(S203)；根据上一帧三维点云和当前帧三维点云，确定目标对象在上一帧三维点云和当前帧三维点云之间的位置变化(S204)；根据位置变化、第一检测信息和第二检测信息，对目标对象进行目标跟踪(S205)。通过目标检测和目标跟踪采用相同的输入，因此，从输入中提取到的特征也是相似或相同的，这些相似或相同的特征节省了重复计算量，有效提升运算效率。

Description

目标检测与跟踪方法、系统、设备及存储介质 技术领域
本申请实施例涉及可移动平台领域，尤其涉及一种目标检测与跟踪方法、系统、设备及存储介质。
背景技术
在自动驾驶系统或高级驾驶辅助系统(Advanced Driving Assistant System，ADAS)中，目标跟踪算法为目标状态估计，例如，对目标运动轨迹的估计、对目标行为的估计等提供了可靠的依据，并且目标跟踪算法的准确性直接影响到了自动驾驶的安全性。另外，在自动驾驶系统或ADAS中，目标检测算法可以为自动驾驶车辆提供周围环境的感知信息，例如，目标检测算法可用于检测目标的位置、尺寸、朝向、类别等信息。
但是，当前同一个自动驾驶系统或ADAS中，目标跟踪算法和目标检测算法是相互分离、相互独立的，若采用两套方法分别进行目标跟踪和目标检测将会造成较大的资源浪费。
发明内容
本申请实施例提供一种目标检测与跟踪方法、系统、设备及存储介质，以避免目标检测和目标跟踪过程中的资源浪费。
本申请实施例的第一方面是提供一种目标检测与跟踪方法，应用于可移动平台，所述可移动平台设置有探测设备，所述探测设备用于探测所述可移动平台周围物体得到三维点云，所述方法包括：
获取上一帧三维点云和当前帧三维点云；
检测所述上一帧三维点云中的目标对象，得到所述目标对象对应的第一检测信息；
检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息；
根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象 在所述上一帧三维点云和所述当前帧三维点云之间的位置变化；
根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪。
本申请实施例的第二方面是提供一种目标检测与跟踪系统，包括：探测设备、存储器和处理器；
所述探测设备用于探测可移动平台周围物体得到三维点云；
所述存储器用于存储程序代码；
所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以下操作：
获取上一帧三维点云和当前帧三维点云；
检测所述上一帧三维点云中的目标对象，得到所述目标对象对应的第一检测信息；
检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息；
根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化；
根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪。
本申请实施例的第三方面是提供一种可移动平台，包括：
机身；
动力系统，安装在所述机身，用于提供移动动力；
以及第二方面所述的目标检测与跟踪系统。
本申请实施例的第四方面是提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行以实现第一方面所述的方法。
本实施例提供的目标检测与跟踪方法、系统、设备及存储介质，通过探测设备探测得到的三维点云，对三维点云中的目标对象同时进行目标检测和目标跟踪，也就是说，目标检测和目标跟踪采用相同的输入，因此，从输入中提取到的特征也是相似或相同的，这些相似或相同的特征可以是目标检测和目标跟踪所共用的，因此，这些相似或相同的特征节省了重复计算量，避免了资源浪费，有效提升运算效率。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例提供的一种应用场景的示意图；
图2为本申请实施例提供的目标检测与跟踪方法的流程图；
图3为本申请实施例提供的基于深度学习的目标检测算法的流程图；
图4为本申请实施例提供的目标对象的示意图；
图5为本申请实施例提供的目标对象的表示形式的示意图；
图6为本申请实施例提供的目标检测与跟踪方法的流程图；
图7为本申请另一实施例提供的目标检测与跟踪方法的流程图；
图8为本申请实施例提供的一种目标跟踪的示意图；
图9为本申请另一实施例提供的目标检测与跟踪方法的流程图；
图10为本申请实施例提供的另一种目标跟踪的示意图；
图11为本申请实施例提供的目标检测与跟踪系统的结构图。
附图标记：
11：车辆；      12：服务器；             13：车辆；
14：车辆；      40：上一帧三维点云；     30：目标对象；
31：目标对象；       32：目标对象；      342：目标对象；
345：目标对象；      351：目标对象；     362：目标对象；
376：目标对象；      80：二维图像；      81：目标对象；
82：目标对象；       83：目标对象；      91：三维外接框；
90：二维图像；       92：三维外接框；    93：三维外接框；
100：二维图像；      101：二维图像；     102：二维图像；
71：目标对象；       72：目标对象；
73：目标对象；       110：目标检测与跟踪系统；   111：探测设备；
112：存储器；        113：处理器。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
需要说明的是，当组件被称为“固定于”另一个组件，它可以直接在另一个组件上或者也可以存在居中的组件。当一个组件被认为是“连接”另一个组件，它可以是直接连接到另一个组件或者可能同时存在居中组件。
除非另有定义，本文所使用的所有的技术和科学术语与属于本申请的技术领域的技术人员通常理解的含义相同。本文中在本申请的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本申请。本文所使用的术语“及/或”包括一个或多个相关的所列项目的任意的和所有的组合。
下面结合附图，对本申请的一些实施方式作详细说明。在不冲突的情况下，下述的实施例及实施例中的特征可以相互组合。
本申请实施例提供一种目标检测与跟踪方法。该方法可应用于可移动平台，所述可移动平台设置有探测设备，所述探测设备用于探测所述可移动平台周围物体得到三维点云。可选的，所述探测设备包括但不限于激光雷达。
在本实施例中，该可移动平台可以是无人机、可移动机器人或车辆。本申请实施例以可移动平台是车辆为例，该车辆可以是无人驾驶车辆，或者是搭载有高级辅助驾驶(Advanced Driver Assistance Systems,ADAS)系统的车辆等。如图1所示，车辆11为搭载有探测设备的载体，该探测设备具体可以是双目立体相机、飞行时间测距法(Time of flight，TOF)相机和/或激光雷达。车辆11在行驶的过程中，探测设备实时探测车辆11周围物体得到三维点云。车辆11周围的物体包括车辆11周围的树木、行人、其他车辆，例如，车辆13和车辆14等。
以激光雷达为例，当该激光雷达发射出的一束激光照射到物体表面时，该物体表面将会对该束激光进行反射，该激光雷达根据该物体表面反射的激光，可确定该物体相对于该激光雷达的方位、距离等信息。若该激光雷 达发射出的该束激光按照某种轨迹进行扫描，例如360度旋转扫描，将得到大量的激光点，因而就可形成该物体的激光点云数据，也就是三维点云。
另外，本实施例并不限定目标检测与跟踪方法的执行主体，该目标检测与跟踪方法可以由车辆中的车载设备执行，也可以由车载设备之外的其他具有数据处理功能的设备执行，例如，如图1所示的服务器12，车辆11和服务器12可进行无线通信或有线通信，车辆11可以将探测设备探测获得的三维点云发送给服务器12，由服务器12执行该目标检测与跟踪方法。下面以车载设备为例对本申请实施例提供的目标检测与跟踪方法进行介绍。其中，车载设备可以是集成在车辆中控台中的具有数据处理功能的设备，或者也可以是放置在车辆内的平板电脑、手机、笔记本电脑等。
图2为本申请实施例提供的目标检测与跟踪方法的流程图。如图2所示，本实施例中的方法，可以包括：
步骤S201、获取上一帧三维点云和当前帧三维点云。
如图1所示，车辆11在行驶过程中，车辆11上搭载的探测设备实时探测车辆11周围物体得到三维点云，该探测设备可以和该车辆11上的车载设备通信连接，从而使得该车辆11上的车载设备可以实时获取到该探测设备探测得到的三维点云，例如，该探测设备在前一时刻探测得到的三维点云记为P0，该探测设备在当前时刻探测得到的三维点云记为P1，相应的，将三维点云P0记为上一帧三维点云，将三维点云P1记为当前帧三维点云。在其他实施例中，上一帧三维点云还可以是探测设备在一小段历史时间段内累积探测得到的三维点云，当前帧三维点云可以是探测设备在一小段当前时间段内累积探测得到的三维点云。
步骤S202、检测所述上一帧三维点云中的目标对象，得到所述目标对象对应的第一检测信息。
由于探测设备探测得到的三维点云包括车辆11周围物体的三维点云，例如，车辆11周围的物体可能包括树木、行人、其他车辆例如车辆13、车辆14等，因此，探测设备探测得到的三维点云中包括车辆11周围树木的三维点云、行人的三维点云、其他车辆例如车辆13、车辆14的三维点云。
例如，上一帧三维点云包括N个点，每个点包括位置信息和反射率。 其中，每个点的位置信息可以是该点在三维坐标系中的三维坐标(x,y,z)，本实施例并不对该三维坐标系进行限定，例如，该三维坐标系具体可以是车体坐标系、地球坐标系、或世界坐标系等。在本实施例中，具体可采用目标检测算法来检测该上一帧三维点云中的目标对象，例如，采用基于深度学习的目标检测算法来检测该上一帧三维点云中的目标对象，得到目标对象对应的第一检测信息。
其中，基于深度学习的目标检测算法的流程图如图3所示，将上一帧三维点云作为输入，通过输入预处理将无序的上一帧三维点云处理成第一卷积神经网络要求的有序输入，例如，将上一帧三维点云处理成一定大小的张量，此处的张量可以理解为高维矩阵，高维矩阵也就是大于二维的矩阵，此处的高维矩阵具体以三维矩阵为例。另外，张量的大小可表示为C*H*W，其中，C表示第一卷积神经网络输入的通道数、H表示高度、W表示宽度。若将上一帧三维点云沿着三维坐标系的Z轴投影到二维平面上，得到二维图像，则该二维图像的宽对应于前面所述的高度H，该二维图像的长对应于前面所述的宽度W。其中，第一卷积神经网络用于进行目标检测。进一步，第一卷积神经网络对一定大小的张量进行处理，以检测该上一帧三维点云中的目标对象，经过输出后处理得到目标检测结果即目标对象对应的检测信息。
由于在后续步骤中还需要对当前帧三维点云中的目标对象进行检测，得到当前帧三维点云中的目标对象的检测信息。因此，为了将上一帧三维点云中目标对象对应的检测信息和当前帧三维点云中目标对象对应的检测信息进行区分，将上一帧三维点云中目标对象对应的检测信息记为第一检测信息，将前帧三维点云中目标对象对应的检测信息记为第二检测信息。
可选的，所述目标对象对应的第一检测信息包括如下至少一种：所述目标对象的第一位置、第一大小、第一方向、所属类别、所述目标对象属于所述类别的第一概率值。
如图4所示，40表示探测设备探测得到的上一帧三维点云，经过如上所述的目标检测算法处理后，可检测出上一帧三维点云40中的目标对象，以及上一帧三维点云40中目标对象的第一检测信息。该目标对象可以是车辆11周围的物体对应的三维点云构成的点云簇。例如，目标对象30是 车辆11周围地面点云构成的点云簇，目标对象31是车辆11周围车辆14对应的三维点云构成的点云簇，目标对象32是车辆11周围车辆13对应的三维点云构成的点云簇。此处不限定从上一帧三维点云中检测到的目标对象的个数，图4所示的几个目标对象只是一种示意性说明。
可以理解的是，目标对象对应的检测信息，例如第一检测信息和第二检测信息可以有多种表示形式，如图5所示的表示形式只是一种示意性说明。在如图5所示的表示形式下，以本车也就是如上所述的车辆11的前向为X轴、以车辆11的右侧为Y轴、以车辆11的下方指向地面的方向为Z轴建立三维坐标系，该三维坐标系为车体坐标系。以如上所述的上一帧三维点云为例，基于如上所述的目标检测算法检测出的目标对象对应的第一检测信息还可以包括目标对象的标识信息，例如图5所示的标号342、345、351、362、376为上一帧三维点云中的多个目标对象的标识信息，也就是说，上一帧三维点云中包括目标对象342、目标对象345、目标对象351、目标对象362、目标对象376。
在本实施例中，目标对象的位置、大小、方向可通过该目标对象的三维外接框来表示。例如图5所示的目标对象342的位置、大小、方向可通过目标对象342的三维外接框来表示，该三维外接框可记为box，该三维外接框在该车体坐标系中的坐标可记为[x0,x1,x2,x3,y0,y1,y2,y3,zmin,zmax]。
其中，(x0,y0),(x1,y1),(x2,y2),(x3,y3)为该三维外接框在俯视图下的4个顶点。zmin为该三维外接框在该车体坐标系Z轴上的最小坐标值，zmax为该三维外接框在该车体坐标系Z轴上的最大坐标值。
另外，将目标对象所属的类别可记为class，目标对象属于该类别的概率值可记为score。该类别可以包括：道路标示线、车辆、行人、树木、道路标识牌等。例如，图5所示的不同目标对象所属的类别可能不同，例如，目标对象342所属的类别为车辆，目标对象376所属的类别为树木。目标对象342属于车辆的概率值为score1，目标对象376属于树木的概率值为score2。
步骤S203、检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息。
具体的，检测当前帧三维点云中的目标对象的过程类似于如上所述的检测上一帧三维点云中的目标对象的过程，此处不再赘述。
可选的，所述目标对象对应的第二检测信息包括如下至少一种：所述目标对象的第二位置、第二大小、第二方向、所属类别、所述目标对象属于所述类别的第二概率值。
可以理解的是，由于目标对象可能是移动的，因此，当前时刻检测到的目标对象对应的第二检测信息和前一时刻检测到的目标对象对应的第一检测信息可能会不同。例如，当前时刻检测到的目标对象的第二位置和前一时刻检测到的目标对象的第一位置可能不同。当前时刻检测到的目标对象的第二大小和前一时刻检测到的目标对象的第一大小可能不同。当前时刻检测到的目标对象的第二方向和前一时刻检测到的目标对象的第一方向可能不同。当前时刻检测到的目标对象所属的类别和前一时刻检测到的目标对象所属的类别可能不同，也可能相同，此处以类别相同为例进行示意性说明。另外，当前时刻检测到的目标对象属于某类别的第二概率值和前一时刻检测到的目标对象属于该类别的第一概率值可能不同。
步骤S204、根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
如图6所示，第一卷积神经网络用于目标检测，第二卷积神经网络用于目标跟踪。例如，左侧的第一卷积神经网络用于检测上一帧三维点云中的目标对象，右侧的第一卷积神经网络用于检测当前帧三维点云中的目标对象。通过第一卷积神经网络对上一帧三维点云进行目标检测的过程与通过第一卷积神经网络对当前帧三维点云进行目标检测的过程类似，此处不再赘述。另外，第一卷积神经网络可包括n个卷积层，不同的卷积层进行的处理计算过程可能不同，也可能相同。具体的，卷积层1的输出可以是卷积层2的输入，卷积层2的输出可以是卷积层3的输入，以此类推。同一侧的卷积层1、卷积层2、…、卷积层n的处理计算过程可能不同，也可能相同。
进一步，获取如图6所示的左右两侧第一卷积神经网络的中间层的输出，例如，获取左侧第一卷积神经网络的卷积层2的输出和右侧第一卷积 神经网络的卷积层2的输出，并对左右两侧第一卷积神经网络的中间层的输出进行特征融合，得到融合后的特征。可以理解，此处只是以左右两侧中每侧的一个中间层为例进行示意性说明，在其他实施例中，还可以分别获取左右两侧第一卷积神经网络的多个中间层的输出，例如，获取左侧第一卷积神经网络的卷积层2和卷积层3的输出、以及右侧第一卷积神经网络的卷积层2和卷积层3的输出，并对左侧卷积层2和卷积层3的输出、以及右侧卷积层2和卷积层3的输出进行特征融合，得到融合后的特征。此外，在其他一些实施例中，还可以获取左右两侧第一卷积神经网络中顶层的卷积层1和/或底层的卷积层n的输出。
进一步，将融合后的特征输入到第二卷积神经网络中，由第二卷积神经网络得到两帧之间目标对象的位置变化，也就是目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。此处的目标对象可以是上一帧三维点云和当前帧三维点云中泛指的目标对象。
步骤S205、根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪。
如图6所示，在确定出两帧之间目标对象的位置变化后，进一步，根据两帧之间目标对象的位置变化、第一检测信息和第二检测信息，进行跟踪后处理，也就是对目标对象进行目标跟踪，从而得到目标对象的标识信息。
另外，可以理解的是，上一帧三维点云中的目标对象和当前帧三维点云中的目标对象可能部分相同，例如，如图5所示的目标对象342、目标对象345、目标对象351、目标对象362、目标对象376是上一帧三维点云中的目标对象。在当前帧三维点云中可能会检测到目标对象345、目标对象351、目标对象362、目标对象376，而检测不到目标对象342。在一些实施例中，在当前帧三维点云中还有可能检测到新的目标对象，即上一帧三维点云中没有出现过的目标对象。本实施例所述的目标跟踪，不仅可以对上一帧三维点云和当前帧三维点云中共有的目标对象进行跟踪，也可以对只出现在上一帧三维点云或当前帧三维点云中的目标对象进行跟踪。
本实施例通过探测设备探测得到的三维点云，对三维点云中的目标对象同时进行目标检测和目标跟踪，也就是说，目标检测和目标跟踪采用相 同的输入，因此，从输入中提取到的特征也是相似或相同的，这些相似或相同的特征可以是目标检测和目标跟踪所共用的，因此，这些相似或相同的特征节省了重复计算量，避免了资源浪费。
本申请实施例提供一种目标检测与跟踪方法。图7为本申请另一实施例提供的目标检测与跟踪方法的流程图。如图7所示，在上述实施例的基础上，所述根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪，可以包括：
步骤S701、根据所述位置变化和所述目标对象的第一位置，确定所述目标对象在所述当前帧三维点云中的预测位置。
如图8所示，80表示将上一帧三维点云沿着三维坐标系的Z轴进行投影后得到的二维图像，90表示将当前帧三维点云沿着三维坐标系的Z轴进行投影后得到的二维图像。目标对象81、目标对象82和目标对象83表示上一帧三维点云中的目标对象。以目标对象81为例，将目标对象81对应的三维外接框记为box0，将目标对象81在上一帧三维点云和当前帧三维点云之间的位置变化记为△box。根据box0和△box可预测目标对象81在当前帧三维点云中的预测位置，该预测位置可以理解成目标对象81经过位置变化△box后预计在当前帧三维点云中出现的位置，目标对象81在该预测位置上对应的三维外接框记为box1，box1＝box0+△box，box1具体可以为如图8所示的三维外接框91。同理，可以确定出上一帧三维点云中的目标对象82和目标对象83分别在当前帧三维点云中的预测位置，以及相应预测位置上的三维外接框，例如，目标对象82在相应预测位置上的三维外接框为三维外接框92，目标对象83在相应预测位置上的三维外接框为三维外接框93。也就是说，上一帧三维点云中的每个目标对象在当前帧三维点云中分别对应有一个预测位置。例如，目标对象81和三维外接框91对应，目标对象82和三维外接框92对应，目标对象83和三维外接框93对应。
步骤S702、根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪。
如图8所示，目标对象A、目标对象B、目标对象C和目标对象D是 通过如上所述的目标检测算法检测出的当前帧三维点云中的目标对象。目标对象A、目标对象B、目标对象C和目标对象D的位置分别记为第二位置。
对于同一个目标对象而言，该目标对象在当前帧三维点云中的预测位置和实际检测到的该目标对象在当前帧三维点云中的位置应该是非常接近的。例如，如果目标对照81和目标对照A是同一个目标对象，则目标对照81在相应预测位置上的三维外接框91与目标对照A应该是非常接近的。因此，可以通过预测位置上的三维外接框和当前帧三维点云中实际检测到的目标对象对应的三维外接框之间的距离来确定哪两个目标对象是同一个目标对象。
可选的，所述根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪，包括：若所述预测位置和所述目标对象的第二位置之间的距离小于预设距离，则将所述目标对象在所述上一帧三维点云中的标识信息作为所述目标对象在所述当前帧三维点云中的标识信息。
例如，将目标对象A对应的三维外接框记为box2，计算box1和box2之间的距离，当该距离小于预设距离时，则确定目标对象81和目标对象A是同一个目标对象，因此，将目标对象81的标识号81作为目标对象A在当前帧三维点云中的标识号，即实现了目标对象81和目标对象A的关联，也就是说，当前帧三维点云中的目标对象A是上一帧三维点云中的目标对象81，从而实现了对目标对象A的目标跟踪。同理，可以确定出当前帧三维点云中与目标对象82关联的目标对象B、以及与目标对象83关联的目标对象C，将目标对象82的标识号82作为目标对象B在当前帧三维点云中的标识号，将目标对象83的标识号83作为目标对象C在当前帧三维点云中的标识号，从而实现了对目标对象B和目标对象C的目标跟踪。
另外，如图8所示，目标对象D是当前帧三维点云中新出现的目标对象，也就是说，在上一帧三维点云中不存在与目标对象D关联的目标对象，因此，可以给目标对象D赋予一个新的标识号，例如84。
本实施例通过上一帧三维点云中的目标对象在当前帧三维点云中的预测位置，判断该预测位置和在当前帧三维点云中检测到的目标对象的位置之间的距离是否小于预设距离，若是，则确定上一帧三维点云中的目标 对象和当前帧三维点云中检测到的目标对象是同一个目标对象，将该目标对象在上一帧三维点云中的标识信息作为该目标对象在当前帧三维点云中的标识信息，从而实现了对目标对象的目标跟踪。
本申请实施例提供一种目标检测与跟踪方法。图9为本申请另一实施例提供的目标检测与跟踪方法的流程图。如图9所示，在上述实施例的基础上，所述根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化，可以包括：
步骤S901、获取所述上一帧三维点云对应的第一特征信息、以及所述当前帧三维点云对应的第二特征信息。
可选的，所述第一特征信息是采用第一卷积神经网络检测所述上一帧三维点云中的目标对象时，所述第一卷积神经网络的至少一个卷积层的输出信息；所述第二特征信息是采用所述第一卷积神经网络检测所述当前帧三维点云中的目标对象时，所述第一卷积神经网络的所述至少一个卷积层的输出信息。
例如，第一特征信息可以是如图6所示的左侧第一卷积神经网络的卷积层2的输出信息，第二特征信息可以是如图6所示的右侧第一卷积神经网络的卷积层2的输出信息。例如，将第一特征信息记为feature0，feature0的维度为c1*H*W。将第二特征信息记为feature1，feature1的维度为c1*H*W。
步骤S902、对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息。
例如，对第一特征信息feature0和第二特征信息feature1进行特征融合，得到融合后的第三特征信息。
可选的，所述对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息，包括：对所述第一特征信息和所述第二特征信息进行相关性计算，得到所述第一特征信息和所述第二特征信息的相关性信息；根据所述相关性信息、所述第一特征信息和所述第二特征信息，得到所述融合后的第三特征信息。
例如，对第一特征信息feature0和第二特征信息feature1进行相关性(correlation)计算，得到feature0和feature1之间的相关性信息，该相关性信息的维度记为c2*H*W。进一步，再将该相关性信息、feature0和feature1进行拼接得到融合后的第三特征信息，融合后的第三特征信息记为fused_feature，fused_feature的维度为(2*c1+c2)*H*W。
其中，correlation具体可以是获取相邻帧时序变化信息的一种操作，例如，将第一特征信息feature0表示为
Figure PCTCN2019111730-appb-000001
将第二特征信息feature1表示为
Figure PCTCN2019111730-appb-000002
其中，t表示上一帧对应的时间，t+τ表示当前帧对应的时间，l表示卷积层的层数，(i,j)表示目标对象的位置，(p,q)表示目标对象在上一帧三维点云和当前帧三维点云之间的位置变化。feature0和feature1之间的相关性信息记为
Figure PCTCN2019111730-appb-000003
Figure PCTCN2019111730-appb-000004
之间的关系可表示为如下公式(1)：
Figure PCTCN2019111730-appb-000005
其中，-d≤p≤d、-d≤q≤d，
Figure PCTCN2019111730-appb-000006
表示
Figure PCTCN2019111730-appb-000007
和
Figure PCTCN2019111730-appb-000008
的内积。其中，(2d+1) 2＝c2，d表示进行相关性计算中的偏移大小，在一种实施方式中，d可以为一预设值。
步骤S903、根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
可选的，所述根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化，包括：将所述第三特征信息输入第二卷积神经网络，通过所述第二卷积神经网络确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
例如，将融合后的第三特征信息fused_feature输入第二卷积神经网络中，由第二卷积神经网络得到两帧之间目标对象的位置变化。
本实施例通过将上一帧三维点云对应的第一特征信息和当前帧三维点云对应的第二特征信息进行融合，得到融合后的第三特征信息，并根据第三特征信息来确定目标对象在上一帧三维点云和当前帧三维点云之间的位置变化，也就是说，根据目标检测的中间结果以确定目标跟踪时目标 对象的位置变化，实现了目标检测和目标跟踪的有效结合，避免目标检测和目标跟踪分别独立运行，提高了资源浪费。
本申请实施例提供一种目标检测与跟踪方法。在上述实施例的基础上，所述检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息，可以包括：采用第一卷积神经网络检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息。
如图6所示，采用右侧的第一卷积神经网络检测当前帧三维点云中的目标对象，得到目标对象对应的第二检测信息。
所述方法还包括：根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正。可选的，所述跟踪信息包括可跟踪到所述目标对象的历史三维点云的帧数、所述目标对象在每个所述历史三维点云中的历史检测信息。可选的，所述历史检测信息包括如下至少一种：所述目标对象的历史位置、历史大小、历史方向、所属类别、所述目标对象属于所述类别的历史概率值。
如图10所示，100表示探测设备在历史时刻t0探测得到的三维点云沿着三维坐标系的Z轴进行投影后得到的二维图像，101表示探测设备在历史时刻t1探测得到的三维点云沿着三维坐标系的Z轴进行投影后得到的二维图像，102表示探测设备在当前时刻t2探测得到的三维点云沿着三维坐标系的Z轴进行投影后得到的二维图像。其中，目标对象71、目标对象81和目标对象A是同一个目标对象，目标对象72、目标对象82和目标对象B是同一个目标对象，目标对象73、目标对象83和目标对象C是同一个目标对象。通过对目标对象A、目标对象B和目标对象C的目标跟踪，可得到目标对象A、目标对象B和目标对象C中每个目标对象对应的跟踪信息，该跟踪信息可记为tracklet。例如，目标对象71的三维外接框记为box01，目标对象71所属的类型为车辆，目标对象71属于车辆的概率值记为score01。目标对象81的三维外接框记为box02，目标对象81所属的类型为车辆，目标对象81属于车辆的概率值记为score02。目标对象A的三维外接框记为box03，目标对象A所属的类型为车辆，目标对象A属于车辆的概率值记为score03。目标对象A对应的tracklet具体可以是 box01和score01、box02和score02构成的序列。也就是说，目标对象A对应的tracklet具体可以是目标对象A在历史三维点云中的历史检测信息。另外，目标对象A对应的tracklet还可以包括历史时间中能跟踪到目标对象A的历史三维点云的帧数，该帧数记为N。
同理，可以得到目标对象B对应的tracklet和目标对象C对应的tracklet，此处不再一一赘述。
在本实施例中，还可以进一步根据每个目标对象对应的tracklet，对该目标对象在当前帧三维点云中对应的第二检测信息进行修正。例如，根据目标对象A对应的tracklet，对目标对象A在当前帧三维点云中对应的box03和score03进行修正，并将修正后的box03和修正后的score03存入目标对象A对应的tracklet中。
在一种可能的实现方式中，所述根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正，包括：根据所述目标对象属于所述类别的历史概率值，对所述目标对象属于所述类别的第二概率值进行修正。
例如，目标对象A对应的tracklet包括box01和score01、box02和score02，进一步，计算score01和score02的和值得到累计概率值SUM_SCORE，对score03进行修正后的值记为score03’，score03’可通过如下公式(2)计算得到：
score03’＝(1-α)*score03+α*SUM_SCORE/N    (2)
其中，0≤α≤1。
由于目标对象71、目标对象81和目标对象A是同一个目标对象，因此，需要将修正后的score03’记录到目标对象A对应的tracklet中，同时，目标对象A对应的tracklet包括的N值加1。更新后的累计概率值SUM_SCORE需要加上score03’。
另外，在一些实施例中，score01和score02也可以是根据类似于公式(2)所述的方法修正后的概率值。
在其他实施例中，如果在历史三维点云中均没有与目标对象A关联的目标对象，则需要建立一个目标对象A对应的tracklet，并将box03和score03存入目标对象A对应的tracklet中，对该tracklet对应的N和SUM_SCORE的更新方法如上所述，此处不再赘述。
在另一种可能的实现方式中，所述根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正，包括如下至少一种：根据所述目标对象的历史位置对所述目标对象的第二位置进行修正；根据所述目标对象的历史大小对所述目标对象的第二大小进行修正；根据所述目标对象的历史方向对所述目标对象的第二方向进行修正。
例如，目标对象A对应的tracklet包括box01和score01、box02和score02，进一步，根据box01和box02可以对box03进行修正，例如，box01对应于目标对象71的位置、方向、大小，box02对应于目标对象81的位置、方向、大小，box03对应于目标对象A的位置、方向、大小，具体的，根据box01和box02对box03进行修正可包括如下至少一种：根据目标对象71的位置和目标对象81的位置对目标对象A的位置进行修正；根据目标对象71的方向和目标对象81的方向对目标对象A的方向进行修正；根据目标对象71的大小和目标对象81的大小对目标对象A的大小进行修正，从而得到修正后的box03，此处，将修正后的box03记为box03’，进一步，将box03’存入目标对象A对应的tracklet中。
另外，在一些实施例中，box01和box02也可以是修正后的三维外接框。
本实施例通过对目标对象进行目标跟踪得到的跟踪信息，对该目标对象的检测信息进行修正，由于跟踪信息中可包括该目标对象对应的时序信息，例如，可跟踪到该目标对象的历史三维点云的帧数、以及该目标对象在每个历史三维点云中的历史检测信息，因此，通过该目标对象的时序信息对该目标对象的检测信息进行修正，可提高对该目标对象的检测精度。例如，若本车的很远处有一辆车，通过单帧的三维点云很难判断出本车的很远处是否有其他车辆，但是，通过对远处车辆进行目标跟踪，确定出能够跟踪到该远处车辆的多帧历史三维点云，以及该远处车辆在每个历史三维点云中的历史检测信息，并根据该远处车辆的跟踪信息，对远处车辆在当前帧三维点云中的检测信息进行修正，即可准确的确定出当前帧三维点云中是否存在该远处车辆。
本申请实施例提供一种目标检测与跟踪系统。图11为本申请实施例 提供的目标检测与跟踪系统的结构图，如图11所示，目标检测与跟踪系统110包括：探测设备111、存储器112和处理器113。其中，探测设备111用于探测可移动平台周围物体得到三维点云。处理器113具体可以是上述实施例中车载设备中的部件，或者是车辆中搭载的具有数据处理功能的其他部件、器件或组件。具体的，存储器112用于存储程序代码；处理器113，调用所述程序代码，当程序代码被执行时，用于执行以下操作：获取上一帧三维点云和当前帧三维点云；检测所述上一帧三维点云中的目标对象，得到所述目标对象对应的第一检测信息；检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息；根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化；根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪。
可选的，所述目标对象对应的第一检测信息包括如下至少一种：所述目标对象的第一位置、第一大小、第一方向、所属类别、所述目标对象属于所述类别的第一概率值。
可选的，所述目标对象对应的第二检测信息包括如下至少一种：所述目标对象的第二位置、第二大小、第二方向、所属类别、所述目标对象属于所述类别的第二概率值。
可选的，处理器113根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪时，具体用于：根据所述位置变化和所述目标对象的第一位置，确定所述目标对象在所述当前帧三维点云中的预测位置；根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪。
可选的，处理器113根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪时，具体用于：若所述预测位置和所述目标对象的第二位置之间的距离小于预设距离，则将所述目标对象在所述上一帧三维点云中的标识信息作为所述目标对象在所述当前帧三维点云中的标识信息。
可选的，处理器113根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的 位置变化时，具体用于：获取所述上一帧三维点云对应的第一特征信息、以及所述当前帧三维点云对应的第二特征信息；对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息；根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
可选的，所述第一特征信息是采用第一卷积神经网络检测所述上一帧三维点云中的目标对象时，所述第一卷积神经网络的至少一个卷积层的输出信息；所述第二特征信息是采用所述第一卷积神经网络检测所述当前帧三维点云中的目标对象时，所述第一卷积神经网络的所述至少一个卷积层的输出信息。
可选的，处理器113对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息时，具体用于：对所述第一特征信息和所述第二特征信息进行相关性计算，得到所述第一特征信息和所述第二特征信息的相关性信息；根据所述相关性信息、所述第一特征信息和所述第二特征信息，得到所述融合后的第三特征信息。
可选的，处理器113根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化时，具体用于：将所述第三特征信息输入第二卷积神经网络，通过所述第二卷积神经网络确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
可选的，处理器113检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息时，具体用于：采用第一卷积神经网络检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息。
可选的，处理器113还用于：根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正。
可选的，所述跟踪信息包括可跟踪到所述目标对象的历史三维点云的帧数、所述目标对象在每个所述历史三维点云中的历史检测信息。
可选的，所述历史检测信息包括如下至少一种：所述目标对象的历史位置、历史大小、历史方向、所属类别、所述目标对象属于所述类别的历 史概率值。
可选的，处理器113根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正时，具体用于：根据所述目标对象属于所述类别的历史概率值，对所述目标对象属于所述类别的第二概率值进行修正。
可选的，处理器113根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正时，具体用于如下至少一种：根据所述目标对象的历史位置对所述目标对象的第二位置进行修正；根据所述目标对象的历史大小对所述目标对象的第二大小进行修正；根据所述目标对象的历史方向对所述目标对象的第二方向进行修正。
可选的，所述探测设备包括激光雷达。
本申请实施例提供的目标检测与跟踪系统的具体原理和实现方式均与上述实施例类似，此处不再赘述。
本申请实施例提供一种可移动平台。该可移动平台包括：机身、动力系统和如上实施例所述的目标检测与跟踪系统。其中，动力系统安装在所述机身，用于提供移动动力。目标检测与跟踪系统可以实现如上所述的目标检测与跟踪方法，该目标检测与跟踪方法的具体原理和实现方式均与上述实施例类似，此处不再赘述。本实施例并不限定该可移动平台的具体形态，例如，该可移动平台可以是无人机、可移动机器人或车辆等。
另外，本实施例还提供一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行以实现上述实施例所述的目标检测与跟踪方法。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。
上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器(processor)执行本申请各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
本领域技术人员可以清楚地了解到，为描述的方便和简洁，仅以上述各功能模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能模块完成，即将装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。上述描述的装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (35)

  1. 一种目标检测与跟踪方法，其特征在于，应用于可移动平台，所述可移动平台设置有探测设备，所述探测设备用于探测所述可移动平台周围物体得到三维点云，所述方法包括：
    获取上一帧三维点云和当前帧三维点云；
    检测所述上一帧三维点云中的目标对象，得到所述目标对象对应的第一检测信息；
    检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息；
    根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化；
    根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪。
  2. 根据权利要求1所述的方法，其特征在于，所述目标对象对应的第一检测信息包括如下至少一种：
    所述目标对象的第一位置、第一大小、第一方向、所属类别、所述目标对象属于所述类别的第一概率值。
  3. 根据权利要求1所述的方法，其特征在于，所述目标对象对应的第二检测信息包括如下至少一种：
    所述目标对象的第二位置、第二大小、第二方向、所属类别、所述目标对象属于所述类别的第二概率值。
  4. 根据权利要求1所述的方法，其特征在于，所述根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪，包括：
    根据所述位置变化和所述目标对象的第一位置，确定所述目标对象在所述当前帧三维点云中的预测位置；
    根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪，包括：
    若所述预测位置和所述目标对象的第二位置之间的距离小于预设距离，则将所述目标对象在所述上一帧三维点云中的标识信息作为所述目标对象在所述当前帧三维点云中的标识信息。
  6. 根据权利要求1所述的方法，其特征在于，所述根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化，包括：
    获取所述上一帧三维点云对应的第一特征信息、以及所述当前帧三维点云对应的第二特征信息；
    对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息；
    根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
  7. 根据权利要求6所述的方法，其特征在于，所述第一特征信息是采用第一卷积神经网络检测所述上一帧三维点云中的目标对象时，所述第一卷积神经网络的至少一个卷积层的输出信息；
    所述第二特征信息是采用所述第一卷积神经网络检测所述当前帧三维点云中的目标对象时，所述第一卷积神经网络的所述至少一个卷积层的输出信息。
  8. 根据权利要求6或7所述的方法，其特征在于，所述对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息，包括：
    对所述第一特征信息和所述第二特征信息进行相关性计算，得到所述第一特征信息和所述第二特征信息的相关性信息；
    根据所述相关性信息、所述第一特征信息和所述第二特征信息，得到所述融合后的第三特征信息。
  9. 根据权利要求8所述的方法，其特征在于，所述根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化，包括：
    将所述第三特征信息输入第二卷积神经网络，通过所述第二卷积神经网络确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之 间的位置变化。
  10. 根据权利要求1所述的方法，其特征在于，所述检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息，包括：
    采用第一卷积神经网络检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息。
  11. 根据权利要求10所述的方法，其特征在于，所述方法还包括：
    根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正。
  12. 根据权利要求11所述的方法，其特征在于，所述跟踪信息包括可跟踪到所述目标对象的历史三维点云的帧数、所述目标对象在每个所述历史三维点云中的历史检测信息。
  13. 根据权利要求12所述的方法，其特征在于，所述历史检测信息包括如下至少一种：
    所述目标对象的历史位置、历史大小、历史方向、所属类别、所述目标对象属于所述类别的历史概率值。
  14. 根据权利要求13所述的方法，其特征在于，所述根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正，包括：
    根据所述目标对象属于所述类别的历史概率值，对所述目标对象属于所述类别的第二概率值进行修正。
  15. 根据权利要求13所述的方法，其特征在于，所述根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正，包括如下至少一种：
    根据所述目标对象的历史位置对所述目标对象的第二位置进行修正；
    根据所述目标对象的历史大小对所述目标对象的第二大小进行修正；
    根据所述目标对象的历史方向对所述目标对象的第二方向进行修正。
  16. 根据权利要求1所述的方法，其特征在于，所述探测设备包括激光雷达。
  17. 一种目标检测与跟踪系统，其特征在于，包括：探测设备、存储器和处理器；
    所述探测设备用于探测可移动平台周围物体得到三维点云；
    所述存储器用于存储程序代码；
    所述处理器，调用所述程序代码，当程序代码被执行时，用于执行以 下操作：
    获取上一帧三维点云和当前帧三维点云；
    检测所述上一帧三维点云中的目标对象，得到所述目标对象对应的第一检测信息；
    检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息；
    根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化；
    根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪。
  18. 根据权利要求17所述的系统，其特征在于，所述目标对象对应的第一检测信息包括如下至少一种：
    所述目标对象的第一位置、第一大小、第一方向、所属类别、所述目标对象属于所述类别的第一概率值。
  19. 根据权利要求17所述的系统，其特征在于，所述目标对象对应的第二检测信息包括如下至少一种：
    所述目标对象的第二位置、第二大小、第二方向、所属类别、所述目标对象属于所述类别的第二概率值。
  20. 根据权利要求17所述的系统，其特征在于，所述处理器根据所述位置变化、所述第一检测信息和所述第二检测信息，对所述目标对象进行目标跟踪时，具体用于：
    根据所述位置变化和所述目标对象的第一位置，确定所述目标对象在所述当前帧三维点云中的预测位置；
    根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪。
  21. 根据权利要求20所述的系统，其特征在于，所述处理器根据所述预测位置和所述目标对象的第二位置，对所述目标对象进行目标跟踪时，具体用于：
    若所述预测位置和所述目标对象的第二位置之间的距离小于预设距离，则将所述目标对象在所述上一帧三维点云中的标识信息作为所述目标 对象在所述当前帧三维点云中的标识信息。
  22. 根据权利要求17所述的系统，其特征在于，所述处理器根据所述上一帧三维点云和所述当前帧三维点云，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化时，具体用于：
    获取所述上一帧三维点云对应的第一特征信息、以及所述当前帧三维点云对应的第二特征信息；
    对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息；
    根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
  23. 根据权利要求22所述的系统，其特征在于，所述第一特征信息是采用第一卷积神经网络检测所述上一帧三维点云中的目标对象时，所述第一卷积神经网络的至少一个卷积层的输出信息；
    所述第二特征信息是采用所述第一卷积神经网络检测所述当前帧三维点云中的目标对象时，所述第一卷积神经网络的所述至少一个卷积层的输出信息。
  24. 根据权利要求22或23所述的系统，其特征在于，所述处理器对所述第一特征信息和所述第二特征信息进行特征融合，得到融合后的第三特征信息时，具体用于：
    对所述第一特征信息和所述第二特征信息进行相关性计算，得到所述第一特征信息和所述第二特征信息的相关性信息；
    根据所述相关性信息、所述第一特征信息和所述第二特征信息，得到所述融合后的第三特征信息。
  25. 根据权利要求24所述的系统，其特征在于，所述处理器根据所述第三特征信息，确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化时，具体用于：
    将所述第三特征信息输入第二卷积神经网络，通过所述第二卷积神经网络确定所述目标对象在所述上一帧三维点云和所述当前帧三维点云之间的位置变化。
  26. 根据权利要求17所述的系统，其特征在于，所述处理器检测所 述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息时，具体用于：
    采用第一卷积神经网络检测所述当前帧三维点云中的目标对象，得到所述目标对象对应的第二检测信息。
  27. 根据权利要求26所述的系统，其特征在于，所述处理器还用于：
    根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正。
  28. 根据权利要求27所述的系统，其特征在于，所述跟踪信息包括可跟踪到所述目标对象的历史三维点云的帧数、所述目标对象在每个所述历史三维点云中的历史检测信息。
  29. 根据权利要求28所述的系统，其特征在于，所述历史检测信息包括如下至少一种：
    所述目标对象的历史位置、历史大小、历史方向、所属类别、所述目标对象属于所述类别的历史概率值。
  30. 根据权利要求29所述的系统，其特征在于，所述处理器根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正时，具体用于：
    根据所述目标对象属于所述类别的历史概率值，对所述目标对象属于所述类别的第二概率值进行修正。
  31. 根据权利要求29所述的系统，其特征在于，所述处理器根据所述目标跟踪得到的跟踪信息，对所述第二检测信息进行修正时，具体用于如下至少一种：
    根据所述目标对象的历史位置对所述目标对象的第二位置进行修正；
    根据所述目标对象的历史大小对所述目标对象的第二大小进行修正；
    根据所述目标对象的历史方向对所述目标对象的第二方向进行修正。
  32. 根据权利要求17所述的系统，其特征在于，所述探测设备包括激光雷达。
  33. 一种可移动平台，其特征在于，包括：
    机身；
    动力系统，安装在所述机身，用于提供移动动力；
    以及权利要求17-32任一项所述的目标检测与跟踪系统。
  34. 根据权利要求33所述的可移动平台，其特征在于，所述可移动 平台包括：无人机、可移动机器人或车辆。
  35. 一种计算机可读存储介质，其特征在于，其上存储有计算机程序，所述计算机程序被处理器执行以实现权利要求1-16任一项所述的方法。
PCT/CN2019/111730 2019-10-17 2019-10-17 目标检测与跟踪方法、系统、设备及存储介质 WO2021072709A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980034227.9A CN112166458B (zh) 2019-10-17 2019-10-17 目标检测与跟踪方法、系统、设备及存储介质
PCT/CN2019/111730 WO2021072709A1 (zh) 2019-10-17 2019-10-17 目标检测与跟踪方法、系统、设备及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/111730 WO2021072709A1 (zh) 2019-10-17 2019-10-17 目标检测与跟踪方法、系统、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2021072709A1 true WO2021072709A1 (zh) 2021-04-22

Family

ID=73860462

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/111730 WO2021072709A1 (zh) 2019-10-17 2019-10-17 目标检测与跟踪方法、系统、设备及存储介质

Country Status (2)

Country Link
CN (1) CN112166458B (zh)
WO (1) WO2021072709A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808161A (zh) * 2021-08-06 2021-12-17 航天时代飞鹏有限公司 基于机器视觉的车载多旋翼无人机车辆跟踪方法
CN114002708A (zh) * 2021-10-20 2022-02-01 华中科技大学 一种面向无人艇应用的尾浪滤除方法
WO2023072269A1 (zh) * 2021-10-29 2023-05-04 上海商汤智能科技有限公司 对象跟踪
CN117037120A (zh) * 2023-10-09 2023-11-10 之江实验室 一种基于时序选择的目标感知方法及装置
CN117111019A (zh) * 2023-10-25 2023-11-24 深圳市先创数字技术有限公司 一种基于雷达探测的目标跟踪监控方法和系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113205552A (zh) * 2021-04-28 2021-08-03 北京亮道智能汽车技术有限公司 一种目标检测方法、装置、电子设备及存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140118716A1 (en) * 2012-10-31 2014-05-01 Raytheon Company Video and lidar target detection and tracking system and method for segmenting moving targets
CN106127802A (zh) * 2016-06-16 2016-11-16 南京邮电大学盐城大数据研究院有限公司 一种运动目标轨迹追踪方法
CN107341819A (zh) * 2017-05-09 2017-11-10 深圳市速腾聚创科技有限公司 目标跟踪方法及存储介质
CN108446585A (zh) * 2018-01-31 2018-08-24 深圳市阿西莫夫科技有限公司 目标跟踪方法、装置、计算机设备和存储介质
CN110018489A (zh) * 2019-04-25 2019-07-16 上海蔚来汽车有限公司 基于激光雷达的目标追踪方法、装置及控制器和存储介质
CN110210389A (zh) * 2019-05-31 2019-09-06 东南大学 一种面向道路交通场景的多目标识别跟踪方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109559330B (zh) * 2017-09-25 2021-09-10 北京金山云网络技术有限公司 运动目标的视觉跟踪方法、装置、电子设备及存储介质
CN110176024B (zh) * 2019-05-21 2023-06-02 腾讯科技（深圳）有限公司 在视频中对目标进行检测的方法、装置、设备和存储介质
CN113538519B (zh) * 2019-06-25 2024-05-17 北京市商汤科技开发有限公司 目标追踪方法及装置、电子设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140118716A1 (en) * 2012-10-31 2014-05-01 Raytheon Company Video and lidar target detection and tracking system and method for segmenting moving targets
CN106127802A (zh) * 2016-06-16 2016-11-16 南京邮电大学盐城大数据研究院有限公司 一种运动目标轨迹追踪方法
CN107341819A (zh) * 2017-05-09 2017-11-10 深圳市速腾聚创科技有限公司 目标跟踪方法及存储介质
CN108446585A (zh) * 2018-01-31 2018-08-24 深圳市阿西莫夫科技有限公司 目标跟踪方法、装置、计算机设备和存储介质
CN110018489A (zh) * 2019-04-25 2019-07-16 上海蔚来汽车有限公司 基于激光雷达的目标追踪方法、装置及控制器和存储介质
CN110210389A (zh) * 2019-05-31 2019-09-06 东南大学 一种面向道路交通场景的多目标识别跟踪方法

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808161A (zh) * 2021-08-06 2021-12-17 航天时代飞鹏有限公司 基于机器视觉的车载多旋翼无人机车辆跟踪方法
CN113808161B (zh) * 2021-08-06 2024-03-15 航天时代飞鹏有限公司 基于机器视觉的车载多旋翼无人机车辆跟踪方法
CN114002708A (zh) * 2021-10-20 2022-02-01 华中科技大学 一种面向无人艇应用的尾浪滤除方法
WO2023072269A1 (zh) * 2021-10-29 2023-05-04 上海商汤智能科技有限公司 对象跟踪
CN117037120A (zh) * 2023-10-09 2023-11-10 之江实验室 一种基于时序选择的目标感知方法及装置
CN117037120B (zh) * 2023-10-09 2024-02-09 之江实验室 一种基于时序选择的目标感知方法及装置
CN117111019A (zh) * 2023-10-25 2023-11-24 深圳市先创数字技术有限公司 一种基于雷达探测的目标跟踪监控方法和系统
CN117111019B (zh) * 2023-10-25 2024-01-09 深圳市先创数字技术有限公司 一种基于雷达探测的目标跟踪监控方法和系统

Also Published As

Publication number Publication date
CN112166458A (zh) 2021-01-01
CN112166458B (zh) 2024-04-26

Similar Documents

Publication Publication Date Title
WO2021072709A1 (zh) 目标检测与跟踪方法、系统、设备及存储介质
KR102221695B1 (ko) 자율주행을 위한 고정밀 지도의 업데이트 장치 및 방법
US20210390329A1 (en) Image processing method, device, movable platform, unmanned aerial vehicle, and storage medium
CN109828592B (zh) 一种障碍物检测的方法及设备
Weon et al. Object Recognition based interpolation with 3d lidar and vision for autonomous driving of an intelligent vehicle
WO2021046716A1 (zh) 目标对象的检测方法、系统、设备及存储介质
CN111563450B (zh) 数据处理方法、装置、设备及存储介质
CN111968229A (zh) 高精度地图制图方法及装置
WO2020237516A1 (zh) 点云的处理方法、设备和计算机可读存储介质
CN110969064A (zh) 一种基于单目视觉的图像检测方法、装置及存储设备
CN110705385B (zh) 一种障碍物角度的检测方法、装置、设备及介质
CN109313809B (zh) 一种图像匹配方法、装置及存储介质
WO2021056516A1 (zh) 目标检测方法、设备及可移动平台
CN113706704B (zh) 基于高精地图规划路线的方法、设备以及自动驾驶车辆
CN109583312A (zh) 车道线识别方法、装置、设备及存储介质
CN115331199A (zh) 障碍物的检测方法、装置、电子设备及存储介质
CN111401190A (zh) 车辆检测方法、装置、计算机设备和存储介质
CN113763504B (zh) 地图更新方法、系统、车载终端、服务器及存储介质
CN111784579B (zh) 制图方法及装置
CN113671523A (zh) 机器人定位方法、装置、存储介质及机器人
KR20220131378A (ko) 포지셔닝 방법, 장치, 전자 기기 및 저장 매체
US11657506B2 (en) Systems and methods for autonomous robot navigation
CN114612895B (zh) 非标准道路场景中的道路检测方法及装置
CN117635721A (zh) 目标定位方法及相关系统、存储介质
JP2023173688A (ja) 移動体、事前地図生成方法、位置推定システム、位置推定方法、事前地図生成方法、およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19949002

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19949002

Country of ref document: EP

Kind code of ref document: A1