WO2022007451A1 - 目标检测方法、装置、计算机可读介质及电子设备 - Google Patents

目标检测方法、装置、计算机可读介质及电子设备 Download PDF

Info

Publication number
WO2022007451A1
WO2022007451A1 PCT/CN2021/085890 CN2021085890W WO2022007451A1 WO 2022007451 A1 WO2022007451 A1 WO 2022007451A1 CN 2021085890 W CN2021085890 W CN 2021085890W WO 2022007451 A1 WO2022007451 A1 WO 2022007451A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
cloud data
target
point
detected
Prior art date
Application number
PCT/CN2021/085890
Other languages
English (en)
French (fr)
Inventor
杨磊
吴凯
Original Assignee
北京京东乾石科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东乾石科技有限公司 filed Critical 北京京东乾石科技有限公司
Priority to US18/004,307 priority Critical patent/US20230222618A1/en
Priority to EP21838542.5A priority patent/EP4177836A4/en
Publication of WO2022007451A1 publication Critical patent/WO2022007451A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/0007—Image acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/778—Active pattern-learning, e.g. online learning of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to a target detection method, a target detection apparatus, a computer-readable medium, and an electronic device.
  • Object detection is an important direction of computer perspective and digital image processing, and is widely used in many fields such as robot navigation, intelligent video surveillance, and automatic driving.
  • point cloud data has problems of difficulty in labeling and high labeling cost, resulting in limited available datasets.
  • target detection relies heavily on labeled data, and the scale of labeled data will directly affect the final performance of the model. Therefore, using data augmentation to expand the given training data set has become an effective way to improve the accuracy of model detection.
  • Data enhancement methods are mainly divided into two types, one is to generate enhanced data based on structured data; the other is to enhance point cloud data based on database.
  • the first method needs to remove the static objects in the scene first, and then add dynamic objects to obtain the training data set.
  • Inner point cloud randomly select a part of the annotation frame from the extracted annotation frame and add it to the point cloud data of the current frame to obtain the enhanced training data set.
  • the training data set lacks authenticity, and the improvement effect on the detection accuracy is poor.
  • the purpose of the present disclosure is to provide a target detection method, a target detection device, a computer-readable medium, and an electronic device, which can overcome the difficulty of target labeling to a certain extent, thereby improving the accuracy of target detection.
  • a target detection method comprising: acquiring original point cloud data including a target to be detected, wherein the original point cloud data includes label information for the target to be detected; using the label information, extract the instance point cloud data corresponding to the target to be detected from the original point cloud data; determine the target position point from the original point cloud data, and analyze the original point cloud data based on the target position point Perform fusion processing with the instance point cloud data to obtain a sample to be detected after fusion processing; combine the original point cloud data and the sample to be detected to detect the target to be detected.
  • the present disclosure also discloses a target detection device, which includes: an original data acquisition module, an instance data acquisition module, a fusion processing module, and a target detection module, wherein: a raw data acquisition module is used to acquire a raw data containing a target to be detected.
  • the original point cloud data includes label information for the target to be detected
  • an instance data acquisition module is used to extract the to-be-detected object from the original point cloud data by using the label information The instance point cloud data corresponding to the target
  • the fusion processing module is used to determine the target position point from the original point cloud data, and perform fusion processing on the original point cloud data and the instance point cloud data based on the target position point , to obtain the sample to be detected after fusion processing
  • the target detection module is used to detect the target to be detected by combining the original point cloud data and the sample to be detected.
  • the present disclosure also discloses an electronic device comprising: one or more processors; and a memory having executable instructions stored thereon, which, when executed by the one or more processors, cause all
  • the electronic device performs the following steps: acquiring original point cloud data containing the target to be detected, wherein the original point cloud data includes label information for the target to be detected; using the label information, from the original point cloud data Extract the instance point cloud data corresponding to the target to be detected; determine the target position point from the original point cloud data, and fuse the original point cloud data and the instance point cloud data based on the target position point processing to obtain the sample to be detected after fusion processing; the target to be detected is detected by combining the original point cloud data and the sample to be detected.
  • the present disclosure also discloses a computer-readable medium on which a computer program is stored.
  • the processor is caused to perform the following steps: acquiring original point cloud data containing a target to be detected,
  • the original point cloud data includes label information for the target to be detected; using the label information, the instance point cloud data corresponding to the target to be detected is extracted from the original point cloud data;
  • the data and the sample to be detected detect the target to be detected.
  • FIG. 1 schematically shows a system architecture diagram for implementing a target detection method according to an embodiment of the present disclosure.
  • FIG. 2 schematically shows a flowchart of a target detection method according to an embodiment of the present disclosure.
  • FIG. 3 schematically shows a flowchart of a target detection method according to another embodiment of the present disclosure.
  • FIG. 4 schematically shows a flowchart of a target detection method according to an embodiment of the present disclosure.
  • FIG. 5 schematically shows a flowchart of a target detection method according to another embodiment of the present disclosure.
  • FIG. 6 schematically shows a flowchart of a target detection method according to an embodiment of the present disclosure.
  • FIG. 7 schematically shows a block diagram of a target detection apparatus according to an embodiment of the present disclosure.
  • FIG. 8 shows a schematic structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present disclosure.
  • the system architecture 100 may include terminal devices 101 , 102 , and 103 , a network 104 and a server 105 .
  • the network 104 is a medium used to provide a communication link between the terminal devices 101 , 102 , 103 and the server 105 .
  • the network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
  • the user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send request instructions and the like.
  • Various communication client applications may be installed on the terminal devices 101 , 102 and 103 , such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, and the like.
  • the terminal devices 101, 102, 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
  • the server 105 may be a server that provides various services, such as a background management server that supports shopping websites browsed by the terminal devices 101 , 102 , and 103 (just an example).
  • the background management server can analyze and process the received product information query request and other data, and feed back the processing results (such as target push information, product information—just an example) to the terminal device.
  • the target detection method provided by the embodiments of the present disclosure is generally executed by the server 105 , and accordingly, the target detection apparatus is generally provided in the server 105 .
  • the target detection method of the present disclosure may also be executed by the terminal device 101, and accordingly, the target detection apparatus may be provided in the terminal device 101, which is not particularly limited in this embodiment.
  • the target detection method may include steps S210, S220, S230 and S240, wherein:
  • Step S210 Obtain original point cloud data including the target to be detected, wherein the original point cloud data includes label information for the target to be detected.
  • Step S220 Using the labeling information, extract instance point cloud data corresponding to the target to be detected from the original point cloud data.
  • Step S230 Determine a target position point from the original point cloud data, and perform fusion processing on the original point cloud data and the instance point cloud data based on the target position point to obtain a sample to be detected after fusion processing.
  • Step S240 Detect the target to be detected by combining the original point cloud data and the sample to be detected.
  • the processed samples to be detected can be obtained.
  • the number of samples is greatly increased, which can reduce the cost of labeling, shorten the labeling cycle, and improve the detection efficiency; at the same time, it can provide a way to control the data fusion process by using the target location points to avoid random fusion leading to lack of samples.
  • Authenticity can improve the effectiveness of the sample to be detected, thereby enhancing the accuracy of target detection; on the other hand, combining the original point cloud data and the sample to be detected for target detection can improve target detection on the basis of increasing the sample size. 's accuracy.
  • step S210 the original point cloud data including the object to be detected is obtained, wherein the original point cloud data includes label information for the object to be detected.
  • the target to be detected may include various objects in the scene, such as building targets, movable vehicles, people, etc.; or may also include other targets, such as roads, rivers, etc., this embodiment is not limited thereto.
  • the original point cloud data corresponding to the scene can be obtained by scanning the scene through a 3D laser scanner, or the original point cloud data of the scene can also be obtained through a camera.
  • the point cloud data may include a large number of point cloud samples, wherein the point cloud is a collection of points on the surface of an object.
  • the target to be detected in the original point cloud data can be marked to obtain the original point cloud data containing the label information of the target to be detected.
  • the target to be detected in the point cloud data can be marked by adding a frame. to obtain the original point cloud data containing the annotation information.
  • step S220 using the labeling information, the instance point cloud data corresponding to the target to be detected is extracted from the original point cloud data.
  • the instance point cloud data is the point cloud corresponding to the target to be detected in the original point cloud data
  • the instance point cloud data may also include the label information of the target to be detected, that is to say, the label can be synchronized when extracting the instance point cloud data. make a copy.
  • the instance point cloud data can be obtained from the original point cloud data by using the annotation information.
  • Each point cloud sample in the original point cloud data can include multiple objects to be detected, that is, including multiple annotation information.
  • the original point cloud data is the point cloud data of urban roads, and the object to be detected is a vehicle.
  • a point cloud sample can contain point clouds of multiple vehicles.
  • step S230 a target position point is determined from the original point cloud data, and based on the target position point, the original point cloud data and the instance point cloud data are fused to obtain the fusion processing to be detected sample.
  • the target location point may be a point belonging to the ground in the original point cloud data.
  • the method for determining the target position point from the original point cloud data may include the following steps S310 and S320, as shown in FIG. 3 .
  • step S310 ground detection is performed on the original point cloud data to determine ground point clouds in the original point cloud data.
  • the point with the smallest z value may be the point close to the ground
  • the n points with the smallest z value calculate its covariance matrix, and perform singularity on the calculated covariance matrix.
  • Value decomposition can obtain eigenvalues and eigenvectors, and the eigenvector corresponding to the smallest eigenvalue is the normal vector of the ground plane, so that the ground point cloud belonging to the ground plane can be obtained.
  • other algorithms can also perform ground detection on the original point cloud data to obtain the ground point cloud. For example, the filtering conditions are determined according to the characteristics of the ground point cloud, and the ground point cloud is selected from the original point cloud data.
  • a candidate position point is output from the ground point cloud, and if the candidate position point satisfies the collision detection condition, the candidate position point is determined as the target position point.
  • the candidate position point satisfies the collision. detection conditions. Specifically, first, a position point may be randomly output from the ground point cloud as the candidate position point; alternatively, a position point in the ground point cloud that is closest to the target point cloud to be detected may be searched as a candidate position point.
  • the instance point cloud data is fused to the candidate position points of the original point cloud, and the instance point cloud data does not collide with any object other than the ground in the original point cloud data, you can
  • the candidate position point is determined as the target position point; if the instance point cloud data is set at the candidate position point of the original point cloud data, the instance point cloud data collides with other objects in the original point cloud data, then give up
  • another candidate position point is reselected from the ground point cloud for judgment until a candidate position point that satisfies the collision detection conditions is found.
  • the instance point cloud data can be integrated into the target position of the original point cloud data to obtain the sample to be detected.
  • the number of samples can be increased proportionally, so that a large number of samples containing annotations can be obtained, which can effectively improve the output index of the target detection model and improve the detection accuracy.
  • the instance point cloud corresponding to the target to be detected is randomly placed on the ground of the original point cloud data, so that the integrated target to be detected conforms to the real situation, and the obtained sample to be detected is more realistic.
  • translation transformation is performed on the instance point cloud data according to the target position point, so as to fuse the instance point cloud data with the original point cloud data to obtain fused point cloud data.
  • the instance point cloud data may be located at the origin of the coordinates, and the instance point cloud data may be transformed to the target position through translation transformation, thereby updating the coordinates of the instance point cloud data and the coordinates corresponding to the annotation information in the instance point cloud data.
  • the instance point cloud data and the original point cloud data can be added to merge into the same frame of image.
  • the fused point cloud can be used as the sample to be detected, and the sample to be detected can be used as the training sample of the model to train the target detection model.
  • the instance point cloud data can be normalized, so as to transform its original coordinates to the position of the coordinate origin for translation transformation.
  • the original point cloud and the instance point cloud may also be fused in other ways, such as flipping, scaling, etc., which all belong to the protection scope of the present disclosure.
  • steps S410 and S420 may be further included, as shown in FIG. 4 .
  • step S410 the occluded point cloud data between the point cloud data to be fused and the original point cloud data is calculated.
  • the occluded point cloud data may include occluded point clouds.
  • the point cloud data may be converted into a spherical coordinate system.
  • the method for calculating the occluded point cloud data may include the following steps S510 and S520, as shown in FIG. 5 .
  • step S510 the fusion point cloud data is projected into a spherical coordinate system to obtain a corresponding spherical projection map. Firstly, coordinate transformation is performed on the fused point cloud data, from the rectangular coordinate system to the spherical coordinate system, and then the spherical projection map of the fused point cloud data can be obtained.
  • step S520 the fusion point cloud data is sampled to obtain a sample point, and the occluded point cloud data can be determined by judging whether the pixel point corresponding to the sampled sample point in the spherical projection image is empty. Since the pixel point in the spherical projection image is the point cloud closest to the origin of the spherical coordinate system, if a point in the fusion point cloud is at the same position as a pixel in the spherical projection image, that is to say, this point in the fusion point cloud If there is a corresponding pixel point at the point, the point may cause occlusion.
  • sampling from the fusion point cloud data obtaining one point at a time, and determining the pixel point in the spherical projection image corresponding to the point, and determining whether the pixel point is empty, if the pixel point is not empty, you can It is determined that there are other points with the same orientation as the point. If the pixel point is empty, it can be determined that the point is an occluded point.
  • the first distance between the sample point and the origin of the spherical coordinate system can be calculated, and the difference between the pixel corresponding to the sample point and the origin can be obtained from the spherical projection map.
  • the second distance it is determined whether the first distance is greater than the second distance, and if the first distance is greater than the second distance, the sample point can be determined as the occluded point cloud data. Then, the next sample point in the fusion point cloud data can be traversed, and so on, and all the occluded point cloud data can be determined by traversing all the sample points in the fusion point cloud.
  • the occluded point cloud data is removed from the fused point cloud data to obtain the to-be-detected sample.
  • the occluded point cloud data may include coordinate information of multiple occluded point clouds, and the occluded point cloud data can be used to remove the occluded point clouds from the fused point cloud data to obtain samples to be detected.
  • the occluded point cloud data can be removed through the spherical projection map in step S410. Specifically, in step S410, if it is determined that the first distance of a sample point is not greater than the second distance, the pixel point corresponding to the sample point can be processed. Update, update the distance between the pixel and the origin to the first distance in the spherical projection map.
  • the distance of each pixel in the spherical projection map is the distance of the point cloud closest to the origin, that is, the pixels in the spherical projection map are all points that are not blocked. Therefore, the spherical projection image is used for inverse transformation to turn it into a point cloud in a rectangular coordinate system, so that a point cloud that does not contain occluded point cloud data can be obtained as a sample to be detected.
  • Step S240 Detect the target to be detected by combining the original point cloud data and the sample to be detected.
  • the sample to be detected is the sample set obtained after data enhancement, and the original point cloud data is the point cloud sample set of the real scene obtained by scanning the scene, and the samples in the sample set all contain annotations, which can be used as the training set of the model for target detection.
  • the model is trained to obtain a trained target detection model, and then the target detection model can be used to detect the target to be detected.
  • a certain amount of original point cloud data can be obtained first through tools such as lidar, camera, etc.
  • the vehicles included in it are marked with a label frame; then the original point cloud data containing the label frame is extracted, and the point cloud corresponding to the vehicle and the label frame are extracted from it, that is, the instance point cloud data is obtained; the instance point cloud corresponding to the vehicle is used.
  • the target detection model can be used to detect movable vehicles in the scene.
  • the method may include steps S601 to S605, as shown in FIG. 6 .
  • step S601 a sample is read from the original point cloud data as the original sample, ground detection is performed on the original sample, and the ground point cloud in the original sample is determined; wherein, each sample in the original point cloud data can be Including the point cloud of the target to be detected and the labeling information of the point cloud; in step S602, randomly read an instance sample from the instance point cloud data; in step S603, randomly select a ground point to perform translation transformation on the instance sample to obtain Fusion point cloud; in step S604, perform collision detection on the fusion point cloud; judge whether the collision detection conditions are met, if so, execute step S605; in step S605, remove the occlusion information in the fusion point cloud; Perform coordinate transformation on the point cloud, filter out the spherical projection coincidence points, and obtain the sample to be detected; the spherical projection coincidence point is the point cloud with the same projected pixel position in the spherical projection image; if the collision detection conditions are not met, go to In step S603, a ground point is randomly selected again to
  • steps S601 to S605 in FIG. 6 are all described in the above-mentioned specific embodiments, and will not be repeated here.
  • a target detection apparatus is also provided, which is used for executing the above target detection method of the present disclosure.
  • the apparatus can be applied to a server or terminal equipment.
  • the target detection apparatus 700 may include: an original data acquisition module 710, an instance data acquisition module 720, a fusion processing module 730, and a target detection module 740, wherein:
  • the original data acquisition module 710 is configured to acquire original point cloud data including the object to be detected, wherein the original point cloud data includes label information for the object to be detected.
  • the instance data obtaining module 720 is configured to extract instance point cloud data corresponding to the target to be detected from the original point cloud data by using the labeling information.
  • the fusion processing module 730 is configured to determine a target position point from the original point cloud data, and perform fusion processing on the original point cloud data and the instance point cloud data based on the target position point, so as to obtain the fusion processing result. sample to be tested.
  • the target detection module 740 is configured to detect the target to be detected by combining the original point cloud data and the sample to be detected.
  • the fusion processing module 730 may be specifically configured to perform translation transformation on the instance point cloud data according to the target position point to obtain fusion point cloud data, and the fusion point cloud data as the sample to be detected.
  • the fusion processing module 730 may specifically include a ground detection unit and a position output unit, wherein:
  • a ground detection unit configured to perform ground detection on the original point cloud data to determine ground point clouds in the original point cloud data.
  • the position output unit is used for outputting a candidate position point from the ground point cloud, and if the candidate position point satisfies the collision detection condition, the candidate position point is determined as the target position point.
  • the position output unit is specifically configured to determine that if the instance point cloud data is located at the candidate position point, the ground point is not to be excluded from the original point cloud data If an object other than the cloud collides, the candidate position point satisfies the collision detection condition.
  • the target detection apparatus further includes an occlusion calculation module and an occlusion removal module, wherein:
  • An occlusion calculation module configured to calculate the occluded point cloud data between the point cloud data to be fused and the original point cloud data.
  • An occlusion removal module configured to remove the occluded point cloud data from the fusion point cloud data to obtain the to-be-detected sample.
  • the occlusion calculation module specifically includes a coordinate transformation unit and an occlusion determination unit, wherein:
  • the coordinate transformation unit is used for projecting the fusion point cloud data into a spherical coordinate system to obtain a corresponding spherical projection image.
  • An occlusion judgment unit configured to sample the fusion point cloud data, and if the pixel points corresponding to the sampled sample points in the spherical projection map are not empty, determine the occluded point cloud data according to the sample points .
  • the occlusion determination unit is specifically configured to obtain a first distance between the sample point and the origin of the spherical coordinate system, and a pixel corresponding to the sample point in the spherical projection map The second distance between the point and the origin of the spherical coordinate system; if the first distance is greater than the second distance, the sample point is determined as the occluded point cloud data.
  • each functional module of the target detection apparatus of the exemplary embodiment of the present disclosure corresponds to the steps of the above-mentioned exemplary embodiment of the target detection method, for details not disclosed in the embodiment of the apparatus of the present disclosure, please refer to the above-mentioned target detection method of the present disclosure example.
  • FIG. 8 shows a schematic structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present disclosure.
  • a computer system 800 includes a central processing unit (CPU) 801, which can be loaded into a random access memory (RAM) 803 according to a program stored in a read only memory (ROM) 802 or a program from a storage section 808 Instead, various appropriate actions and processes are performed.
  • RAM random access memory
  • ROM read only memory
  • various programs and data required for system operation are also stored.
  • the CPU 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804.
  • An input/output (I/O) interface 805 is also connected to bus 804 .
  • the following components are connected to the I/O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc. ; and a communication section 809 including a network interface card such as a LAN card, a modem, and the like. The communication section 809 performs communication processing via a network such as the Internet.
  • a drive 810 is also connected to the I/O interface 805 as needed.
  • a removable medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.
  • embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart.
  • the computer program may be downloaded and installed from the network via the communication portion 809, and/or installed from the removable medium 811.
  • CPU central processing unit
  • the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two.
  • the computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions.
  • the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
  • the units involved in the embodiments of the present disclosure may be implemented in software or hardware, and the described units may also be provided in a processor. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances.
  • the present application also provides a computer-readable medium.
  • the computer-readable medium may be included in the electronic device described in the above embodiments; it may also exist alone without being assembled into the electronic device. middle.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by an electronic device, causes the electronic device to implement the methods described in the following embodiments. For example, the electronic device can implement various steps as shown in FIG. 1 and FIG. 2 .
  • modules or units of the apparatus for action performance are mentioned in the above detailed description, this division is not mandatory. Indeed, according to embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one module or unit described above may be further divided into multiple modules or units to be embodied.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

一种目标检测方法、目标检测装置、计算机可读介质及电子设备，涉及图像处理技术领域。方法包括：获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息（S210）；利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据（S220）；从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本（S230）；结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测（S240）。该方法能够在一定程度上克服标注成本高的问题，进而提升目标检测的准确率。

Description

目标检测方法、装置、计算机可读介质及电子设备
相关申请的交叉引用
本申请要求于2020年07月06日提交的申请号为202010641217.6、名称为“目标检测方法、装置、计算机可读介质及电子设备”的中国专利申请的优先权，该中国专利申请的全部内容通过引用全部并入本文。
技术领域
本公开涉及图像处理技术领域，特别是涉及一种目标检测方法、目标检测装置、计算机可读介质及电子设备。
背景技术
目标检测是计算机视角与数字图像处理的重要方向，广泛地应用于机器人导航、智能视频监控、自动驾驶等诸多领域。
在实际工程中，点云数据存在标注困难、标注成本高等问题，导致可用的数据集有限。然而目标检测严重依赖于标注数据，标注数据的规模会直接影响模型最终的表现，因此采用数据增强的方式对给定的训练数据集进行扩充成为了提升模型检测准确率的有效方式。数据增强方式主要分为两种，一种是基于结构化数据生成增强数据；一种是基于数据库的点云数据增强方式。其中，第一种方式需要先去掉场景中的静态物体，然后再添加动态物体得到训练数据集，过程比较复杂难以实现在线数据增强；而第二种方式首先提取训练数据集中的标注框(包含框内点云)，随机从提取出的标注框中选取一部分标注框添加至当前帧点云数据中，得到增强后的训练数据集，这种方式虽然处理过程比较简单，但随机放置标注框导致生成的训练数据集缺乏真实性，对检测准确率的提升效果较差。
需要说明的是，在上述背景技术部分公开的信息仅用于加强对本公开的背景的理解，因此可以包括不构成对本领域普通技术人员已知的现有技术的信息。
发明内容
本公开的目的在于提供一种目标检测方法、目标检测装置、计算机可读介质及电子设备，能够在一定程度上克服目标标注难度大的问题，进而提升目标检测的准确率。
根据本公开的一个方面，提供一种目标检测方法，包括：获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息；利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据；从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本；结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
本公开还公开了一种目标检测装置，该装置包括：包括原始数据获取模块、实例数据获取模块、融合处理模块以及目标检测模块，其中：原始数据获取模块，用于获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息；实例数据获取模块，用于利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据；融合处理模块，用于从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本；目标检测模块，用于结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
本公开还公开了一种电子设备，包括：一个或多个处理器；和其上存储有可执行指令的存储器，当由所述一个或多个处理器执行所述可执行指令时，使得所述电子设备执行如下步骤：获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息；利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据；从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本；结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
本公开还公开了一种计算机可读介质，其上存储有计算机程序，当所述计算机程序被处理器执行时，使得所述处理器执行如下步骤：获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息；利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据；从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本；结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本公开。
附图说明
此处的附图被并入说明书中并构成本说明书的一部分，示出了符合本申请的实施例，并与说明书一起用于解释本公开的原理。显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。在附图中：
图1示意性示出了根据本公开的一个实施例的用于实现目标检测方法的系统架构图。
图2示意性示出了根据本公开的一个实施例的目标检测方法的流程图。
图3示意性示出了根据本公开的另一个实施例的目标检测方法的流程图。
图4示意性示出了根据本公开的一个实施例的目标检测方法的流程图。
图5示意性示出了根据本公开的另一个实施例的目标检测方法的流程图。
图6示意性示出了根据本公开的一个实施例的目标检测方法的流程图。
图7示意性示出了根据本公开的一个实施例的目标检测装置的框图。
图8示出了适于用来实现本公开实施例的电子设备的计算机系统的结构示意图。
具体实施方式
为使本公开的上述目的、特征和优点能够更加明显易懂，下面结合附图和具体实施方式对本公开作进一步详细的说明。
首先，本公开的示例性实施例中提供一种用于实现该目标检测方法的系统架构。参考图1所示，该系统架构100可以包括终端设备101、102、103，网络104和服务器105。网络104用以在终端设备101、102、103和服务器105之间提供通信链路的介质。网络104可以包括各种连接类型，例如有线、无线通信链路或者光纤电缆等等。
用户可以使用终端设备101、102、103通过网络104与服务器105交互，以接收或发送请求指令等。终端设备101、102、103上可以安装有各种通讯客户端应用，例如购物类应用、网页浏览器应用、搜索类应用、即时通信工具、邮箱客户端、社交平台软件等。
终端设备101、102、103可以是具有显示屏并且支持网页浏览的各种电子设备，包括但不限于智能手机、平板电脑、膝上型便携计算机和台式计算机等等。
服务器105可以是提供各种服务的服务器，例如对用户利用终端设备101、102、103所浏览的购物类网站提供支持的后台管理服务器(仅为示例)。后台管理服务器可以对接收到的产品信息查询请求等数据进行分析等处理，并将处理结果(例如目标推送信息、产品信息--仅为示例)反馈给终端设备。
需要说明的是，本公开实施方式所提供的目标检测方法一般由服务器105执行，相应地，目标检测装置一般设置于服务器105中。但是，本领域技术人员能够理解的是，本公开的目标检测方法也可以由终端设备101执行，相应地，目标检测装置可以设置于终端设备101中，本实施方式对此不做特殊限定。
基于上述系统架构，本示例实施方式提供了一种目标检测方法。参考图2所示，该目标检测方法可以包括步骤S210、步骤S220、步骤S230以及步骤S240，其中：
步骤S210：获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息。
步骤S220：利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据。
步骤S230：从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本。
步骤S240：结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
在本公开的一示例实施方式所提供的目标检测方法中，一方面，通过实例点云数据与原始点云数据进行融合处理，可以得到处理后的待检测样本，从而在原始点云数据的基础 上大大增加了样本数量，能够减小标注成本，缩短标注周期，进而提高检测效率；同时，能够提供一种另一方面，利用目标位置点能够对数据融合过程进行控制，避免随机融合导致样本缺乏真实性，可以提高待检测样本的有效性，进而增强目标检测的准确性；再一方面，结合原始点云数据与待检测样本进行目标检测，能够在增大样本规模的基础上，提高目标检测的准确率。
下面，对于本示例实施方式的上述步骤进行更加详细的说明。
在步骤S210中，获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息。
本实施方式中，待检测目标可以包括场景中的各种物体，例如建筑物目标、可移动车辆、人等；或者还可以包括其他目标，例如道路、河流等，本实施方式不限于此。通过三维激光扫描仪对场景进行扫描可以获得场景对应的原始点云数据，或者通过摄像头也可以获得场景的原始点云数据。点云数据中可以包括大量的点云样本，其中，点云为物体表面的点的集合。并且可以对该原始点云数据中的待检测目标进行标注，得到包含待检测目标的标注信息的原始点云数据，例如，可以通过添加标注框的方式将该点云数据中的待检测目标标出来，从而得到包含标注信息的原始点云数据。
在步骤S220中，利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据。
其中，实例点云数据为原始点云数据中待检测目标对应的点云，并且，实例点云数据中也可以包括待检测目标的标注信息，也就是说提取实例点云数据时可以将标注同步进行复制。本实施方式中，利用标注信息可以从原始点云数据中获得实例点云数据。原始点云数据中每一点云样本中可以包括多个待检测目标，即包括多个标注信息，例如原始点云数据为城市道路的点云数据，待检测目标为车辆，则每扫描一次得到的点云样本中可以包含多个车辆的点云。通过一一对标注信息进行读取，可以将原始点云数据中的待检测目标对应的实例点云提取出来，从而得到大量的实例点云数据，便于进行数据增强。
在步骤S230中，从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本。
示例性的，目标位置点可以为原始点云数据中属于地面的点。从原始点云数据中确定目标位置点的方法可以包括以下步骤S310和步骤S320，如图3所示。
在步骤S310中，对所述原始点云数据进行地面检测，以确定所述原始点云数据中的地面点云。示例性的，首先取靠近地面的n个点，z值最小的点可以是靠近地面的点，则取n个z值最小的点，计算其协方差矩阵，对计算得到的协方差矩阵进行奇异值分解可以得到特征值和特征向量，其中最小特征值对应的特征向量则为地面平面的法向量，从而可以得到属于地面平面中的地面点云。此外，通过其他算法也可以对原始点云数据进行地面检测，得到地面点云，例如，根据地面点云的特征确定筛选条件，从原始点云数据中筛选出地面点云等。
在步骤S320中，从所述地面点云中输出一候选位置点，若所述候选位置点满足碰撞检测条件，则将所述候选位置点确定为所述目标位置点。示例性的，若所述实例点云数据位于所述候选位置点处时，不与所述原始点云数据中除所述地面点云之外的物体发生碰撞，则所述候选位置点满足碰撞检测条件。具体的，首先从地面点云中可以随机输出一个位置点作为该候选位置点；或者，也可以查找该地面点云中离待检测目标点云最近的一个位置点作为候选位置点。输出候选位置点之后，如果将该实例点云数据融合至该原始点云的候选位置点处，该实例点云数据不与原始点云数据中除地面之外的任何物体发生碰撞，则可以将该候选位置点确定为目标位置点；如果该实例点云数据设置于该原始点云数据的该候选位置点处时，该实例点云数据与原始点云数据中的其他物体发生碰撞，则放弃该候选位置点，从地面点云中重新选择另一候选位置点进行判断，直到找出满足碰撞检测条件的候选位置点点。
得到目标位置点之后，可以将实例点云数据融入原始点云数据的目标位置处，从而得到待检测样本。通过原始点云与实例点云的融合可以成比例地增加样本数量，从而得到大量包含标注的样本，可以有效地提升目标检测模型的输出指标，提高检测的准确率。并且将待检测目标对应的实例点云随机放入原始点云数据的地面中，使得融入的待检测目标符合真实情况，得到的待检测样本更加具有真实性。
示例性实施方式中，对所述实例点云数据按照所述目标位置点进行平移变换，以将实例点云数据与原始点云数据进行融合，获得融合点云数据。示例性的，实例点云数据可以位于坐标原点的位置，通过平移变换将实例点云数据变换至目标位置点，从而更新实例点云数据的坐标，以及实例点云数据中标注信息对应的坐标。平移变换之后可以将实例点云数据与原始点云数据进行相加，从而融合为同一帧图像。融合之后的点云可作为待检测样本，待检测样本可以作为模型的训练样本，对目标检测模型进行训练。如果实例点云数据不在坐标原点则可以对实例点云数据进行归一化，从而将其原始坐标转化至坐标原点的位置，以进行平移变换。此外，在本公开的其他实施方式中，还可以同其他方式对原始点云与实例点云进行融合处理，例如翻转、缩放、等等，这些均属于本公开的保护范围。
示例性实施方式中，在将实例点云数据融合至原始点云数据中之后，还可以包括以下步骤S410和步骤S420，如图4所示。
在步骤S410中，计算所述待融合点云数据与所述原始点云数据之间的被遮挡点云数据。对场景进行扫描得到点云时，扫描点与物体之间的相对位置存在遮挡，相对靠近扫描点的物体可能会遮挡住远离扫描点的物体。本实施方式中，被遮挡点云数据可以包括被遮挡的点云。为了计算被遮挡点云数据可以将点云数据转换至球面坐标系中，具体的，计算被遮挡点云数据的方法可以包括以下步骤S510和步骤S520，如图5所示。
在步骤S510中，将所述融合点云数据投影至球面坐标系中，得到对应的球面投影图。首先对融合点云数据进行坐标变换，从直角坐标系变换为球面坐标系，然后可以得到融合点云数据的球面投影图。
在步骤S520中，对所述融合点云数据进行取样得到一个样本点，通过判断该取样的样本点在所述球面投影图中对应的像素点是否为空可以确定被遮挡点云数据。由于球面投影图中的像素点为与球面坐标系原点距离最近的点云，因此如果融合点云中的一点与球面投影图中的一像素点位置相同，也就是说，融合点云中的该点存在对应的像素点，则该点处有可能造成遮挡。具体的，从融合点云数据中采样，每次获取其中的一个点，并确定该点对应的球面投影图中的像素点，判断该像素点是否为空，如果该像素点不为空则可以确定存在与该点相同方位的其他点，如果该像素点为空，则可以确定该点是被遮挡住的点。
示例性的，如果取样的样本点对应的像素不为空，则可以计算该样本点与球面坐标系原点的第一距离，并从该球面投影图中获取该样本点对应的像素点与原点的第二距离，判断第一距离是否大于第二距离，如果第一距离大于第二距离，则可以将该样本点确定为被遮挡点云数据。然后可以遍历融合点云数据中的下一个样本点，依次类推，将融合点云中样本点全部遍历一遍可以确定出所有的被遮挡点云数据。
在步骤S420中，从所述融合点云数据中去除所述被遮挡点云数据，以获得所述待检测样本。被遮挡点云数据中可以包括多个被遮挡的点云的坐标信息，利用该被遮挡点云数据可以从融合点云数据中将被遮挡的点云去掉，得到待检测样本。通过步骤S410中的球面投影图可以去掉被遮挡点云数据，具体的，在步骤S410中，若确定一样本点的第一距离不大于第二距离，则可以将该样本点对应的像素点进行更新，在球面投影图中将该像素点与原点的距离更新为第一距离。对该球面投影图更新结束后，该球面投影图中的每一像素点的距离均为离原点最近的点云的距离，也就是说，球面投影图中的像素点均为不被遮挡的点云，从而利用该球面投影图进行反变换，将其变成为直角坐标系中的点云，从而可以得到不包含被遮挡点云数据的点云，作为待检测样本。
步骤S240：结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
待检测样本是数据增强后得到的样本集，而原始点云数据为对场景进行扫描获取的真实场景的点云样本集，并且样本集中的样本均包含标注，可以作为模型的训练集对目标检测模型进行训练，得到训练后的目标检测模型，进而通过该目标检测模型可以对待检测目标进行检测。
示例性的实施方式中，以自动驾驶场景为例，如果待检测目标为车辆，则可以首先通过激光雷达、照相机等工具获取一定数量的原始点云数据，对该原始点云数据进行标注，将其中包含的车辆用标注框标出来；然后对该包含标注框的原始点云数据进行提取，从中提取出车辆对应的点云以及标注框，即得到实例点云数据；利用车辆对应的实例点云数据与原始点云数据进行融合，可以得到一定数量的待检测样本，完成对样本的数据增强处理；进而利用增强处理后的待检测样本与原始点云数据均作为训练样本集，可以使得样本集数量大大增加，从而在采用深度学习算法训练目标检测模型时可以提高模型的准确率等输出指标。在采用自动驾驶技术的设备中，可以利用该目标检测模型对场景中的可移动车辆进行检测。
示例性实施方式中，该方法可以包括步骤S601至步骤S605，如图6所示。
在步骤S601中，从原始点云数据中读取一个样本作为原始样本，对该原始样本进行地面检测，确定该原始样本中的地面点云；其中，原始点云数据中的每个样本均可以包含待检测目标的点云以及点云的标注信息；在步骤S602中，从实例点云数据中随机读取一个实例样本；在步骤S603中，随机选取一地面点对实例样本进行平移变换，得到融合点云；在步骤S604中，对融合点云进行碰撞检测；判断是否满足碰撞检测条件，如果满足，则执行步骤S605；在步骤S605中，去除该融合点云中的遮挡信息；将该融合点云进行坐标变换，过滤掉球面投影重合点，得到待检测样本；其中球面投影重合点即为在球面投影图中的投影像素点位置相同的点云；如果不满足碰撞检测条件，则转至步骤S603中，再次随机选取一地面点重新对实例样本进行平移变换，直到得到满足碰撞检测条件的融合点云。最终得到的待检测样本中包含融合后的待检测目标以及对该待检测目标的标注信息，利用该待检测样本以及原始点云数据中的原始样本可以进行模型训练。
需要说明的是，图6中的步骤S601～步骤S605在上述具体实施例中均对其进行了说明，此处不再赘述。
进一步的，本示例实施方式中，还提供了一种目标检测装置，用于执行本公开上述的目标检测方法。该装置可以应用于一服务器或终端设备。
参考图7所示，该目标检测装置700可以包括：原始数据获取模块710、实例数据获取模块720、融合处理模块730以及目标检测模块740，其中：
原始数据获取模块710，用于获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息。
实例数据获取模块720，用于利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据。
融合处理模块730，用于从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本。
目标检测模块740，用于结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
在本公开的一种示例性实施方式中，融合处理模块730可以具体用于对所述实例点云数据按照所述目标位置点进行平移变换，以得到融合点云数据，将所述融合点云数据作为所述待检测样本。
在本公开的一种示例性实施方式中，融合处理模块730具体可以包括地面检测单元以及位置输出单元，其中：
地面检测单元，用于对所述原始点云数据进行地面检测，以确定所述原始点云数据中的地面点云。
位置输出单元，用于从所述地面点云中输出一候选位置点，若所述候选位置点满足碰 撞检测条件，则将所述候选位置点确定为所述目标位置点。
在本公开的一种示例性实施方式中，位置输出单元具体用于确定若在所述实例点云数据位于所述候选位置点处时，不与所述原始点云数据中除所述地面点云之外的物体发生碰撞，则所述候选位置点满足碰撞检测条件。
在本公开的一种示例性实施方式中，该目标检测装置还包括遮挡计算模块以及遮挡去除模块，其中：
遮挡计算模块，用于计算所述待融合点云数据与所述原始点云数据之间的被遮挡点云数据。
遮挡去除模块，用于从所述融合点云数据中去除所述被遮挡点云数据，以获得所述待检测样本。
在本公开的一种示例性实施方式中，所述遮挡计算模块具体包括坐标变换单元以及遮挡判断单元，其中：
坐标变换单元，用于将所述融合点云数据投影至球面坐标系中，得到对应的球面投影图。
遮挡判断单元，用于对所述融合点云数据进行取样，若取样的样本点在所述球面投影图中对应的像素点不为空，则根据所述样本点确定所述被遮挡点云数据。
在本公开的一种示例性实施方式中，遮挡判断单元具体用于获取所述样本点与所述球面坐标系原点的第一距离，以及所述样本点在所述球面投影图中对应的像素点与所述球面坐标系原点的第二距离；若所述第一距离大于第二距离，则将所述样本点确定为所述被遮挡点云数据。
由于本公开的示例实施例的目标检测装置的各个功能模块与上述目标检测方法的示例实施例的步骤对应，因此对于本公开装置实施例中未披露的细节，请参照本公开上述的目标检测方法的实施例。
图8示出了适于用来实现本公开实施例的电子设备的计算机系统的结构示意图。
需要说明的是，图8示出的电子设备的计算机系统800仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，计算机系统800包括中央处理单元(CPU)801，其可以根据存储在只读存储器(ROM)802中的程序或者从存储部分808加载到随机访问存储器(RAM)803中的程序而执行各种适当的动作和处理。在RAM 803中，还存储有系统操作所需的各种程序和数据。CPU 801、ROM 802以及RAM 803通过总线804彼此相连。输入/输出(I/O)接口805也连接至总线804。
以下部件连接至I/O接口805：包括键盘、鼠标等的输入部分806；包括诸如阴极射线管(CRT)、液晶显示器(LCD)等以及扬声器等的输出部分807；包括硬盘等的存储部分808；以及包括诸如LAN卡、调制解调器等的网络接口卡的通信部分809。通信部分809经由诸如因特网的网络执行通信处理。驱动器810也根据需要连接至I/O接口805。 可拆卸介质811，诸如磁盘、光盘、磁光盘、半导体存储器等等，根据需要安装在驱动器810上，以便于从其上读出的计算机程序根据需要被安装入存储部分808。
特别地，根据本公开的实施例，下文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信部分809从网络上被下载和安装，和/或从可拆卸介质811被安装。在该计算机程序被中央处理单元(CPU)801执行时，执行本申请的方法和装置中限定的各种功能。
需要说明的是，本公开所示的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：无线、电线、光缆、RF等等，或者上述的任意合适的组合。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，上述模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图或流程图中的每个方框、以及框图或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现，所描述的单元也可以设置在处理器中。其中，这些单元的名称在某种情况下并不构成对该单元本身的限定。
作为另一方面，本申请还提供了一种计算机可读介质，该计算机可读介质可以是上述实施例中描述的电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被一个该电子设备执行时，使得该电子设备实现如下述实施例中所述的方法。例如，所述的电子设备可以实现如图1和图2所示的各个步骤等。
应当注意，尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元，但是这种划分并非强制性的。实际上，根据本公开的实施方式，上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。反之，上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的，本公开并不局限于上面已经描述并在附图中示出的精确结构，并且可以在不脱离其范围进行各种修改和改变，本公开的真正范围和精神由下面的权利要求指出。

Claims (16)

  1. 一种目标检测方法，其特征在于，包括：
    获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息；
    利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据；
    从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本；
    结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
  2. 根据权利要求1所述的方法，其特征在于，所述基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，包括：
    对所述实例点云数据按照所述目标位置点进行平移变换，以得到融合点云数据，将所述融合点云数据作为所述待检测样本。
  3. 根据权利要求1所述的方法，其特征在于，所述从所述原始点云数据中确定目标位置点，包括：
    对所述原始点云数据进行地面检测，以确定所述原始点云数据中的地面点云；
    从所述地面点云中输出一候选位置点，若所述候选位置点满足碰撞检测条件，则将所述候选位置点确定为所述目标位置点。
  4. 根据权利要求3所述的方法，其特征在于，所述若所述候选位置点满足碰撞检测条件，包括：
    若所述实例点云数据位于所述候选位置点处时，不与所述原始点云数据中除所述地面点云之外的物体发生碰撞，则所述候选位置点满足碰撞检测条件。
  5. 根据权利要求2所述的方法，其特征在于，得到融合点云数据之后，还包括：
    计算所述待融合点云数据与所述原始点云数据之间的被遮挡点云数据；
    从所述融合点云数据中去除所述被遮挡点云数据，以获得所述待检测样本。
  6. 根据权利要求5所述的方法，其特征在于，所述计算所述待融合点云数据与所述原始点云数据之间的被遮挡点云数据，包括：
    将所述融合点云数据投影至球面坐标系中，得到对应的球面投影图；
    对所述融合点云数据进行取样，若取样的样本点在所述球面投影图中对应的像素点不为空，则根据所述样本点确定所述被遮挡点云数据。
  7. 根据权利要求6所述的方法，其特征在于，所述根据所述样本点确定所述被遮挡点云数据，包括：
    获取所述样本点与所述球面坐标系原点的第一距离，以及所述样本点在所述球面投影图中对应的像素点与所述球面坐标系原点的第二距离；
    若所述第一距离大于第二距离，则将所述样本点确定为所述被遮挡点云数据。
  8. 一种目标检测装置，其特征在于，包括：
    原始数据获取模块，用于获取包含待检测目标的原始点云数据，其中所述原始点云数据中包含对于所述待检测目标的标注信息；
    实例数据获取模块，用于利用所述标注信息，从所述原始点云数据中提取出所述待检测目标对应的实例点云数据；
    融合处理模块，用于从所述原始点云数据中确定目标位置点，基于所述目标位置点对所述原始点云数据与所述实例点云数据进行融合处理，以获得融合处理后的待检测样本；
    目标检测模块，用于结合所述原始点云数据与所述待检测样本对所述待检测目标进行检测。
  9. 根据权利要求8所述的装置，其特征在于，所述融合处理模块用于：
    对所述实例点云数据按照所述目标位置点进行平移变换，以得到融合点云数据，将所述融合点云数据作为所述待检测样本。
  10. 根据权利要求8所述的装置，其特征在于，所述融合处理模块包括：
    地面检测单元，用于对所述原始点云数据进行地面检测，以确定所述原始点云数据中的地面点云；
    位置输出单元，用于从所述地面点云中输出一随机位置，若所述随机位置满足碰撞检测条件，则将所述随机位置确定为所述目标位置点。
  11. 根据权利要求10所述的装置，其特征在于，所述位置输出单元还用于：
    若所述实例点云数据位于所述随机位置处时，不与所述原始点云数据中除所述地面点云之外的物体发生碰撞，则所述随机位置满足碰撞检测条件。
  12. 根据权利要求9所述的装置，其特征在于，所述装置还包括：
    遮挡计算模块，用于计算所述待融合点云数据与所述原始点云数据之间的遮挡信息；
    遮挡去除模块，用于从所述融合点云数据中去除所述遮挡信息，以获得所述待检测样本。
  13. 根据权利要求12所述的装置，其特征在于，所述遮挡计算模块包括：
    坐标变换单元，用于将所述融合点云数据投影至球面坐标系中，得到对应的球面投影图；
    遮挡判断单元，用于对所述融合点云数据进行取样，若取样的样本点在所述球面投影图中对应的像素点不为空，则根据所述样本点确定所述遮挡信息。
  14. 根据权利要求13所述的装置，其特征在于，所述遮挡判断单元还用于：
    获取所述样本点与所述球面坐标系原点的第一距离，以及所述样本点在所述球面投影图中对应的像素点与所述球面坐标系原点的第二距离；
    若所述第一距离大于第二距离，则将所述样本点确定为所述遮挡信息。
  15. 一种计算机可读介质，其上存储有计算机程序，其特征在于，所述计算机程序被 处理器执行时实现权利要求1-7中任意一项所述的方法。
  16. 一种电子设备，其特征在于，包括：
    处理器；
    存储器，用于存储所述处理器的可执行指令；
    其中，所述处理器被配置为经由执行所述可执行指令来执行权利要求1-7中任意一项所述的方法。
PCT/CN2021/085890 2020-07-06 2021-04-08 目标检测方法、装置、计算机可读介质及电子设备 WO2022007451A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US18/004,307 US20230222618A1 (en) 2020-07-06 2021-04-08 Object detection method and device
EP21838542.5A EP4177836A4 (en) 2020-07-06 2021-04-08 METHOD AND APPARATUS FOR TARGET DETECTION, COMPUTER READABLE MEDIUM AND ELECTRONIC DEVICE

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010641217.6 2020-07-06
CN202010641217.6A CN111784774B (zh) 2020-07-06 2020-07-06 目标检测方法、装置、计算机可读介质及电子设备

Publications (1)

Publication Number Publication Date
WO2022007451A1 true WO2022007451A1 (zh) 2022-01-13

Family

ID=72758011

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/085890 WO2022007451A1 (zh) 2020-07-06 2021-04-08 目标检测方法、装置、计算机可读介质及电子设备

Country Status (4)

Country Link
US (1) US20230222618A1 (zh)
EP (1) EP4177836A4 (zh)
CN (1) CN111784774B (zh)
WO (1) WO2022007451A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114565644A (zh) * 2022-03-02 2022-05-31 湖南中科助英智能科技研究院有限公司 三维运动目标检测方法、装置和设备
CN115422387A (zh) * 2022-11-04 2022-12-02 山东矩阵软件工程股份有限公司 基于多维点云融合数据的点云数据处理方法及系统

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111784774B (zh) * 2020-07-06 2024-03-05 北京京东乾石科技有限公司 目标检测方法、装置、计算机可读介质及电子设备
CN113792661B (zh) * 2021-09-15 2024-06-11 北京市商汤科技开发有限公司 图像检测方法、装置、电子设备以及存储介质
WO2024008081A1 (zh) * 2022-07-04 2024-01-11 梅卡曼德（北京）机器人科技有限公司 图像生成方法和模型训练方法
CN115082795A (zh) * 2022-07-04 2022-09-20 梅卡曼德（北京）机器人科技有限公司 虚拟图像的生成方法、装置、设备、介质及产品
CN117409161A (zh) * 2022-07-06 2024-01-16 戴尔产品有限公司 点云处理方法和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110264468A (zh) * 2019-08-14 2019-09-20 长沙智能驾驶研究院有限公司 点云数据标注、分割模型确定、目标检测方法及相关设备
CN110728210A (zh) * 2019-09-25 2020-01-24 上海交通大学 一种三维点云数据的半监督目标标注方法和系统
US20200174107A1 (en) * 2018-11-30 2020-06-04 Lyft, Inc. Lidar and camera rotational position calibration using multiple point cloud comparisons
CN111241969A (zh) * 2020-01-06 2020-06-05 北京三快在线科技有限公司 目标检测方法、装置及相应模型训练方法、装置
CN111784774A (zh) * 2020-07-06 2020-10-16 北京京东乾石科技有限公司 目标检测方法、装置、计算机可读介质及电子设备

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107025642B (zh) * 2016-01-27 2018-06-22 百度在线网络技术（北京）有限公司 基于点云数据的车辆轮廓检测方法和装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200174107A1 (en) * 2018-11-30 2020-06-04 Lyft, Inc. Lidar and camera rotational position calibration using multiple point cloud comparisons
CN110264468A (zh) * 2019-08-14 2019-09-20 长沙智能驾驶研究院有限公司 点云数据标注、分割模型确定、目标检测方法及相关设备
CN110728210A (zh) * 2019-09-25 2020-01-24 上海交通大学 一种三维点云数据的半监督目标标注方法和系统
CN111241969A (zh) * 2020-01-06 2020-06-05 北京三快在线科技有限公司 目标检测方法、装置及相应模型训练方法、装置
CN111784774A (zh) * 2020-07-06 2020-10-16 北京京东乾石科技有限公司 目标检测方法、装置、计算机可读介质及电子设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4177836A4 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114565644A (zh) * 2022-03-02 2022-05-31 湖南中科助英智能科技研究院有限公司 三维运动目标检测方法、装置和设备
CN114565644B (zh) * 2022-03-02 2023-07-21 湖南中科助英智能科技研究院有限公司 三维运动目标检测方法、装置和设备
CN115422387A (zh) * 2022-11-04 2022-12-02 山东矩阵软件工程股份有限公司 基于多维点云融合数据的点云数据处理方法及系统

Also Published As

Publication number Publication date
CN111784774A (zh) 2020-10-16
US20230222618A1 (en) 2023-07-13
CN111784774B (zh) 2024-03-05
EP4177836A4 (en) 2024-05-22
EP4177836A1 (en) 2023-05-10

Similar Documents

Publication Publication Date Title
WO2022007451A1 (zh) 目标检测方法、装置、计算机可读介质及电子设备
KR101856120B1 (ko) 이미지로부터 상가 발견
CN109582880B (zh) 兴趣点信息处理方法、装置、终端及存储介质
CN109308681B (zh) 图像处理方法和装置
US20190340746A1 (en) Stationary object detecting method, apparatus and electronic device
JP7273129B2 (ja) 車線検出方法、装置、電子機器、記憶媒体及び車両
WO2022227770A1 (zh) 目标对象检测模型的训练方法、目标对象检测方法和设备
CN111488873B (zh) 一种基于弱监督学习的字符级场景文字检测方法和装置
CN110349212B (zh) 即时定位与地图构建的优化方法及装置、介质和电子设备
US11255678B2 (en) Classifying entities in digital maps using discrete non-trace positioning data
CN111563398A (zh) 用于确定目标物的信息的方法和装置
CN111832579B (zh) 地图兴趣点数据处理方法、装置、电子设备以及可读介质
WO2023155581A1 (zh) 一种图像检测方法和装置
CN108597034B (zh) 用于生成信息的方法和装置
CN111881777A (zh) 一种视频处理方法和装置
JP2022110132A (ja) 陳列シーン認識方法、モデルトレーニング方法、装置、電子機器、記憶媒体およびコンピュータプログラム
CN114993328B (zh) 车辆定位评估方法、装置、设备和计算机可读介质
CN114429631B (zh) 三维对象检测方法、装置、设备以及存储介质
CN112819953B (zh) 三维重建方法、网络模型训练方法、装置及电子设备
CN111383337B (zh) 用于识别对象的方法和装置
CN111401182B (zh) 针对饲喂栏的图像检测方法和装置
CN114596442A (zh) 一种图像识别方法、装置、设备及存储介质
CN111797183A (zh) 挖掘信息点的道路属性的方法、装置及电子设备
CN111398961B (zh) 用于检测障碍物的方法和装置
EP4075383A1 (en) Portable device positioning data processing method and apparatus, device, and storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21838542

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021838542

Country of ref document: EP

Effective date: 20230206