CN113986504A - Point cloud data processing method and device, electronic equipment and storage medium - Google Patents
Point cloud data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113986504A CN113986504A CN202111272641.9A CN202111272641A CN113986504A CN 113986504 A CN113986504 A CN 113986504A CN 202111272641 A CN202111272641 A CN 202111272641A CN 113986504 A CN113986504 A CN 113986504A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- cloud data
- processing
- task
- grid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 180
- 238000000034 method Methods 0.000 claims abstract description 81
- 230000008569 process Effects 0.000 claims abstract description 31
- 238000001514 detection method Methods 0.000 claims description 120
- 238000000605 extraction Methods 0.000 claims description 26
- 239000012634 fragment Substances 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 12
- 238000010586 diagram Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000004458 analytical method Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 206010000117 Abnormal behaviour Diseases 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 2
- 230000004913 activation Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 101100498818 Arabidopsis thaliana DDR4 gene Proteins 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The disclosure provides a point cloud data processing method, a point cloud data processing device, an electronic device and a storage medium, wherein the method comprises the following steps: acquiring point cloud data to be processed; processing data of a plurality of task stages according to a preset subtask processing sequence on the point cloud data by using processing resources pre-allocated to each task stage to obtain a processing result; each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on point cloud data in advance. According to the method and the device, the processing task of the point cloud data is divided into a plurality of subtasks, different processing resources can be scheduled to perform data processing according to the characteristics of different subtasks, the processing efficiency of the point cloud data is remarkably improved, and the adaptability in various application fields is improved.
Description
Technical Field
The disclosure relates to the technical field of point cloud data processing, and in particular to a point cloud data processing method and device, electronic equipment and a storage medium.
Background
With the continuous development of the laser radar technology, the collection of point cloud data by applying the laser radar is widely applied to various fields, such as target perception, three-Dimensional (3D) target detection, 3D target reconstruction, automatic driving and the like.
Taking target perception as an example, because the amount of point cloud data generated by the laser radar is large and the point cloud data is updated in real time, if the collected point cloud data is directly processed, the processing effect is poor, and the real-time performance of target perception is further poor.
Disclosure of Invention
The embodiment of the disclosure at least provides a method and a device for processing point cloud data, electronic equipment and a storage medium.
In a first aspect, an embodiment of the present disclosure provides a method for processing point cloud data, including:
acquiring point cloud data to be processed;
processing the point cloud data in a plurality of task stages according to a preset subtask processing sequence by using processing resources pre-allocated to each task stage to obtain a processing result;
each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on the point cloud data in advance.
By adopting the point cloud data processing method, under the condition that the point cloud data to be processed is obtained, the processing resources pre-allocated to each task stage can be utilized to perform data processing of the task stage on the point cloud data at least once according to the preset subtask processing sequence, so that the processing result is obtained. According to the method and the device, the processing task of the point cloud data is divided into a plurality of subtasks, different processing resources can be scheduled to perform data processing according to the characteristics of different subtasks, the processing efficiency of the point cloud data is remarkably improved, and the adaptability in various application fields is improved.
In a possible implementation manner, in a case that the subtask includes a point cloud rasterization task corresponding to a first task stage, the processing resource pre-allocated to each task stage is utilized to perform data processing of a plurality of task stages on the point cloud data according to a preset subtask processing sequence, so as to obtain a processing result, where the processing result includes:
and rasterizing the point cloud data by using hardware resources or software resources pre-allocated to the first task stage to obtain point cloud data corresponding to each grid.
Here, the point cloud data may be rasterized so as to implement a uniform analysis of data of point cloud points within the same raster.
In a possible implementation manner, when the subtask further includes a grid feature extraction task corresponding to a second task stage, the performing, by using a processing resource pre-allocated to each task stage, data processing of a plurality of task stages on the point cloud data according to a preset subtask processing sequence to obtain a processing result includes:
and performing feature extraction on the obtained point cloud data corresponding to each grid by using hardware resources pre-allocated to the second task stage, and determining point cloud feature information of each grid.
Here, the grid feature extraction may be performed on the rasterized point cloud data, and the efficiency of the point cloud feature extraction is improved by performing the whole point cloud feature extraction on the point cloud data corresponding to each grid by taking the grid as a unit.
In one possible implementation, the pre-allocated hardware resources for the second task phase include a first hardware accelerator; after obtaining the point cloud data corresponding to each grid, the method further includes:
sequentially storing the point cloud data corresponding to each grid into a storage medium according to the position sequence among the point cloud points;
the step of performing feature extraction on the obtained point cloud data corresponding to each grid by using the hardware resources pre-allocated to the second task stage to determine point cloud feature information of each grid includes:
and sequentially reading the data of each point cloud point from the point cloud data corresponding to each grid stored in the storage medium by using the first hardware accelerator according to the position sequence among the grids, and determining the point cloud characteristic information of each grid based on the read data of each point cloud point.
Here, in the case of sequentially storing the data of the cloud points of each point in the storage medium by using the grid as a unit, before feature extraction is performed on each grid, the point cloud data corresponding to each grid may be sequentially read from the storage medium, and then the point cloud feature information corresponding to each grid is determined, so that the efficiency of extracting the point cloud features is ensured on the premise of improving the efficiency of data reading.
In a possible embodiment, the determining point cloud feature information of each grid based on the read data of the point cloud points includes:
determining point cloud characteristic information of each point cloud point in the grid based on the read data of each point cloud point;
determining coding feature information corresponding to point cloud data of the grid based on point cloud feature information of each point cloud point included in the grid, and taking the coding feature information as the point cloud feature information of the grid; and the feature dimension degree of the coded feature information is greater than the dimension degree of the point cloud feature information.
Here, to implement grid feature extraction, feature enhancement may be performed on each cloud point of a plurality of cloud points based on coordinate information of the cloud points, so as to obtain feature-enhanced coded feature information, which is equivalent to fusion of features of a plurality of cloud points in one grid.
In a possible implementation manner, in a case that the subtask further includes a target detection task corresponding to a third task stage, and the processing resource pre-allocated to the third task stage includes a second hardware accelerator, after determining point cloud feature information of each grid, the method further includes:
and inputting the determined point cloud characteristic information of each grid into a target detection network deployed on the second hardware accelerator for target detection to obtain a target detection result.
Here, it is considered that the point cloud feature information corresponding to the grid presents the spatial distribution condition of the relevant target object, so that the point cloud feature information extracted in the grid feature extraction process can be used as the input information of the target detection network to detect the target object information, and the target detection efficiency is improved on the premise that the point cloud feature extraction speed is high.
In a possible implementation, the target detection network includes a plurality of convolution layers with different convolution kernel sizes, and the inputting the determined point cloud feature information of each grid into the target detection network deployed on the second hardware accelerator for target detection includes:
inputting the point cloud characteristic information of the grid into a plurality of convolution layers included in the target detection network to obtain convolution characteristic information output by each convolution layer;
and determining the target detection result based on the convolution characteristic information output by the plurality of convolution layers.
In one possible embodiment, the inputting the determined point cloud feature information of each grid into the target detection network deployed on the second hardware accelerator for target detection includes:
and taking the point cloud characteristic information of each grid under the same characteristic dimension as a group of input information, and parallelly inputting different groups of input information corresponding to different characteristic dimensions into the target detection network for target detection.
In order to better extract target object information, before the point cloud characteristic information is input into a target detection network, the point cloud characteristic information of the grid can be decomposed in characteristic dimensions, the decomposed point cloud characteristic information can show the influence of different characteristic dimensions on a target object detection result, and the detection accuracy is improved.
In a possible embodiment, in a case that the point cloud data to be processed includes a plurality of sliced point cloud data, before rasterizing the point cloud data, the method further includes:
aiming at the point cloud data of each segment, extracting overlapped point cloud data from the point cloud data of one segment before the segment based on a preset overlapping range, and updating the point cloud data of the segment based on the overlapped point cloud data to obtain updated point cloud data of the segment;
the rasterizing processing of the point cloud data includes:
and executing rasterization processing on the updated point cloud data of the plurality of fragments in parallel.
Here, by performing special processing on the overlapped point cloud data between the slices, the processing effect on the point cloud data at the boundary can be optimized.
In a possible implementation manner, in a case that the subtask further includes a target tracking task corresponding to a fourth task stage, the method further includes:
under the condition that the target detection task is executed based on the point cloud data of the multiple fragments, integrating target detection results corresponding to the point cloud data of the multiple fragments to obtain an integrated detection result corresponding to the point cloud data to be processed;
and determining tracking track information of the target object based on the integrated detection result corresponding to the point cloud data to be processed.
Here, the target tracking task may be implemented based on single-frame or multi-frame point cloud data, and for point cloud data to be processed formed by multiple fragments, the target detection results corresponding to the fragments may be integrated first, and then the target tracking is performed, so as to ensure that the tracking of a complete target object is achieved.
In a possible implementation manner, before determining tracking track information of a target object based on an integrated detection result corresponding to the point cloud data to be processed, the method further includes:
aiming at the point cloud data of each fragment, extracting a sub-detection result corresponding to the overlapped point cloud data in the point cloud data of the fragment from a target detection result corresponding to the point cloud data of the fragment;
updating the integrated detection result based on the sub-detection result;
determining tracking track information of a target object based on an integrated detection result corresponding to the point cloud data to be processed, wherein the integrated detection result comprises the following steps:
and determining tracking track information of the target object based on the updated integrated detection result.
Here, for the overlapped point cloud data added in the previous link, before target tracking is performed, updating of a corresponding detection result is required to ensure accuracy of subsequent target tracking.
In a second aspect, an embodiment of the present disclosure further provides a device for processing point cloud data, including:
the acquisition module is used for acquiring point cloud data to be processed;
the processing module is used for processing the point cloud data in a plurality of task stages according to a preset subtask processing sequence by using processing resources pre-allocated to each task stage to obtain a processing result;
each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on the point cloud data in advance.
In a third aspect, an embodiment of the present disclosure further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine readable instructions when executed by the processor performing the steps of the method of processing point cloud data according to the first aspect and any of its various embodiments.
In a fourth aspect, the disclosed embodiments also provide a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps of the point cloud data processing method according to the first aspect and any one of the various embodiments.
For the description of the effects of the processing apparatus, the electronic device, and the computer-readable storage medium of the point cloud data, reference is made to the description of the processing method of the point cloud data, and details are not repeated here.
In order to make the aforementioned objects, features and advantages of the present disclosure more comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly described below, and the drawings herein incorporated in and forming a part of the specification illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the technical solutions of the present disclosure. It is appreciated that the following drawings depict only certain embodiments of the disclosure and are therefore not to be considered limiting of its scope, for those skilled in the art will be able to derive additional related drawings therefrom without the benefit of the inventive faculty.
Fig. 1 shows a flowchart of a method for processing point cloud data according to an embodiment of the present disclosure;
fig. 2 is a diagram illustrating a specific example of a rasterization process in a point cloud data processing method provided by an embodiment of the present disclosure;
fig. 3(a) shows a specific example of a point cloud data processing method before feature encoding according to an embodiment of the present disclosure;
fig. 3(b) shows a specific example diagram after feature encoding in the method for processing point cloud data provided by the embodiment of the disclosure;
fig. 4 is a schematic diagram illustrating an application of a method for processing point cloud data according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram illustrating a processing apparatus for point cloud data provided by an embodiment of the disclosure;
fig. 6 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only a part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present disclosure, presented in the figures, is not intended to limit the scope of the claimed disclosure, but is merely representative of selected embodiments of the disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the disclosure without making creative efforts, shall fall within the protection scope of the disclosure.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
The term "and/or" herein merely describes an associative relationship, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.
Research shows that the point cloud data generated by the laser radar is large in amount and is updated in real time, so that the point cloud data is difficult to process efficiently and deploy easily. On one hand, the related acceleration method has higher cost performance and energy efficiency and is easy to deploy so as to be convenient for popularization and landing; on the other hand, the prediction precision based on the point cloud data is maintained at a reasonable level while the real-time performance is improved and the deployment is easy; finally, in order to satisfy the universality and flexibility, the implementation method needs to be compatible with the subsequent model and the iterative update of the point cloud characteristics.
Based on the research, the present disclosure provides a scheme for processing point cloud data based on a pipeline manner, so as to improve data processing efficiency and improve adaptability in each application field.
To facilitate understanding of the present embodiment, first, a detailed description is given to a method for processing point cloud data disclosed in an embodiment of the present disclosure, and an execution subject of the method for processing point cloud data provided in the embodiment of the present disclosure is generally an electronic device with certain computing capability, where the electronic device includes, for example: a terminal device, which may be a User Equipment (UE), a mobile device, a cellular phone, a cordless phone, a Personal Digital Assistant (PDA), a handheld device, a computing device, a vehicle mounted device, a wearable device, or a server or other processing device. In some possible implementations, the method for processing the point cloud data may be implemented by a processor calling computer readable instructions stored in a memory.
Referring to fig. 1, which is a flowchart of a method for processing point cloud data provided in the embodiment of the present disclosure, the method includes steps S101 to S102, where:
s101: acquiring point cloud data to be processed;
s102: processing data of a plurality of task stages according to a preset subtask processing sequence on the point cloud data by using processing resources pre-allocated to each task stage to obtain a processing result; each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on point cloud data in advance.
In order to facilitate understanding of the method for processing point cloud data provided by the embodiments of the present disclosure, first, a brief description is provided to an application scenario of the method. The method for processing the point cloud data can be mainly applied to the fields of target detection and three-dimensional target reconstruction, and can also be applied to any other related fields needing point cloud data processing, and no specific limitation is made here. In view of the wide application of the target detection scenario, the following description will mostly take target detection as an example.
In consideration of the fact that the point cloud data generated by the laser radar in the related technology is large in amount and is updated in real time, if the acquired point cloud data is directly processed, the processing effect is poor. In order to solve the above problem, an embodiment of the present disclosure provides a method for processing point cloud data in a pipeline processing manner, so as to improve processing efficiency.
The point cloud data to be processed in the embodiment of the present disclosure may be acquired by using radar equipment. The radar device may be a rotary scanning laser radar, and may also be other radar devices, which are not particularly limited. Taking a rotary scanning laser radar as an example, the laser radar can acquire three-dimensional point cloud data related to the surrounding environment when the laser radar rotates and scans in the horizontal direction.
In the process of rotary scanning, the laser radar can adopt a multi-line scanning mode, namely, a plurality of laser tubes are used for transmitting in sequence, and the structure is that a plurality of laser tubes are longitudinally arranged, namely, in the process of rotary scanning in the horizontal direction, multilayer scanning in the vertical direction is carried out. A certain included angle is formed between every two laser tubes, the vertical emission view field can be 30-40 degrees, therefore, a data packet returned by the laser emitted by the laser tubes can be obtained when the laser radar equipment rotates for one scanning angle, the data packets obtained by the scanning angles are spliced to obtain a frame of point cloud data (corresponding to 360-degree scanning in one rotation), and after the laser radar scans for one circle, the frame of point cloud data can be obtained by scanning.
The embodiment of the disclosure may collect the multi-frame point cloud data obtained by scanning as point cloud data to be processed, for example, the multi-frame point cloud data collected for a preset application scene (e.g., a road traffic scene) within a preset time (e.g., 3 minutes) may be used as the point cloud data to be processed.
In practical applications, the point cloud data to be processed may also be a frame of point cloud data composed of point cloud data of multiple slices, for example, the point cloud data may be composed of slices corresponding to two half frames, may also be composed of slices corresponding to four quarter frames, and may also be composed of other slices. Besides, the point cloud data to be processed here may also be obtained in other slice-forming manners, and here may be determined based on different types of radar devices, which is not limited herein.
For the point cloud data to be processed, the data processing of multiple task stages can be performed on the point cloud data according to a preset subtask processing sequence, so as to obtain a processing result. Corresponding processing resources are pre-allocated to each task stage, and under the condition that the subtasks corresponding to the corresponding task stages are executed, the allocated processing resources can be used for carrying out data processing of the corresponding task stages.
It can be known that, in the process of processing point cloud data, the processing task can be divided into a plurality of subtasks, and data processing in a corresponding task phase is realized through decoupling operation between task phases corresponding to the subtasks, so that software and hardware segmentation and collaborative design can be conveniently performed by using heterogeneous computing resources, the problems of accelerated flexibility and universality are solved, and deployment is facilitated.
In the embodiment of the present disclosure, the processing tasks to be executed on the point cloud data may be determined based on different application scenarios, and the sub-tasks that the processing tasks corresponding to the different application scenarios may be divided into may be the same or different, and are not limited specifically herein.
In the process of dividing the processing tasks, the processing sequence among the subtasks can be determined, so that in the process of executing the corresponding subtasks, corresponding data processing can be performed based on the processing resources pre-allocated to the task stage where the subtask is located, so as to obtain the processing result of the corresponding subtask. Taking the target detection task as an example, the plurality of sub-tasks obtained by division may be a point cloud rasterization task, a grid feature extraction task, and a target detection task in sequence. Taking a three-dimensional point cloud reconstruction task as an example, the plurality of sub-tasks obtained by division can be a point cloud rasterization task, a grid feature extraction task and a three-dimensional reconstruction task in sequence.
In the embodiment of the present disclosure, the output of the previous task stage may be used as the input of the next task stage, and so on until the processing result corresponding to the whole processing task can be obtained. Here, by performing pipeline processing on each task stage by using heterogeneous computing resources, an acceleration bottleneck on a low-computation-effort chip can be solved, processing time consumption is reduced, a processing frame rate is improved, and balance between performance and real-time performance is achieved.
When the processing task includes a point cloud rasterization task corresponding to the first task stage, the point cloud data may be rasterized by using hardware resources or software resources pre-allocated to the first task stage, so as to obtain point cloud data corresponding to each grid.
The point cloud rasterizing process in the embodiment of the present disclosure is a process of rasterizing point cloud data. In view of the fact that in practical applications, the amount of raw point cloud data collected by radar devices is typically large, typically stored to off-chip memory, such as DDR3\ DDR4 in Double Data Rate Dynamic Random Access Memory (DDR SDRAM), from the aspect of processing efficiency, the point cloud rasterization in the method for processing point cloud data provided by the embodiment of the disclosure may rely on the storage of an on-chip memory, and therefore, during the rasterization process, a plurality of point cloud points corresponding to the point cloud data can be read according to a reading mechanism of a first storage medium (corresponding to an off-chip memory), then, the grid mapped by each cloud point can be determined for each cloud point, so that the cloud points can be written into the corresponding storage address of the second storage medium (corresponding to the on-chip memory) according to the grid information.
It should be noted that, in the process of writing each point cloud point into the storage medium, coordinate sorting may be performed on each point cloud point falling into one grid first, for example, the cloud points of each point are sorted in the order of horizontal coordinate values from small to large, so that sequential writing may be performed according to the sorting result.
The process of writing the data of the point cloud point relates to out-of-order access to the storage medium. The method mainly considers that after data of a point cloud point is read, a grid to which the point cloud point belongs needs to be determined, wherein before the point cloud point is written into a storage medium corresponding to the grid, disorder access can be performed on other point cloud points correspondingly stored in the grid to which the point cloud point belongs, and after sorting operation, the sorted point cloud points can be written into the storage medium sequentially, so that sequential reading of the data in the grid in a subsequent grid feature extraction stage is facilitated.
In the embodiment of the present disclosure, the initial feature of the cloud point may be coordinate information of the cloud point, the coordinate information may include a horizontal coordinate value, a vertical coordinate value, a height coordinate value, and a depth coordinate value, and the coordinate values may be represented by { X, Y, Z, I } correspondence (as shown in the example of the cloud point P1 shown in fig. 2), and it is known that the feature dimension of the initial feature of the cloud point is 4 dimensions. For example, in the case of determining a total of 16 cloud points in a grid, the sorted 16 cloud points { P1, P2 … …, P16} may be sequentially written into the storage address of a grid, as shown in fig. 2, and information such as the number of point cloud points written in other grids is not specifically illustrated.
When the point cloud rasterization task corresponding to the first task stage is executed, the feature extraction may be performed on the obtained point cloud data corresponding to each grid by using the hardware resources pre-allocated for the second task stage. In the embodiment of the present disclosure, the grid feature extraction process may be a process of extracting features of a point cloud data set corresponding to a grid by using a first hardware accelerator.
In the process of extracting the grid features, a unified analysis process may be performed on the point cloud data in the grid. In specific application, the point cloud data corresponding to each grid can be read firstly, then feature enhancement is carried out on each point cloud point in the point cloud data in the grid, the feature dimension of the point cloud point before feature enhancement is smaller than that of the point cloud point after feature enhancement, namely, the feature information of each point cloud point in the grid is highlighted, and finally, feature coding of the point cloud data corresponding to the grid can be realized based on the feature information of each point cloud point after enhancement, so that feature extraction of the grid is realized.
In practical applications, for point cloud data in a grid, convolution feature information corresponding to each point cloud point in the grid may be determined first. Here, taking the example that the point cloud data in one grid includes 16 point cloud points, and the dimension of the enhanced feature information of each point cloud point after enhancement is 9, the convolution operation may be performed on the 16 point cloud points, for example, the convolution operation may be performed by adopting 1 × 1 convolution, so as to obtain the convolution feature information corresponding to each point cloud point.
The 16 pieces of convolution feature information are subjected to summation operation, so that convolution feature information corresponding to the grid can be obtained, and at this time, the point cloud data can be processed by using an activation function, so that corresponding encoding feature information, for example, encoding feature information which can be encoded into 32 dimensions (corresponding to a second feature dimension), is obtained.
Fig. 3(a) -3 (b) are specific exemplary diagrams for feature encoding on one of 32 rasterized grids. As shown in fig. 3(a), three axial directions respectively indicate the number of grids, as shown in W ═ 32, the number of point cloud points in a grid, as shown in H ═ 16, and the dimension of the enhanced feature information of each point cloud point after enhancement (as shown in the enhanced feature information of the point cloud point P1), as shown in C ═ 9, so that after the encoding is implemented according to the above feature encoding method, the three axial directions respectively indicate W ═ 32, H ═ 1, and C ═ 32, that is, the fusion of the features of each point cloud point included in the point cloud data in one grid is implemented, corresponding to the encoded feature information, as shown in fig. 3 (b).
In specific application, an activation function of a Linear rectification function (ReLU) may be used for encoding, where the function may correspondingly encode point cloud data indicating that a convolution characteristic value is less than or equal to zero as 0, and correspondingly encode point cloud data indicating that a convolution characteristic value is greater than zero as 1, where the position of the obtained encoded characteristic information at 1 may correspond to a target object, and the position of 0 may correspond to a background, so as to facilitate subsequent target detection.
It should be noted that, based on the sequential write operation of the point cloud points during the task of point cloud rasterization, the written data of the point cloud points can be sequentially read before the raster feature extraction is performed.
Here, corresponding interfaces may be respectively set for the point cloud rasterization processing operation and the raster feature extraction operation described above, and here, an interface between the point cloud rasterization operation and the raster feature extraction operation may be set on the first hardware accelerator, and this interface may be used to implement sequential access to rasterized point cloud data stored in the storage medium.
The storage medium can be an on-chip memory or an off-chip memory, the off-chip memory can be adopted under the condition that the on-chip cache cannot meet the access requirement, the on-chip memory can be directly adopted on a chip with sufficient on-chip cache resources, and the off-chip memory does not need to be read and written, so that the time delay is reduced, and the access real-time performance is improved.
In the embodiment of the present disclosure, after the point cloud feature information corresponding to each grid is determined according to the above method, the point cloud feature information may be used as input information of a target detection network to implement detection of target object information.
Wherein the determined target object is different based on different application scenarios. For example, for automatic driving, the target object may be a person in front of an automatic driving automobile, and may also be a vehicle in front of the automatic driving automobile, which is not specifically limited by the embodiment of the present disclosure.
In the embodiment of the present disclosure, the above-mentioned related target detection network may be trained on the second hardware accelerator. The second hardware accelerator may be a custom hardware accelerator for adapting to training and application of the target detection network.
In the process of training the target detection network, the corresponding relationship between the point cloud data sample and the target object labeling result for the point cloud data sample can be trained, that is, the point cloud characteristics of the point cloud data sample can be used as the input data of the target detection network to be trained, and the target object labeling result for the point cloud data sample can be used as the supervision data of the output result of the target detection network to be trained to perform multiple rounds of training on the target detection network, so that the trained target detection network can be obtained. Therefore, under the condition that the grid feature extraction task corresponding to the second task stage is executed, the target detection task corresponding to the third task stage related to the target can be quickly realized by calling the second hardware accelerator with the trained target detection network deployed.
In order to facilitate detection of target objects with different sizes, convolution operations of different convolution parameters can be performed through the plurality of convolution layers included in the target detection network, so that a target detection result is determined through convolution characteristic information output by the plurality of convolution layers, and the accuracy of the target detection result is higher.
In the embodiment of the present disclosure, in order to adapt to the data receiving mode of the target detection network, feature disassembly may also be performed. Here, the point cloud feature information of each grid in the same feature dimension may be used as a set of input information, different sets of input information corresponding to different feature dimensions may be input in parallel to the trained target detection network, and the target object information may be determined from the target scene. Here, because different feature dimensions can represent some relevant information of the target object to a certain extent, through feature disassembly, the detection precision can be further improved on the premise of ensuring that the target detection network can be adapted.
For example, when the dimension of the point cloud feature information of the grid is determined to be 32 dimensions, the point cloud feature information of each grid under the uniform feature dimension may be used as a set of input information, different feature dimensions correspond to different sets of input information, and 32 sets of input information are totally corresponding to each set of input information, and the determination of the target object information may be achieved by inputting the 32 sets of input information into the target detection network.
It is considered that the point cloud data to be processed may be point cloud data including a plurality of slices, for example, point cloud data of two half frames, and point cloud data in other slice forms. In order to further improve the efficiency of processing the point cloud data, a splicing operation may be added between the slices, and then the rasterization process and other processes may be performed by executing the point cloud data of a plurality of slices in parallel.
The splicing operation can extract overlapped point cloud data from the point cloud data of one segment before the segment based on a preset overlapping range aiming at the point cloud data of each segment, and update the point cloud data of the segment based on the overlapped point cloud data to obtain updated point cloud data of the segment.
Taking field fragmentation as an example, as shown in fig. 4, a part of point cloud data of a first field can be filled in a neighboring boundary of a second field, and in the process of processing the point cloud data of the two fields, pipeline processing flows of four stages, namely a point cloud rasterization task, a grid feature extraction task and a target detection task, are multiplexed, so that the throughput rate of the whole processing task is improved, and the processing frame rate is improved.
Under the condition that the target detection task is executed based on the point cloud data of the multiple fragments, target detection results corresponding to the point cloud data of the multiple fragments can be integrated to obtain an integrated detection result corresponding to the point cloud data to be processed, and then tracking track information of the target object is determined based on the integrated detection result corresponding to the point cloud data to be processed.
Here, target detection results corresponding to the point cloud data of the plurality of slices may be integrated to determine an integrated detection result corresponding to the point cloud data to be processed. Still taking the field slicing as an example, when the point cloud data of the upper field detects one part of the target object and the point cloud data of the lower field detects another part of the same target object, the related information of the whole target object corresponding to the integrated detection result can be obtained. As shown in fig. 4, the processing task for one frame of point cloud data includes a target tracking task corresponding to the fourth task stage.
In view of the fact that the splicing operation of the point cloud data is performed during the rasterization process of each segment, in order to further ensure the accuracy of the tracking result, before determining the tracking track of the target object, for the point cloud data of each segment, a sub-detection result corresponding to the overlapped point cloud data in the point cloud data of the segment may be extracted from the target detection result corresponding to the point cloud data of the segment, corresponding to a post-processing stage after the target detection and classification in fig. 4.
In this way, in the case where the integrated detection result is updated based on the sub-detection result, more accurate tracking trajectory information can be determined. The tracking track information can well describe the behavior track of the target object, and effective management of the target object can be realized through behavior track analysis, for example, when the target object performs some abnormal behaviors, the target object can be timely found based on the tracking track, so that adverse effects of the abnormal behaviors on the current environment are avoided.
It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.
Based on the same inventive concept, the embodiment of the present disclosure further provides a device for processing point cloud data corresponding to the method for processing point cloud data, and since the principle of solving the problem of the device in the embodiment of the present disclosure is similar to the method for processing point cloud data in the embodiment of the present disclosure, the implementation of the device may refer to the implementation of the method, and repeated details are not repeated.
Referring to fig. 5, a schematic diagram of a processing apparatus for point cloud data according to an embodiment of the present disclosure is shown, the apparatus includes: an acquisition module 501 and a processing module 502; wherein,
an obtaining module 501, configured to obtain point cloud data to be processed;
the processing module 502 is configured to perform data processing of multiple task stages on the point cloud data according to a preset subtask processing sequence by using processing resources pre-allocated to each task stage, so as to obtain a processing result;
each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on point cloud data in advance.
By adopting the processing device for the point cloud data, under the condition of acquiring the point cloud data to be processed, the processing resources pre-allocated to each task stage can be utilized to perform data processing of the task stage on the point cloud data at least once according to the preset subtask processing sequence, so that the processing result is obtained. According to the method and the device, the processing task of the point cloud data is divided into a plurality of subtasks, different processing resources can be scheduled to perform data processing according to the characteristics of different subtasks, the processing efficiency of the point cloud data is remarkably improved, and the adaptability in various application fields is improved.
In a possible implementation manner, in a case that a subtask includes a point cloud rasterization task corresponding to a first task stage, the processing module 502 is configured to perform, according to the following steps, data processing of multiple task stages on point cloud data according to a preset subtask processing sequence by using processing resources pre-allocated to each task stage, so as to obtain a processing result:
and rasterizing the point cloud data by using hardware resources or software resources pre-allocated to the first task stage to obtain point cloud data corresponding to each grid.
In a possible implementation manner, in a case that the subtask further includes a grid feature extraction task corresponding to the second task stage, the processing module 502 is configured to perform data processing of multiple task stages on the point cloud data according to a preset subtask processing sequence by using a processing resource pre-allocated to each task stage according to the following steps, so as to obtain a processing result:
and performing feature extraction on the obtained point cloud data corresponding to each grid by using hardware resources pre-allocated to the second task stage, and determining point cloud feature information of each grid.
In one possible embodiment, the pre-allocated hardware resources for the second task phase include a first hardware accelerator; a processing module 502, configured to determine point cloud feature information of each grid according to the following steps:
after point cloud data corresponding to each grid are obtained, sequentially storing the point cloud data corresponding to each grid into a storage medium according to the position sequence among the point cloud points;
and sequentially reading the data of each point cloud point from the point cloud data corresponding to each grid stored in the storage medium by using the first hardware accelerator according to the position sequence among the grids, and determining the point cloud characteristic information of each grid based on the read data of each point cloud point.
In a possible implementation, the processing module 502 is configured to determine point cloud feature information of each grid based on the read data of the point cloud points according to the following steps:
determining point cloud characteristic information of each point cloud point in the grid based on the read data of each point cloud point;
determining coding characteristic information corresponding to point cloud data of the grid based on point cloud characteristic information of each point cloud point included by the grid, and taking the coding characteristic information as the point cloud characteristic information of the grid; the feature dimension degree of the coded feature information is larger than the dimension degree of the point cloud feature information.
In a possible implementation manner, in a case that the subtask further includes a target detection task corresponding to the third task phase, and the processing resource pre-allocated to the third task phase includes a second hardware accelerator, the processing module 502 is further configured to:
after the point cloud characteristic information of each grid is determined, the determined point cloud characteristic information of each grid is input into a target detection network deployed on the second hardware accelerator for target detection, and a target detection result is obtained.
In a possible implementation manner, the target detection network includes a plurality of convolution layers with different convolution kernel sizes, and the processing module 502 is configured to input the determined point cloud feature information of each grid into the target detection network deployed on the second hardware accelerator for target detection according to the following steps:
inputting the point cloud characteristic information of the grid into a plurality of convolution layers included in the target detection network to obtain convolution characteristic information output by each convolution layer;
and determining a target detection result based on the convolution characteristic information output by the plurality of convolution layers.
In a possible implementation, the processing module 502 is configured to input the determined point cloud feature information of each grid into a target detection network deployed on the second hardware accelerator for target detection according to the following steps:
and taking the point cloud characteristic information of each grid under the same characteristic dimension as a group of input information, and parallelly inputting different groups of input information corresponding to different characteristic dimensions into a target detection network for target detection.
In a possible implementation manner, in the case that the point cloud data to be processed includes a plurality of sliced point cloud data, the processing module 502 is configured to perform rasterization processing on the point cloud data according to the following steps:
before rasterizing the point cloud data, aiming at the point cloud data of each segment, extracting overlapped point cloud data from the point cloud data of one segment before the segment based on a preset overlapping range, and updating the point cloud data of the segment based on the overlapped point cloud data to obtain updated point cloud data of the segment;
and executing rasterization processing on the updated point cloud data of the plurality of fragments in parallel.
In a possible implementation manner, in the case that the subtask further includes a target tracking task corresponding to the fourth task stage, the processing module 502 is further configured to:
under the condition that a target detection task is executed based on the point cloud data of the multiple fragments, integrating target detection results corresponding to the point cloud data of the multiple fragments to obtain an integrated detection result corresponding to the point cloud data to be processed;
and determining tracking track information of the target object based on an integrated detection result corresponding to the point cloud data to be processed.
In a possible implementation, the processing module 502 is configured to determine tracking trajectory information of the target object according to the following steps:
before determining tracking track information of a target object based on an integrated detection result corresponding to point cloud data to be processed, extracting a sub-detection result corresponding to overlapped point cloud data in the segmented point cloud data from a target detection result corresponding to the segmented point cloud data aiming at the point cloud data of each segment;
updating the integrated detection result based on the sub-detection result;
and determining tracking track information of the target object based on the updated integrated detection result.
The description of the processing flow of each module in the device and the interaction flow between the modules may refer to the related description in the above method embodiments, and will not be described in detail here.
An embodiment of the present disclosure further provides an electronic device, as shown in fig. 6, which is a schematic structural diagram of the electronic device provided in the embodiment of the present disclosure, and the electronic device includes: a processor 601, a memory 602, and a bus 603. The memory 602 stores machine-readable instructions executable by the processor 601 (for example, execution instructions corresponding to the obtaining module 501 and the processing module 502 in the apparatus in fig. 5, and the like), when the electronic device is operated, the processor 601 and the memory 602 communicate via the bus 603, and the machine-readable instructions, when executed by the processor 601, perform the following processes:
acquiring point cloud data to be processed;
processing data of a plurality of task stages according to a preset subtask processing sequence on the point cloud data by using processing resources pre-allocated to each task stage to obtain a processing result;
each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on point cloud data in advance.
The embodiments of the present disclosure also provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the method for processing point cloud data described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.
The embodiments of the present disclosure also provide a computer program product, where the computer program product carries a program code, and instructions included in the program code may be used to execute the steps of the point cloud data processing method described in the foregoing method embodiments, which may be referred to specifically for the foregoing method embodiments, and are not described herein again.
The computer program product may be implemented by hardware, software or a combination thereof. In an alternative embodiment, the computer program product is embodied in a computer storage medium, and in another alternative embodiment, the computer program product is embodied in a Software product, such as a Software Development Kit (SDK), or the like.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the system and the apparatus described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present disclosure. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: the above-mentioned embodiments are merely specific embodiments of the present disclosure, which are used for illustrating the technical solutions of the present disclosure and not for limiting the same, and the scope of the present disclosure is not limited thereto, and although the present disclosure is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive of the technical solutions described in the foregoing embodiments or equivalent technical features thereof within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present disclosure, and should be construed as being included therein. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (14)
1. A method for processing point cloud data is characterized by comprising the following steps:
acquiring point cloud data to be processed;
processing the point cloud data in a plurality of task stages according to a preset subtask processing sequence by using processing resources pre-allocated to each task stage to obtain a processing result;
each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on the point cloud data in advance.
2. The processing method according to claim 1, wherein when the subtask includes a point cloud rasterization task corresponding to a first task stage, the processing resource pre-allocated to each task stage is utilized to perform data processing of a plurality of task stages on the point cloud data according to a preset subtask processing sequence, so as to obtain a processing result, and the processing method includes:
and rasterizing the point cloud data by using hardware resources or software resources pre-allocated to the first task stage to obtain point cloud data corresponding to each grid.
3. The processing method according to claim 2, wherein when the subtask further includes a raster feature extraction task corresponding to a second task stage, the processing method performs data processing of a plurality of task stages on the point cloud data according to a preset subtask processing sequence by using processing resources pre-allocated to each task stage to obtain a processing result, and includes:
and performing feature extraction on the obtained point cloud data corresponding to each grid by using hardware resources pre-allocated to the second task stage, and determining point cloud feature information of each grid.
4. The processing method according to claim 3, wherein the pre-allocated hardware resources for the second task phase comprise a first hardware accelerator; after obtaining the point cloud data corresponding to each grid, the method further includes:
sequentially storing the point cloud data corresponding to each grid into a storage medium according to the position sequence among the point cloud points;
the step of performing feature extraction on the obtained point cloud data corresponding to each grid by using the hardware resources pre-allocated to the second task stage to determine point cloud feature information of each grid includes:
and sequentially reading the data of each point cloud point from the point cloud data corresponding to each grid stored in the storage medium by using the first hardware accelerator according to the position sequence among the grids, and determining the point cloud characteristic information of each grid based on the read data of each point cloud point.
5. The processing method according to claim 4, wherein the determining point cloud feature information of each grid based on the read data of the point cloud points comprises:
determining point cloud characteristic information of each point cloud point in the grid based on the read data of each point cloud point;
determining coding feature information corresponding to point cloud data of the grid based on point cloud feature information of each point cloud point included in the grid, and taking the coding feature information as the point cloud feature information of the grid; and the feature dimension degree of the coded feature information is greater than the dimension degree of the point cloud feature information.
6. The processing method according to any one of claims 3 to 5, wherein in a case where the subtask further includes a target detection task corresponding to a third task stage and the processing resource pre-allocated to the third task stage includes a second hardware accelerator, after determining the point cloud feature information of each grid, the method further includes:
and inputting the determined point cloud characteristic information of each grid into a target detection network deployed on the second hardware accelerator for target detection to obtain a target detection result.
7. The processing method of claim 6, wherein the target detection network comprises a plurality of convolution layers with different convolution kernel sizes, and the inputting the determined point cloud feature information of each grid into the target detection network deployed on the second hardware accelerator for target detection comprises:
inputting the point cloud characteristic information of the grid into a plurality of convolution layers included in the target detection network to obtain convolution characteristic information output by each convolution layer;
and determining the target detection result based on the convolution characteristic information output by the plurality of convolution layers.
8. The processing method of claim 6, wherein the inputting the determined point cloud feature information of each grid into an object detection network deployed on the second hardware accelerator for object detection comprises:
and taking the point cloud characteristic information of each grid under the same characteristic dimension as a group of input information, and parallelly inputting different groups of input information corresponding to different characteristic dimensions into the target detection network for target detection.
9. The processing method according to any one of claims 6 to 8, wherein in a case where the point cloud data to be processed includes a plurality of sliced point cloud data, before rasterizing the point cloud data, the method further includes:
aiming at the point cloud data of each segment, extracting overlapped point cloud data from the point cloud data of one segment before the segment based on a preset overlapping range, and updating the point cloud data of the segment based on the overlapped point cloud data to obtain updated point cloud data of the segment;
the rasterizing processing of the point cloud data includes:
and executing rasterization processing on the updated point cloud data of the plurality of fragments in parallel.
10. The processing method according to claim 9, wherein in a case where the subtask further includes a target tracking task corresponding to a fourth task stage, the method further includes:
under the condition that the target detection task is executed based on the point cloud data of the multiple fragments, integrating target detection results corresponding to the point cloud data of the multiple fragments to obtain an integrated detection result corresponding to the point cloud data to be processed;
and determining tracking track information of the target object based on the integrated detection result corresponding to the point cloud data to be processed.
11. The processing method according to claim 10, wherein before determining tracking trajectory information of a target object based on the integrated detection result corresponding to the point cloud data to be processed, the method further comprises:
aiming at the point cloud data of each fragment, extracting a sub-detection result corresponding to the overlapped point cloud data in the point cloud data of the fragment from a target detection result corresponding to the point cloud data of the fragment;
updating the integrated detection result based on the sub-detection result;
determining tracking track information of a target object based on an integrated detection result corresponding to the point cloud data to be processed, wherein the integrated detection result comprises the following steps:
and determining tracking track information of the target object based on the updated integrated detection result.
12. An apparatus for processing point cloud data, comprising:
the acquisition module is used for acquiring point cloud data to be processed;
the processing module is used for processing the point cloud data in a plurality of task stages according to a preset subtask processing sequence by using processing resources pre-allocated to each task stage to obtain a processing result;
each task stage is an execution process of a subtask corresponding to the task stage, and the subtask corresponding to the task stage is one of a plurality of subtasks obtained by dividing a processing task executed on the point cloud data in advance.
13. An electronic device, comprising: processor, memory and bus, the memory storing machine readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is running, the machine readable instructions when executed by the processor performing the steps of the method of processing point cloud data according to any of claims 1 to 11.
14. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, performs the steps of the method for processing point cloud data according to any one of claims 1 to 11.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111272641.9A CN113986504A (en) | 2021-10-29 | 2021-10-29 | Point cloud data processing method and device, electronic equipment and storage medium |
| PCT/CN2022/103235 WO2023071273A1 (en) | 2021-10-29 | 2022-07-01 | Point cloud data processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111272641.9A CN113986504A (en) | 2021-10-29 | 2021-10-29 | Point cloud data processing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113986504A true CN113986504A (en) | 2022-01-28 |
Family
ID=79744448
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111272641.9A Pending CN113986504A (en) | 2021-10-29 | 2021-10-29 | Point cloud data processing method and device, electronic equipment and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN113986504A (en) |
| WO (1) | WO2023071273A1 (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115242778A (en) * | 2022-07-21 | 2022-10-25 | 中电金信软件有限公司 | File transmission method and device, electronic equipment and readable storage medium |
| WO2023071273A1 (en) * | 2021-10-29 | 2023-05-04 | 上海商汤智能科技有限公司 | Point cloud data processing |
| CN117215774A (en) * | 2023-08-21 | 2023-12-12 | 上海瞰融信息技术发展有限公司 | Engine system and method for automatically identifying and adapting live-action three-dimensional operation task |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116659523B (en) * | 2023-05-17 | 2024-07-23 | 深圳市保臻社区服务科技有限公司 | Automatic position locating method and device based on community entering vehicles |
| CN116452403B (en) * | 2023-06-16 | 2023-09-01 | 瀚博半导体(上海)有限公司 | Point cloud data processing method and device, computer equipment and storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109885388A (en) * | 2019-01-31 | 2019-06-14 | 上海赜睿信息科技有限公司 | A kind of data processing method and device suitable for heterogeneous system |
| US20200311569A1 (en) * | 2019-03-26 | 2020-10-01 | Vathys, Inc. | Low latency and high throughput inference |
| CN111079652B (en) * | 2019-12-18 | 2022-05-13 | 北京航空航天大学 | 3D target detection method based on point cloud data simple coding |
| CN111652907B (en) * | 2019-12-25 | 2021-08-27 | 珠海大横琴科技发展有限公司 | Multi-target tracking method and device based on data association and electronic equipment |
| CN112950622B (en) * | 2021-03-29 | 2023-04-18 | 上海商汤临港智能科技有限公司 | Target detection method and device, computer equipment and storage medium |
| CN113111787A (en) * | 2021-04-15 | 2021-07-13 | 北京沃东天骏信息技术有限公司 | Target detection method, device, equipment and storage medium |
| CN113986504A (en) * | 2021-10-29 | 2022-01-28 | 上海商汤临港智能科技有限公司 | Point cloud data processing method and device, electronic equipment and storage medium |
-
2021
- 2021-10-29 CN CN202111272641.9A patent/CN113986504A/en active Pending
-
2022
- 2022-07-01 WO PCT/CN2022/103235 patent/WO2023071273A1/en unknown
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2023071273A1 (en) * | 2021-10-29 | 2023-05-04 | 上海商汤智能科技有限公司 | Point cloud data processing |
| CN115242778A (en) * | 2022-07-21 | 2022-10-25 | 中电金信软件有限公司 | File transmission method and device, electronic equipment and readable storage medium |
| CN117215774A (en) * | 2023-08-21 | 2023-12-12 | 上海瞰融信息技术发展有限公司 | Engine system and method for automatically identifying and adapting live-action three-dimensional operation task |
| CN117215774B (en) * | 2023-08-21 | 2024-05-28 | 上海瞰融信息技术发展有限公司 | Engine system and method for automatically identifying and adapting live-action three-dimensional operation task |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2023071273A1 (en) | 2023-05-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113986504A (en) | Point cloud data processing method and device, electronic equipment and storage medium | |
| CN109829399B (en) | Vehicle-mounted road scene point cloud automatic classification method based on deep learning | |
| CN110572362B (en) | Network attack detection method and device for multiple types of unbalanced abnormal traffic | |
| CN113971221A (en) | Point cloud data processing method and device, electronic equipment and storage medium | |
| CN110990516B (en) | Map data processing method, device and server | |
| CN112669463B (en) | Method for reconstructing curved surface of three-dimensional point cloud, computer device and computer-readable storage medium | |
| US10217017B2 (en) | Systems and methods for containerizing multilayer image segmentation | |
| WO2018136963A1 (en) | Distributed and parallelized visualization framework | |
| CN114140683A (en) | Aerial image target detection method, equipment and medium | |
| CN114066718A (en) | Image style migration method and device, storage medium and terminal | |
| JP7368623B2 (en) | Point cloud processing method, computer system, program and computer readable storage medium | |
| US20230419659A1 (en) | Method and system for processing point-cloud data | |
| CN111597845A (en) | Two-dimensional code detection method, device and equipment and readable storage medium | |
| CN114419570A (en) | Point cloud data identification method and device, electronic equipment and storage medium | |
| CN112801109A (en) | Remote sensing image segmentation method and system based on multi-scale feature fusion | |
| CN109377552A (en) | Image occlusion test method, apparatus calculates equipment and storage medium | |
| CN110969641A (en) | Image processing method and device | |
| CN113902793A (en) | End-to-end building height prediction method and system based on single vision remote sensing image and electronic equipment | |
| CN117496477A (en) | Point cloud target detection method and device | |
| CN111325821B (en) | Grid model processing method and device, equipment and storage medium | |
| Li et al. | Pillar‐based 3D object detection from point cloud with multiattention mechanism | |
| CN111062473B (en) | Data calculation method, image processing method and device in neural network model | |
| CN113591827A (en) | Text image processing method and device, electronic equipment and readable storage medium | |
| CN105022746A (en) | Character library generation method, server and system | |
| Guo et al. | Efficient triangulation of Poisson-disk sampled point sets |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 40061874 Country of ref document: HK |