CN115756841B - Efficient data generation system and method based on parallel processing - Google Patents
Efficient data generation system and method based on parallel processing Download PDFInfo
- Publication number
- CN115756841B CN115756841B CN202211423227.8A CN202211423227A CN115756841B CN 115756841 B CN115756841 B CN 115756841B CN 202211423227 A CN202211423227 A CN 202211423227A CN 115756841 B CN115756841 B CN 115756841B
- Authority
- CN
- China
- Prior art keywords
- laser
- parallel processing
- blocks
- point cloud
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Numerical Control (AREA)
- Optical Radar Systems And Details Thereof (AREA)
- Laser Beam Processing (AREA)
Abstract
The efficient data generation system based on parallel processing comprises a block module and a plurality of parallel processing modules, wherein the parallel processing modules are respectively connected with the block module; the blocking module distributes any one of the plurality of laser origin cloud data sub-blocks and the position and posture data sub-block corresponding to any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules; any one of the plurality of parallel processing modules is used for processing the received sub-blocks to obtain laser point cloud result data. The original laser point cloud data and the position posture data are respectively segmented to form segmented data processing tasks, the data processing tasks are automatically distributed according to the monitored cluster resource state to form a distributed processing environment, the distributed processing environment is processed in parallel in the GPU, rapid conversion of the segmented data is achieved, and finally the converted segmented point cloud data are fused to generate complete laser point cloud result data. And the clusters and the GPU are used for processing the laser point cloud data in parallel, so that the processing efficiency of the laser point cloud data is greatly improved.
Description
Technical Field
The invention relates to the field of laser point cloud processing, in particular to a system and a method for generating high-efficiency data based on parallel processing.
Background
Along with the development of mapping informatization, mapping information goes deep into the aspects of life of people: the conventional map which is simply displayed by taking a symbol as the center can not meet the existing use scene requirements from public services such as land planning, city construction, emergency dispatch, intelligent transportation and the like to individual live-action navigation, travel route planning and the like.
In order to quickly, accurately and comprehensively collect the space geographic information, a mobile measurement technology is developed. The mobile measurement relies on the laser radar to collect laser origin cloud data and inertial navigation sensors to collect laser radar position and posture data, but the laser origin cloud data cannot be directly used for browsing, analyzing and applying, and the laser origin cloud data and the position and posture data need to be processed to obtain laser origin cloud achievement data.
In the prior art, laser original point cloud data and position posture data are processed in a single mode to obtain laser point cloud result data, and the mode has high requirement on single configuration and low efficiency in processing mass data, so that business application of the laser point cloud result data is severely restricted.
Content of the application
Purpose of (one) application
In view of the above, the invention provides a system and a method for generating high-efficiency data based on parallel processing, which are used for solving the problems that the requirement on single machine configuration is high and the efficiency is low when processing massive data when processing laser original point cloud data and position posture data to obtain laser point cloud result data in the prior art.
(II) technical scheme
The application discloses a high-efficiency data generation system based on parallel processing, which comprises a block module and a plurality of parallel processing modules, wherein the plurality of parallel processing modules are respectively connected with the block module;
the blocking module is used for blocking the laser original point cloud data to obtain a plurality of laser original point cloud data sub-blocks, and is also used for blocking the position and posture data to obtain a plurality of position and posture data sub-blocks; the blocking module distributes any one of the plurality of laser origin cloud data sub-blocks and position and posture data sub-blocks corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules;
any one of the plurality of parallel processing modules is used for processing any one of the plurality of received laser origin cloud data sub-blocks and the position and posture data sub-block corresponding to the any one of the plurality of laser origin cloud data sub-blocks to obtain laser point cloud result data.
In one possible implementation manner, the blocking module includes a laser origin cloud data blocking unit, a position and posture data blocking unit, and a distributing unit, where the laser origin cloud data blocking unit is configured to block the laser origin cloud data, the position and posture data blocking unit is configured to block the position and posture data, and the distributing unit is configured to distribute any one of the plurality of laser origin cloud data sub-blocks and a position and posture data sub-block corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules.
In one possible implementation, the blocking module blocks the laser origin cloud data and the position and posture data according to a time interval, and generates the plurality of laser origin cloud data sub-blocks and the plurality of position and posture data sub-blocks according to the time interval.
In a possible implementation manner, each parallel processing module is in heartbeat connection with the blocking module, and each parallel processing module sends idle information to the blocking module every time t to report whether the parallel processing module is in an idle state or not; after the blocking module receives the idle information sent by each parallel processing module, the blocking module records the serial number of the parallel processing module and the time for receiving the idle information sent by the parallel processing module, and the blocking module orders the parallel processing modules for sending the idle information according to the time sequence for receiving the idle information sent by the parallel processing module.
In one possible implementation manner, any one of the plurality of parallel processing modules includes a GPU parallel processing unit, where the GPU parallel processing unit reads laser origin clouds in any one of the plurality of received laser origin cloud data sub-blocks in time sequence, reads m laser origin clouds each time, and when the m laser origin clouds are less than, reads all the laser origin clouds, and for each laser origin cloud, the GPU parallel processing unit converts an angular distance of each read laser origin cloud into absolute coordinates according to each laser origin cloud and position posture data corresponding to the laser origin cloud read in time matching; after all the angle distances of the laser original point clouds in the laser original point cloud data sub-blocks received by the parallel processing module are converted into absolute coordinates, the parallel processing module stores the laser original point cloud data sub-blocks into an open point cloud format LAS file and records the open point cloud format LAS file as a block LAS.
As a second aspect of the present invention, the present invention also discloses a method for generating efficient data based on parallel processing, comprising the steps of,
s1, a blocking module blocks laser original point cloud data to obtain a plurality of laser original point cloud data sub-blocks, and the blocking module also blocks position and posture data to obtain a plurality of position and posture data sub-blocks; the blocking module distributes any one of the plurality of laser origin cloud data sub-blocks and the position and posture data sub-blocks corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the parallel processing modules;
s2, any parallel processing module processes any received plurality of laser original point cloud data sub-blocks and position and posture data sub-blocks corresponding to any laser original point cloud data sub-blocks to obtain laser point cloud result data.
In one possible implementation manner, the blocking module includes a laser origin cloud data blocking unit, a position and posture data blocking unit, and a distributing unit, where the laser origin cloud data blocking unit is configured to block the laser origin cloud data, the position and posture data blocking unit is configured to block the position and posture data, and the distributing unit is configured to distribute any one of the plurality of laser origin cloud data sub-blocks and a position and posture data sub-block corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules.
In one possible implementation, the blocking module blocks the laser origin cloud data and the position and posture data according to a time interval, and generates the plurality of laser origin cloud data sub-blocks and the plurality of position and posture data sub-blocks according to the time interval.
In one possible implementation manner, each parallel processing module is in heartbeat connection with the blocking module, and each parallel processing module sends idle information to the blocking module every time t to report whether the parallel processing module is in an idle state or not; after the blocking module receives the idle information sent by each parallel processing module, the blocking module records the serial number of the parallel processing module and the time for receiving the idle information sent by the parallel processing module, and the blocking module orders the parallel processing modules for sending the idle information according to the time sequence for receiving the idle information sent by the parallel processing module.
In one possible implementation manner, any one of the plurality of parallel processing modules includes a GPU parallel processing unit, where the GPU parallel processing unit reads laser origin clouds in any one of the plurality of received laser origin cloud data sub-blocks in time sequence, reads m laser origin clouds each time, and when the m laser origin clouds are less than, reads all the laser origin clouds, and for each laser origin cloud, the GPU parallel processing unit converts an angular distance of each read laser origin cloud into absolute coordinates according to each laser origin cloud and position posture data corresponding to the laser origin cloud read in time matching; after all the angle distances of the laser original point clouds in the laser original point cloud data sub-blocks received by the parallel processing module are converted into absolute coordinates, the parallel processing module stores the laser original point cloud data sub-blocks into an open point cloud format LAS file and records the open point cloud format LAS file as a block LAS.
(III) beneficial effects
Compared with the prior art, the invention has the following beneficial effects: the invention is mainly characterized in that the method combines the distributed environment and GPU parallelization to realize the rapid conversion from laser point original data to result data. The original laser point cloud data and the position posture data are respectively segmented to form segmented data processing tasks, the data processing tasks are automatically distributed according to the monitored cluster resource state to form a distributed processing environment, the distributed processing environment is processed in parallel in the GPU, rapid conversion of the segmented data is achieved, and finally the converted segmented point cloud data are fused to generate complete laser point cloud result data. The cluster and the GPU are used for processing the laser point cloud data in parallel, so that the processing efficiency of the laser point cloud data is greatly improved, the resource configuration is optimized, and a scientific and feasible method is provided for the application of the laser point cloud data in the industry.
Additional advantages, objects, and features of the application will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the application. The objects and other advantages of the present application may be realized and attained by the written description which follows.
Drawings
The embodiments described below with reference to the drawings are exemplary and intended for the purpose of illustrating and explaining the present application and are not to be construed as limiting the scope of protection of the present application.
FIG. 1 is a system block diagram of the present application;
FIG. 2 is a block module architecture diagram of the present application;
FIG. 3 is a block diagram of a parallel processing module of the present application;
FIG. 4 is a block diagram of GPU parallel processing in accordance with the present application;
fig. 5 is a flow chart of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, which are generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, as provided in the accompanying drawings, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further definition or explanation thereof is necessary in the following figures.
Referring to fig. 1, a parallel processing-based efficient data generating system provided by an embodiment of the present invention includes a block module and a plurality of parallel processing modules, where the plurality of parallel processing modules are respectively connected with the block module, and the plurality of parallel processing modules include a first parallel processing module, a second parallel processing module, and a third parallel processing module.
The blocking module is used for blocking the laser original point cloud data to obtain a plurality of laser original point cloud data sub-blocks, and is also used for blocking the position and posture data to obtain a plurality of position and posture data sub-blocks; the blocking module distributes any one of the plurality of laser origin cloud data sub-blocks and position and posture data sub-blocks corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules; any one of the plurality of parallel processing modules is used for processing any one of the plurality of received laser origin cloud data sub-blocks and the position and posture data sub-block corresponding to the any one of the plurality of laser origin cloud data sub-blocks to obtain laser point cloud result data.
The original laser point cloud data and the position posture data are respectively segmented to form segmented data processing tasks, the data processing tasks are automatically distributed according to the monitored cluster resource state to form a distributed processing environment, the distributed processing environment is processed in parallel in the GPU, rapid conversion of the segmented data is achieved, and finally the converted segmented point cloud data are fused to generate complete laser point cloud result data. And the clusters and the GPU are used for processing the laser point cloud data in parallel, so that the processing efficiency of the laser point cloud data is greatly improved.
The blocking module comprises a laser original point cloud data blocking unit, a position posture data blocking unit and a distribution unit, wherein the laser original point cloud data blocking unit is used for blocking the laser original electric cloud data, the position posture data blocking unit is used for blocking the position posture data, and the distribution unit is used for distributing any one of the plurality of laser original point cloud data sub-blocks and the position posture data sub-block corresponding to any one of the plurality of laser original point cloud data sub-blocks to any one of the plurality of parallel processing modules; the blocking module blocks the laser original point cloud data and the position posture data according to time intervals, and generates a plurality of laser original point cloud data sub-blocks and a plurality of position posture data sub-blocks according to the time intervals. The time interval is equal to a preset dt, the original laser point cloud data are segmented, and the total block number is n:
n＝(Te-Ts)/dt
wherein, ts is the original laser point cloud data recording start time, and Te is the original laser point cloud data recording end time.
The starting time and the ending time of the front n-1 block point cloud are as follows:
wherein Ts i For the i-th block point cloud to start recording time, te i And ending the recording time for the ith block point cloud.
The starting time and ending time of the nth block of original laser point cloud data sub-blocks are:
wherein Ts n Recording time, te for the n-th original laser point cloud data sub-block n Recording time for the end of the nth block of original laser point cloud data sub-block.
As shown in fig. 2, the original laser point cloud data is segmented according to the start recording time and the end recording time of each original laser point cloud data sub-block, and recorded as a block i Representing an i-th block of original laser point cloud data sub-blocks.
Meanwhile, the position and posture data are segmented according to the start recording time and the end recording time of each block and recorded as PosSeg i Indicating the i-th segment position and orientation data.
As in fig. 3, a data processing task is generated by combining the corresponding point cloud block and the position gesture segment i Representing the ith task, the partitioning module distributes the data processing tasks to the parallel processing modules.
Each parallel processing module is connected with the blocking module in a heartbeat mode, and each parallel processing module sends idle information to the blocking module at intervals of time t to report whether the parallel processing module is in an idle state or not; after the blocking module receives the idle information sent by each parallel processing module, the blocking module records the serial number of the parallel processing module and the time for receiving the idle information sent by the parallel processing module, the parallel processing modules sending the idle information to the blocking module are ordered according to the time sequence for receiving the idle information sent by the parallel processing modules, the blocking module distributes data processing tasks to the parallel processing modules according to the time sequence, the blocking module also checks the task queue of each parallel processing module at intervals of time T, and if the task queue of each parallel processing module has a task to be processed, the blocking module takes out the task and distributes the task to the parallel processing module which enters the idle state first.
As shown in fig. 4, any one of the plurality of parallel processing modules includes a GPU parallel processing unit, the GPU parallel processing unit reads laser origin clouds in any one of the plurality of received laser origin cloud data sub-blocks in time sequence, reads m laser origin clouds each time, and when the m laser origin clouds are less than, reads all the laser origin clouds, and for each laser origin cloud, the GPU parallel processing unit converts an angle distance of each read laser origin cloud into absolute coordinates according to each laser origin cloud and position posture data corresponding to the laser origin cloud read in time matching; after all the angle distances of the laser original point clouds in the laser original point cloud data sub-blocks received by the parallel processing module are converted into absolute coordinates, the parallel processing module stores the laser original point cloud data sub-blocks into an open point cloud format LAS file and records the open point cloud format LAS file as a block LAS.
The m laser origin clouds read by the GPU each time satisfy the following conditions:
k×m＜S inner part
K is the storage size of each original point cloud, m is the number of points, and S is the memory size of the parallel processing module.
Further, recording the starting time of the LAS file and the corresponding laser original point cloud data sub-block, and defining the size of the final result LAS file as M; and writing the block LAS file into a final result LAS file according to the starting time sequence of the laser original point cloud data sub-blocks, and if M is exceeded, creating a new LAS file for writing.
As shown in fig. 5, as a second aspect of the present application, there is also provided a parallel processing-based efficient data generation method, including the steps of:
s1, a blocking module blocks laser original point cloud data to obtain a plurality of laser original point cloud data sub-blocks, and the blocking module also blocks position and posture data to obtain a plurality of position and posture data sub-blocks; the blocking module distributes any one of the plurality of laser origin cloud data sub-blocks and the position and posture data sub-blocks corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the parallel processing modules; the blocking module comprises a laser original point cloud data blocking unit, a position posture data blocking unit and a distribution unit, wherein the laser original point cloud data blocking unit is used for blocking the laser original electric cloud data, the position posture data blocking unit is used for blocking the position posture data, and the distribution unit is used for distributing any one of the plurality of laser original point cloud data sub-blocks and the position posture data sub-block corresponding to any one of the plurality of laser original point cloud data sub-blocks to any one of the plurality of parallel processing modules; the blocking module blocks the laser original point cloud data and the position posture data according to time intervals, and generates a plurality of laser original point cloud data sub-blocks and a plurality of position posture data sub-blocks according to the time intervals. The time interval is equal to a preset dt, the original laser point cloud data are segmented, and the total block number is n:
n＝(Te-Ts)/dt
wherein, ts is the original laser point cloud data recording start time, and Te is the original laser point cloud data recording end time.
The starting time and the ending time of the front n-1 block point cloud are as follows:
wherein Ts i For the i-th block point cloud to start recording time, te i And ending the recording time for the ith block point cloud.
The starting time and ending time of the nth block of original laser point cloud data sub-blocks are:
wherein Ts n Recording time, te for the n-th original laser point cloud data sub-block n For the end recording time of the nth block of original laser point cloud data sub-blocks, the original laser point cloud data is segmented according to the start recording time and the end recording time of each block of original laser point cloud data sub-blocks, and recorded as blocks i Representing an i-th block of original laser point cloud data sub-blocks.
Meanwhile, the position and posture data are segmented according to the start recording time and the end recording time of each block and recorded as PosSeg i Indicating the i-th segment position and orientation data. Generating a data processing task by combining the corresponding point cloud block and the position gesture segment i Representing the ith task, the partitioning module distributes the data processing tasks to the parallel processing modules.
Each parallel processing module is connected with the blocking module in a heartbeat mode, and each parallel processing module sends idle information to the blocking module at intervals of time t to report whether the parallel processing module is in an idle state or not; after the blocking module receives the idle information sent by each parallel processing module, the blocking module records the serial number of the parallel processing module and the time for receiving the idle information sent by the parallel processing module, the parallel processing modules sending the idle information to the blocking module are ordered according to the time sequence for receiving the idle information sent by the parallel processing modules, the blocking module distributes data processing tasks to the parallel processing modules according to the time sequence, the blocking module also checks the task queue of each parallel processing module at intervals of time T, and if the task queue of each parallel processing module has a task to be processed, the blocking module takes out the task and distributes the task to the parallel processing module which enters the idle state first.
S2, any parallel processing module processes any received multiple laser original point cloud data sub-blocks and position posture data sub-blocks corresponding to the multiple laser original point cloud data sub-blocks to obtain laser point cloud result data, and any parallel processing module comprises a GPU parallel processing unit, wherein the GPU parallel processing unit reads laser original point clouds in any received multiple laser original point cloud data sub-blocks in time sequence, and when m laser original point clouds are read each time, all the laser original point clouds are read each time, and each laser original point cloud is read, and the GPU parallel processing unit converts the angle distance of each read laser original point cloud into absolute coordinates according to each laser point cloud and position posture data corresponding to the laser point cloud according to time matching reading; after all the angle distances of the laser original point clouds in the laser original point cloud data sub-blocks received by the parallel processing module are converted into absolute coordinates, the parallel processing module stores the laser original point cloud data sub-blocks into an open point cloud format LAS file and records the open point cloud format LAS file as a block LAS.
The m laser origin clouds read by the GPU each time satisfy the following conditions:
k×m＜S inner part
K is the storage size of each original point cloud, m is the number of points, and S is the memory size of the parallel processing module.
Further, recording the starting time of the LAS file and the corresponding laser original point cloud data sub-block, and defining the size of the final result LAS file as M; and writing the block LAS file into a final result LAS file according to the starting time sequence of the laser original point cloud data sub-blocks, and if M is exceeded, creating a new LAS file for writing.
Finally, it is noted that the above embodiments are only for illustrating the technical solution of the present application and not for limiting the same, and although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present application may be modified or replaced without departing from the spirit and scope of the technical solution of the present application, and all such modifications are included in the scope of the claims of the present application.
Claims (10)
1. The efficient data generation system based on parallel processing is characterized by comprising a block module and a plurality of parallel processing modules, wherein the plurality of parallel processing modules are respectively connected with the block module;
the blocking module is used for blocking the laser original point cloud data to obtain a plurality of laser original point cloud data sub-blocks, and is also used for blocking the position and posture data to obtain a plurality of position and posture data sub-blocks; the blocking module automatically distributes data processing tasks according to the monitored cluster resource state, and distributes any one of the plurality of laser origin cloud data sub-blocks and position and posture data sub-blocks corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules to form a distributed processing environment;
any one of the plurality of parallel processing modules is used for processing any one of the plurality of received laser origin point cloud data sub-blocks and the position posture data sub-block corresponding to the any one of the plurality of laser origin point cloud data sub-blocks in parallel in the GPU, realizing rapid conversion of block data, and finally fusing the converted block point cloud data to generate complete laser point cloud result data.
2. The efficient data generation system based on parallel processing according to claim 1, wherein the blocking module includes a laser origin cloud data blocking unit for blocking the laser origin cloud data, a position posture data blocking unit for blocking the position posture data, and a distribution unit for distributing any one of the plurality of laser origin cloud data sub-blocks and position posture data sub-blocks corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules.
3. The parallel processing-based efficient data generation system according to claim 1 or 2, wherein the blocking module blocks the laser origin cloud data and the position and orientation data at time intervals, and generates the plurality of laser origin cloud data sub-blocks and the plurality of position and orientation data sub-blocks according to the time intervals.
4. A parallel processing based efficient data generation system according to claim 3 wherein each of said parallel processing modules maintains a heartbeat connection with said blocking module, each of said parallel processing modules transmitting idle information to said blocking module every time t reporting itself in an idle state; after the blocking module receives the idle information sent by each parallel processing module, the blocking module records the serial number of the parallel processing module and the time for receiving the idle information sent by the parallel processing module, and the blocking module orders the parallel processing modules for sending the idle information according to the time sequence for receiving the idle information sent by the parallel processing module.
5. The efficient data generation system based on parallel processing of claim 4, wherein any one of the plurality of parallel processing modules comprises a GPU parallel processing unit that reads laser origin clouds in any one of the received plurality of laser origin cloud data sub-blocks in time order, each time readmThe original point cloud of the laser is insufficientmWhen the laser origin clouds are all read, for each laser origin cloud, the GPU parallel processing unit converts the angle distance of each read laser origin cloud into absolute coordinates according to each laser origin cloud read in a time matching mode and the corresponding position and posture data of each laser origin cloud; after all the angle distances of the laser original point clouds in the laser original point cloud data sub-blocks received by the parallel processing module are converted into absolute coordinates, the parallel processing module stores the laser original point cloud data sub-blocks into an open point cloud format LAS file and records the open point cloud format LAS file as a block LAS.
6. A method for generating high-efficiency data based on parallel processing is characterized by comprising the following steps,
s1, a blocking module blocks laser original point cloud data to obtain a plurality of laser original point cloud data sub-blocks, and the blocking module also blocks position and posture data to obtain a plurality of position and posture data sub-blocks; the blocking module automatically distributes data processing tasks according to the monitored cluster resource state, and distributes any one of the plurality of laser original point cloud data sub-blocks and position posture data sub-blocks corresponding to the any one of the plurality of laser original point cloud data sub-blocks to any one of the parallel processing modules to form a distributed processing environment;
s2, any parallel processing module processes any received multiple laser original point cloud data sub-blocks and position posture data sub-blocks corresponding to the laser original point cloud data sub-blocks in parallel in the GPU, rapid conversion of block data is achieved, and finally the converted block point cloud data are fused to generate complete laser point cloud result data.
7. The efficient data generation method based on parallel processing according to claim 6, wherein the blocking module comprises a laser origin cloud data blocking unit, a position posture data blocking unit and a distribution unit, the laser origin cloud data blocking unit is used for blocking the laser origin cloud data, the position posture data blocking unit is used for blocking the position posture data, and the distribution unit is used for distributing any one of the plurality of laser origin cloud data sub-blocks and position posture data sub-blocks corresponding to the any one of the plurality of laser origin cloud data sub-blocks to any one of the plurality of parallel processing modules.
8. The efficient data generation method based on parallel processing according to claim 6 or 7, wherein the blocking module blocks the laser origin cloud data and the position posture data at time intervals, and generates the plurality of laser origin cloud data sub-blocks and the plurality of position posture data sub-blocks according to the time intervals.
9. The efficient data generation method based on parallel processing according to claim 8, wherein each parallel processing module is in heartbeat connection with the block module, and each parallel processing module sends idle information to the block module every time t to report whether the parallel processing module is in an idle state or not; after the blocking module receives the idle information sent by each parallel processing module, the blocking module records the serial number of the parallel processing module and the time for receiving the idle information sent by the parallel processing module, and the blocking module orders the parallel processing modules for sending the idle information according to the time sequence for receiving the idle information sent by the parallel processing module.
10. The efficient data generation method based on parallel processing according to claim 9, wherein any one of the plurality of parallel processing modules includes a GPU parallel processing unit that reads laser origin clouds in any one of the plurality of received laser origin cloud data sub-blocks in time sequence, each time the laser origin clouds are readmThe original point cloud of the laser is insufficientmWhen the laser origin clouds are all read, for each laser origin cloud, the GPU parallel processing unit converts the angle distance of each read laser origin cloud into absolute coordinates according to each laser origin cloud read in a time matching mode and the corresponding position and posture data of each laser origin cloud; after all the angle distances of the laser original point clouds in the laser original point cloud data sub-blocks received by the parallel processing module are converted into absolute coordinates, the parallel processing module stores the laser original point cloud data sub-blocks into an open point cloud format LAS file and records the open point cloud format LAS file as a block LAS.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211423227.8A CN115756841B (en) | 2022-11-15 | 2022-11-15 | Efficient data generation system and method based on parallel processing |
| PCT/CN2022/132345 WO2024103309A1 (en) | 2022-11-15 | 2022-11-16 | Parallel-processing-based efficient data generation system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211423227.8A CN115756841B (en) | 2022-11-15 | 2022-11-15 | Efficient data generation system and method based on parallel processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115756841A CN115756841A (en) | 2023-03-07 |
| CN115756841B true CN115756841B (en) | 2023-07-11 |
Family
ID=85370644
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211423227.8A Active CN115756841B (en) | 2022-11-15 | 2022-11-15 | Efficient data generation system and method based on parallel processing |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN115756841B (en) |
| WO (1) | WO2024103309A1 (en) |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW200803464A (en) * | 2006-01-10 | 2008-01-01 | Brightscale Inc | Method and apparatus for scheduling the processing of multimedia data in parallel processing systems |
| CN107515952B (en) * | 2017-09-21 | 2023-10-27 | 北京星闪世图科技有限公司 | Method and system for point cloud data storage, parallel computing and real-time retrieval |
| CN109145969B (en) * | 2018-08-03 | 2020-07-28 | 百度在线网络技术（北京）有限公司 | Method, device, equipment and medium for processing point cloud data of three-dimensional object |
| CN111936821A (en) * | 2019-07-12 | 2020-11-13 | 北京航迹科技有限公司 | System and method for positioning |
| CN111882977B (en) * | 2020-05-06 | 2022-04-29 | 北京嘀嘀无限科技发展有限公司 | High-precision map construction method and system |
| CN111338804B (en) * | 2020-05-19 | 2020-11-24 | 北京数字绿土科技有限公司 | Resolving method and device for laser radar point cloud data and multithreading processing system |
| CN111694012A (en) * | 2020-05-29 | 2020-09-22 | 北京航空航天大学合肥创新研究院 | Three-dimensional terrain online generation method and system based on airborne laser radar |
| CN112363837A (en) * | 2020-11-19 | 2021-02-12 | 北京航天泰坦科技股份有限公司 | Cluster environment-based point cloud data parallel processing method, device, equipment and storage medium |
-
2022
- 2022-11-15 CN CN202211423227.8A patent/CN115756841B/en active Active
- 2022-11-16 WO PCT/CN2022/132345 patent/WO2024103309A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024103309A1 (en) | 2024-05-23 |
| CN115756841A (en) | 2023-03-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Real-time GIS for smart cities | |
| Karimi | Big Data: techniques and technologies in geoinformatics | |
| Dangermond et al. | Building geospatial infrastructure | |
| US11367346B2 (en) | Digitizing and mapping the public space using collaborative networks of mobile agents and cloud nodes | |
| Eken et al. | A smart bus tracking system based on location-aware services and QR codes | |
| CN107590250A (en) | A kind of space-time orbit generation method and device | |
| Li et al. | Big data and cloud computing | |
| CN103020228A (en) | Interest point (POI) data positioning in an image | |
| US20200042657A1 (en) | Multi-dimensional event model generation | |
| US20110285715A1 (en) | Method and System for Providing Scene Data of Virtual World | |
| Jisha et al. | An android application for school bus tracking and student monitoring system | |
| JP2023159079A (en) | Data management method of memory device and data management system | |
| JP5388243B2 (en) | Management server, population information calculation management server, absent area management method, and population information calculation method | |
| CN112785030A (en) | Ship behavior recognition system, method and platform | |
| CN103096129A (en) | System and method of multi-satellite remote sensing data remote real-time reporting | |
| CN108509495A (en) | The processing method and processing device of seismic data, storage medium, processor | |
| Kramar et al. | Augmented Reality-assisted Cyber-Physical Systems of Smart University Campus | |
| CN114818218A (en) | Cable laying path determining method, device and storage medium | |
| CN110505444A (en) | Safety defense monitoring system based on big data | |
| CN115756841B (en) | Efficient data generation system and method based on parallel processing | |
| Birnbaum et al. | Similarity-based compression of GPS trajectory data | |
| CN112203054B (en) | Monitoring video point location marking method and device, storage medium and electronic equipment | |
| Lv et al. | Research on ship dynamic map based on AIS | |
| Muslim et al. | Automatic Image annotation for small and ad hoc intelligent applications using Raspberry Pi | |
| CN104966143A (en) | GIS-based emergency resource situation map generating system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |