CN111722887A - Method and system for processing and displaying massive scattered point data - Google Patents
Method and system for processing and displaying massive scattered point data Download PDFInfo
- Publication number
- CN111722887A CN111722887A CN202010482582.7A CN202010482582A CN111722887A CN 111722887 A CN111722887 A CN 111722887A CN 202010482582 A CN202010482582 A CN 202010482582A CN 111722887 A CN111722887 A CN 111722887A
- Authority
- CN
- China
- Prior art keywords
- data
- processing
- displaying
- request
- configuration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012545 processing Methods 0.000 title claims abstract description 75
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000005070 sampling Methods 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 239000008186 active pharmaceutical agent Substances 0.000 abstract description 6
- 238000012423 maintenance Methods 0.000 abstract description 5
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 abstract 1
- 230000006870 function Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4494—Execution paradigms, e.g. implementations of programming paradigms data driven
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method and a system for processing and displaying massive scattered point data. The method comprises the following steps: converting external raw data into a designed intermediate cache format and using a working thread; after a user initiates a request, configuring, requesting and calculating the data in the intermediate cache format; and after configuration, request and calculation are finished, the processing result is fed back to the user in a segmented mode. The invention does not depend on a display card, especially a display card of a specific brand; parallel computing APIs (CUDA and OpenCL) with higher thresholds are not required to be additionally introduced, so that the learning and maintenance cost is reduced; compared with a conventional single-working thread or single-thread linear processing mode, the scheme can finish processing and displaying a large amount of data in a short time, and the fluency experience of a user is improved.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a method and a system for processing and displaying massive scattered point data.
Background
The current processing idea for discrete data is 'multi-line start-up' -which utilizes the 'context-free' characteristic of simultaneous processing on multiple pipelines. The mature scheme is that CUDA is used, but the CUDA relies on the display card of great in itself, which means that the dependence of the program on hardware is increased, and the performance of the display card is tested; in addition, OpenCL is also sufficient, but compared with CUDA, although the restriction of a display card of a specific brand is removed, the use difficulty is further improved. Wherein:
context-free: the data individuals have no association constraint, and the respective processing process and result do not influence other data.
CUDA: CUDA (computer Unified Device architecture), which is a computing platform introduced by NVIDIA (video card vendor). CUDA is a general-purpose parallel computing architecture derived from NVIDIA that enables GPUs to solve complex computational problems. It contains the CUDA Instruction Set Architecture (ISA) and the parallel computing engine inside the GPU. Developers can write programs for the CUDA architecture using the C language, which is one of the most widely used high-level programming languages. The written program can run with ultra-high performance on a processor that supports CUDA.
OpenCL: is called Open Computing Language in all. OpenCL is a framework for writing programs for heterogeneous platforms, which may be composed of CPUs, GPUs, or other types of processors. OpenCL consists of a language (based on C99) for writing kernels (functions running on OpenCL devices) and a set of APIs for defining and controlling the platform. OpenCL provides a parallel computing mechanism based on task segmentation and data segmentation.
The working threads of the thread pool can avoid accumulating the work in the main thread, thereby causing interface blockage; multithreading parallel computing can greatly improve efficiency. When the interface is drawn, the data volume is large and is only an expression, and what is involved behind the interface is the huge overhead generated by the execution mode, namely the more scattered points, the more the instructions of drawing points in a single time need to be executed, so that the data is transmitted to the target all at once as much as possible, and the method is also an effective means for reducing the overhead. Therefore, a method and a system for processing and displaying a large amount of scatter point data are urgently needed to be provided to solve the problems of time consumption and jamming caused by processing when the scatter point data in the prior art is very huge. For the problem that the discrete data processing needs to depend on a display card, parallel computing APIs (CUDA and OpenCL) with higher thresholds need to be additionally introduced, and the learning and maintenance costs are reduced.
Disclosure of Invention
In view of the foregoing defects in the prior art, the technical problem to be solved by the present invention is to provide a method and a system for processing and displaying a large amount of discrete data, so as to solve the problems in the prior art that a graphics card is required to be relied on for processing discrete data, and parallel computing APIs (CUDA and OpenCL) with a higher threshold need to be additionally introduced, thereby reducing the learning and maintenance costs.
In order to achieve the above object, the present invention provides a method for processing and displaying massive scattered point data, comprising the following steps:
converting external raw data into a designed intermediate cache format and using a working thread;
after a user initiates a request, configuring, requesting and calculating the data in the intermediate cache format;
and after configuration, request and calculation are finished, the processing result is fed back to the user in a segmented mode.
Preferably, the cache is designed in a manner of using a multi-level hash table or a hash table with sequential containers.
Preferably, the worker thread is a worker thread or a thread pool.
Preferably, the configuring, requesting and calculating are specifically:
configuration: organizing configuration items for requesting calculation according to a designed format, wherein the content of the configuration items comprises the position and the size of sampling and the scaling in the horizontal and vertical directions;
requesting: transmitting the configuration items to a processing unit and enabling the processing unit to run processing;
and (3) calculating: and dividing the cache data according to the configuration items, creating a thread pool, distributing and cutting blocks, and asynchronously finishing the processing work of each block by a plurality of working threads.
Preferably, the configuration process further includes a display requirement for any scaling factor of a certain sampling region, and the processing of the scatter data has the following formula:
Xaisplay＝Scalex*(Xsplash-Xtopleft)
Ydisplay＝Scaley，*(Ysplash-Ytopleft)
wherein display refers to display, scale is zoom magnification, and splash is scatter; toplex is the upper left corner coordinate of the sample area.
Preferably, after the user obtains the processing result, the user displays the result on the GUI interface: if the selected output is a block data set, namely the block data set comprises coordinates of the upper left corner and a scattered point set, using the function of drawing points; if the obtained picture is an 8-bit picture, the function of drawing the picture is used.
A system for processing and displaying a large amount of scatter data, comprising:
the data conversion module is used for converting external original data into a designed intermediate cache format and needing to use a working thread;
the data request processing module is used for configuring, requesting and calculating the data in the intermediate cache format after a user initiates a request;
and the data feedback module is used for feeding back the processing result to the user in a segmented manner after completing configuration, request and calculation.
Preferably, the data conversion module is configured to convert external raw data into a designed intermediate cache format: the cache is designed in a mode of using a multi-level hash table or a hash table matched with a sequential container.
Preferably, the work thread used by the data conversion module is a work thread or a thread pool.
Preferably, the data request processing module configures, requests, and calculates the data in the intermediate cache format after the user initiates the request, specifically:
configuration: organizing configuration items for requesting calculation according to a designed format, wherein the content of the configuration items comprises the position and the size of sampling and the scaling in the horizontal and vertical directions;
requesting: transmitting the configuration items to a processing unit and enabling the processing unit to run processing;
and (3) calculating: and dividing the cache data according to the configuration items, creating a thread pool, distributing and cutting blocks, and asynchronously finishing the processing work of each block by a plurality of working threads.
Preferably, the configuration process of the data request processing module further includes a display requirement for any zoom factor of a certain sampling region, and the processing of the scatter data has the following formula:
Xdisplay＝Scalex*(Xsplash-Xtopleft)
Ydisplay＝Scale3*(Ysplash-Ytopleft)
wherein display refers to display, scale is zoom magnification, and splash is scatter; toplex is the upper left corner coordinate of the sample area.
Preferably, the system further comprises an interface refreshing module, configured to display the result on the GUI interface after the user obtains the processing result: if the selected output is a block data set, namely the block data set comprises coordinates of the upper left corner and a scattered point set, using the function of drawing points; if the obtained picture is an 8-bit picture, the function of drawing the picture is used.
The invention has the beneficial effects that:
the invention does not depend on a display card, especially a display card of a specific brand; parallel computing APIs (CUDA and OpenCL) with higher thresholds are not required to be additionally introduced, so that the learning and maintenance cost is reduced; compared with a conventional single-working thread or single-thread linear processing mode, the scheme can finish processing and displaying a large amount of data in a short time, and the fluency experience of a user is improved.
The conception, the specific structure and the technical effects of the present invention will be further described with reference to the accompanying drawings to fully understand the objects, the features and the effects of the present invention.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a schematic block diagram of the system of the present invention.
FIG. 3 is a schematic diagram of an 8-bit picture according to the present invention in which each pixel value is recorded with only one index value pointing to a color table.
Detailed Description
As shown in fig. 1, the present invention provides a method for processing and displaying a large amount of scattered point data, which comprises the following steps:
the first step is as follows: converting external raw data into a designed intermediate cache format, and using a working thread (the working thread is a working thread or a thread pool), wherein the consideration is three;
1) on the premise that the original data is not changed, the converted format is convenient for repeated segmentation sampling aiming at different requests;
2) the cached data needs to provide support for rapid pick-up detection, so that a user can rapidly judge which scattered points should be picked up at the mouse position;
3) the use of the working thread avoids the blockage of the main thread caused by the conversion of a large amount of data. (if the main thread blocks, the program will not be able to respond to the user behavior in time at this point, affecting the experience).
The cache is designed in a mode of using a multi-level hash table or a hash table matched with a sequential container. Such as hash < x, hash < y, vector < pointf > > > and hash < x, hash < y, set < pointf > >, also can be hash < point, vector < pointf > > or hash < point, set < pointf >. The hash refers to a hash table with a key and value pair, the vector is a sequential container, the set is the hash table with only keys, x and y refer to integers of horizontal and vertical coordinates of scatter points, the point consists of x and y, the point is the original data of the scatter points, and the horizontal and vertical coordinates are floating point numbers. Since multiple floating point coordinates may have the same integer coordinate after approximating an integer, x, y, or point maps to a container (vector or set) instead of a single point. The nature of the hash table makes data very fast both in inserting and looking for matches, which is consistent with the required positioning to handle large amounts of scatter data.
The second step is that: after a user initiates a request, configuring, requesting and calculating the data in the intermediate cache format, wherein the configuring, requesting and calculating specifically comprise:
configuration: according to a designed format, organizing configuration items for requesting calculation, wherein the content of the configuration items comprises the positions and the sizes of the samples and the scaling ratios in the horizontal and vertical directions:
requesting: transmitting the configuration items to a processing unit and enabling the processing unit to run processing;
and (3) calculating: and dividing the cache data according to the configuration items, creating a thread pool, distributing and cutting blocks, and asynchronously finishing the processing work of each block by a plurality of working threads.
Meanwhile, the configuration process also comprises the display requirement of any zoom magnification of a certain sampling region, and the processing of the scattered point data has the following formula:
Xdisplay＝Scalex*(Xsplash-Xtopleft)
Ydisplay＝Scaley*(Ysplash-Ytopleft)
wherein display refers to display, scale is zoom magnification, and splash is scatter; toplex is the upper left corner coordinate of the sample area. There are two definitions according to the actual treatment method:
1) sharing a unified upper left corner by one request, wherein the coordinate of the upper left corner is the upper left corner of the sampling area;
2) and each block calculates the upper left corner of the local area of the block and independently calculates and processes the block by taking the block as a unit. The obtained result (x, y) is stored in a set as point, and the purpose of the result is to eliminate repeated coordinate points as key values, so that the effect similar to clipping and removing is realized, and unnecessary subsequent processing actions are reduced. (after the set is collected, the container can be optionally changed to a conventional container, such as a vector and then fed back to the next step)
The third step: and after configuration, request and calculation are finished, the processing result is fed back to the user in a segmented mode. The block information contains a starting point (i.e. the upper left-hand corner coordinates, which is helpful to be compatible with the two processing methods mentioned in the previous step), and a set of processed scatters (optionally, a set or vector, etc. container is used for holding). Before this, the block data may be processed again: because of the display of scatter data, each point is a single pixel point, and at least in the same batch, the colors are consistent. According to the conventional characteristic, an 8-bit image can be selected to replace a scatter set, and the result of the pixel map is directly fed back. As shown in fig. 3, in the 8-bit picture, each pixel value is recorded by only one index value pointing to the color table, not by the individual R, G, B color values, so that the picture is simple and fast to store and can be directly written in the form of an array. (specifically, Qt's QImage can be considered, and Format selects QImage:: Format _ Index 8).
And finally, after the user obtains the processing result, displaying the result on a GUI: if the selected output is a block data set, namely the block data set comprises coordinates of the upper left corner and a scattered point set, using the function of drawing points; if the obtained picture is an 8-bit picture, the function of drawing the picture is used.
As shown in fig. 2, the present invention further provides a system for processing and displaying a large amount of scattered point data, which includes:
the data conversion module is used for converting external original data into a designed intermediate cache format and needing to use a working thread;
the data request processing module is used for configuring, requesting and calculating the data in the intermediate cache format after a user initiates a request;
and the data feedback module is used for feeding back the processing result to the user in a segmented manner after completing configuration, request and calculation.
The specific implementation method has already been described above, and is not described herein again.
In conclusion, the invention does not depend on the display card, especially the display card of a specific brand; parallel computing APIs (CUDA and 0penCL) with higher thresholds are not required to be additionally introduced, so that the learning and maintenance cost is reduced; compared with a conventional single-working thread or single-thread linear processing mode, the scheme can finish processing and displaying a large amount of data in a short time, and the fluency experience of a user is improved.
The foregoing detailed description of the preferred embodiments of the invention has been presented. It should be understood that numerous modifications and variations could be devised by those skilled in the art in light of the present teachings without departing from the inventive concepts. Therefore, the technical solutions available to those skilled in the art through logic analysis, reasoning and limited experiments based on the prior art according to the concept of the present invention should be within the scope of protection defined by the claims.
Claims (10)
1. A method for processing and displaying massive scattered point data is characterized by comprising the following steps:
converting external raw data into a designed intermediate cache format and using a working thread; after a user initiates a request, configuring, requesting and calculating the data in the intermediate cache format;
after configuration, request and calculation are completed, the processing result is fed back to the user in a segmented manner;
the configuration, request and calculation specifically include:
configuration: organizing configuration items for requesting calculation according to a designed format, wherein the content of the configuration items comprises the position and the size of sampling and the scaling in the horizontal and vertical directions;
requesting: transmitting the configuration items to a processing unit and enabling the processing unit to run processing;
and (3) calculating: and dividing the cache data according to the configuration items, creating a thread pool, distributing and cutting blocks, and asynchronously finishing the processing work of each block by a plurality of working threads.
2. The method for processing and displaying mass scattered point data as claimed in claim 1, wherein: the cache is designed in a mode of using a multi-level hash table or a hash table matched with a sequential container.
3. The method for processing and displaying mass scattered point data as claimed in claim 1, wherein: the worker thread is a worker thread or a thread pool.
4. The method as claimed in claim 1, wherein the configuration process further includes an arbitrary zoom ratio display requirement for a certain sampling region, and the processing of the scatter data has the following formula:
Xdisplay＝Scalex*(Xsplash-Xtopleft)
Ydisplay＝Scaley*(Ysplash-Ytopleft)
wherein display refers to display, scale is zoom magnification, and splash is scatter; toplex is the upper left corner coordinate of the sample area.
5. The method for processing and displaying mass scattered point data as claimed in claim 1, wherein: and after the user obtains the processing result, displaying the result on a GUI (graphical user interface): if the selected output is a block data set, namely the block data set comprises coordinates of the upper left corner and a scattered point set, using the function of drawing points; if the obtained picture is an 8-bit picture, the function of drawing the picture is used.
6. A system for processing and displaying a large amount of scatter data, comprising:
the data conversion module is used for converting external original data into a designed intermediate cache format and needing to use a working thread;
the data request processing module is used for configuring, requesting and calculating the data in the intermediate cache format after a user initiates a request;
the data feedback module is used for feeding back the processing result to the user in a segmented manner after completing configuration, request and calculation;
the data request processing module configures, requests and calculates data in an intermediate cache format after a user initiates a request, and specifically comprises:
configuration: organizing configuration items for requesting calculation according to a designed format, wherein the content of the configuration items comprises the position and the size of sampling and the scaling in the horizontal and vertical directions;
requesting: transmitting the configuration items to a processing unit and enabling the processing unit to run processing;
and (3) calculating: and dividing the cache data according to the configuration items, creating a thread pool, distributing and cutting blocks, and asynchronously finishing the processing work of each block by a plurality of working threads.
7. The system for processing and displaying mass scattered point data as claimed in claim 6, wherein the data conversion module is configured to convert external raw data into a designed intermediate cache format: the cache is designed in a mode of using a multi-level hash table or a hash table matched with a sequential container.
8. The system for processing and displaying mass scattered point data as claimed in claim 7, wherein the working thread used by the data conversion module is a working thread or a thread pool.
9. The system for processing and displaying massive amounts of scatter data as claimed in claim 8, wherein the configuration process of the data request processing module further comprises an arbitrary zoom ratio exhibition requirement for a certain sampling region, and the scatter data is processed according to the following formula:
Xdisplay＝Scalex*(Xsplash-Xtopleft)
Ydisplay＝Scaley*(Ysplash-Ytopleft)
wherein display refers to display, scale is zoom magnification, and splash is scatter; toplex is the upper left corner coordinate of the sample area.
10. The system for processing and displaying mass scattered point data as claimed in claim 6, further comprising an interface refreshing module for displaying the result on a GUI interface after the user obtains the processing result: if the selected output is a block data set, namely the block data set comprises coordinates of the upper left corner and a scattered point set, using the function of drawing points; if the obtained picture is an 8-bit picture, the function of drawing the picture is used.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010482582.7A CN111722887A (en) | 2020-05-30 | 2020-05-30 | Method and system for processing and displaying massive scattered point data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010482582.7A CN111722887A (en) | 2020-05-30 | 2020-05-30 | Method and system for processing and displaying massive scattered point data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111722887A true CN111722887A (en) | 2020-09-29 |
Family
ID=72565419
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010482582.7A Pending CN111722887A (en) | 2020-05-30 | 2020-05-30 | Method and system for processing and displaying massive scattered point data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111722887A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018018941A1 (en) * | 2016-07-28 | 2018-02-01 | 武汉斗鱼网络科技有限公司 | Method and system for zooming image by using cooperation between opengl and opencl |
| CN110647013A (en) * | 2019-08-30 | 2020-01-03 | 合肥芯碁微电子装备有限公司 | GDSII format-based parallel data processing method for direct-write lithography machine |
-
2020
- 2020-05-30 CN CN202010482582.7A patent/CN111722887A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018018941A1 (en) * | 2016-07-28 | 2018-02-01 | 武汉斗鱼网络科技有限公司 | Method and system for zooming image by using cooperation between opengl and opencl |
| CN110647013A (en) * | 2019-08-30 | 2020-01-03 | 合肥芯碁微电子装备有限公司 | GDSII format-based parallel data processing method for direct-write lithography machine |
Non-Patent Citations (1)
| Title |
|---|
| 李涛等: "基于线程池的GPU任务并行计算模式研究", 计算机学报, no. 10, 29 December 2017 (2017-12-29) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11049018B2 (en) | Transforming convolutional neural networks for visual sequence learning | |
| US10733691B2 (en) | Fragment shaders perform vertex shader computations | |
| US8063903B2 (en) | Edge evaluation techniques for graphics hardware | |
| US8639730B2 (en) | GPU assisted garbage collection | |
| US10013731B2 (en) | Maximizing parallel processing in graphics processors | |
| US7847798B1 (en) | Clipping and culling artwork using a graphics processing unit | |
| CN106340055A (en) | Multithreading-based OpenGL quick drawing method | |
| CN106023091B (en) | The real-time defogging method of image based on graphics processor | |
| US20140347375A1 (en) | Developer controlled layout | |
| CN113393564B (en) | Pool-based spatio-temporal importance resampling using global illumination data structures | |
| WO2018018941A1 (en) | Method and system for zooming image by using cooperation between opengl and opencl | |
| Wang et al. | Parallelization and performance optimization on face detection algorithm with OpenCL: A case study | |
| US10152310B2 (en) | Fusing a sequence of operations through subdividing | |
| CN108648213A (en) | A kind of implementation method of KCF track algorithms on TMS320C6657 | |
| WO2018044437A1 (en) | Mechanism to increase thread parallelism in a graphics processor | |
| US8264503B1 (en) | Batch early engagement rules graphics processing | |
| US11397615B2 (en) | Methods and apparatuses for coalescing function calls for ray-tracing | |
| CN106886974A (en) | Image accelerator equipment and correlation technique | |
| US9019284B2 (en) | Input output connector for accessing graphics fixed function units in a software-defined pipeline and a method of operating a pipeline | |
| CN111722887A (en) | Method and system for processing and displaying massive scattered point data | |
| Zhang et al. | Weighted mean shift object tracking implemented on gpu for embedded sustems | |
| US20140362094A1 (en) | System, method, and computer program product for recovering from a memory underflow condition associated with generating video signals | |
| Saidovich et al. | Analysis Database Systems and Solve Medical Problems | |
| EP3321793A1 (en) | Method and system for real-time slide displaying of scanned image | |
| TW201635243A (en) | Facilitating culling of composite objects in graphics processing units when such objects produce no visible change in graphics images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |