CN111158898A - BIM data processing method and device aiming at power transmission and transformation project site arrangement standardization - Google Patents
BIM data processing method and device aiming at power transmission and transformation project site arrangement standardization Download PDFInfo
- Publication number
- CN111158898A CN111158898A CN201911164702.2A CN201911164702A CN111158898A CN 111158898 A CN111158898 A CN 111158898A CN 201911164702 A CN201911164702 A CN 201911164702A CN 111158898 A CN111158898 A CN 111158898A
- Authority
- CN
- China
- Prior art keywords
- data
- gpu
- bim
- node
- initializing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/544—Buffers; Shared memory; Pipes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention provides a BIM data processing method and a device aiming at power transmission and transformation project site arrangement standardization. The technical scheme provided by the embodiment of the invention can effectively solve the technical problem of slow data processing caused by rapid increase of data volume, various data types and high data updating speed when the BIM data processing with standardized arrangement in the power transmission and transformation project site is carried out.
Description
Technical Field
The invention relates to the field of power transmission and transformation engineering, in particular to a BIM data processing method and device aiming at power transmission and transformation engineering site layout standardization.
Background
In recent years, with the continuous expansion of the scale of a power grid and the new stage of rapid development of the construction of an extra-high voltage power grid, the number of new and improved projects of a transformer substation tends to increase rapidly. When the construction of the transformer substation is rapidly developed, the problems of short project period, difficult guarantee of engineering quality, poor cost controllability and the like are increasingly highlighted. The BIM technology is a typical representative of the new technology, and the BIM refers to a method and technology for managing and optimizing the whole process of design, construction and the like of a project under construction by integrating and extracting data such as engineering project geometry, attributes, materials and the like, creating and utilizing a digital model. Through a virtual information platform, the model is used for carrying out simulation research on the performance, cost, constructability and operation condition of the building in advance at the actual engineering decision stage, and the found problems are solved in advance or a solution is designated, so that the optimal project implementation scheme is obtained. At present, the popularization and application of the BIM technology in the construction of the transformer substation are gradually carried out, the intellectualization of the layout of the power transmission and transformation engineering field is the development direction of the construction of the current transformer substation, and the BIM technology is already applied to large-scale power engineering projects in China and achieves good results.
When standardized operation is arranged aiming at a power transmission and transformation project site based on the BIM technology, a three-dimensional model is required to be used as a carrier, information of each stage such as early stage, project establishment, bid inviting, design, construction and the like is completely reserved, and the deliverable digital assets are formed through processing and analysis of the information. However, when BIM data is processed, the data volume is increased rapidly, the data types are various, the data updating speed is high, the large-scale task processing capability in a large data environment is weak, and the requirement for fast processing of data is increasingly difficult to meet.
Disclosure of Invention
The present invention is proposed in view of the above problems, so as to provide a BIM data processing method and apparatus for power transmission and transformation project site layout standardization.
In one embodiment of the invention, a BIM data processing method for power transmission and transformation project site arrangement standardization is provided, and is characterized by comprising the following steps:
BIM data stream input data, and writing the data into a memory after initialization; establishing a plurality of task queues for parallel processing, wherein each task queue is provided with a plurality of nodes including non-leaf nodes and leaf nodes;
judging whether the load of the BIM data exceeds the upper limit of a system, if so, reducing the BIM data, and performing feature selection, feature extraction, instance reduction and data discretization on the data of the BIM data stream;
initializing the size of a sliding window so as to distribute the video memory of the GPU;
initializing a control kernel of a calculation algorithm, and copying the BIM data in a memory to a video memory of the GPU by using a block copy technology;
initializing and distributing a GPU (graphics processing Unit) video memory for storing a summary data structure, calling a kernel program of an execution equipment end to extract the summary data structure and perform parallel computation, and storing the result in a video memory area for storing the summary data structure;
initializing and allocating a GPU (graphics processing Unit) video memory for storing output results, calling a kernel program of an execution equipment end to calculate a data stream mining algorithm in parallel, and storing processing results in an output result video memory area;
initializing and allocating a CPU memory for storing the exchange output result of the GPU;
copying a result in the GPU video memory to a GPU memory by using a block copying technology;
and the CPU formats and visualizes the output result.
In another embodiment of the present invention, a BIM data processing apparatus standardized for power transmission and transformation project site layout is characterized by having a server side, and the server side executes the following method:
BIM data stream input data, and writing the data into a memory after initialization; establishing a plurality of task queues for parallel processing, wherein each task queue is provided with a plurality of nodes including non-leaf nodes and leaf nodes;
judging whether the load of the BIM data exceeds the upper limit of a system, if so, reducing the BIM data, and performing feature selection, feature extraction, instance reduction and data discretization on the data of the BIM data stream;
initializing the size of a sliding window so as to distribute the video memory of the GPU;
initializing a control kernel of a calculation algorithm, and copying the BIM data in a memory to a video memory of the GPU by using a block copy technology;
initializing and distributing a GPU (graphics processing Unit) video memory for storing a summary data structure, calling a kernel program of an execution equipment end to extract the summary data structure and perform parallel computation, and storing the result in a video memory area for storing the summary data structure;
initializing and allocating a GPU (graphics processing Unit) video memory for storing output results, calling a kernel program of an execution equipment end to calculate a data stream mining algorithm in parallel, and storing processing results in an output result video memory area;
initializing and allocating a CPU memory for storing the exchange output result of the GPU;
copying a result in the GPU video memory to a GPU memory by using a block copying technology;
and the CPU formats and visualizes the output result.
The embodiment of the invention provides a BIM data processing method and a device aiming at power transmission and transformation project site arrangement standardization. The technical scheme provided by the embodiment of the invention can effectively solve the technical problem of slow data processing caused by rapid increase of data volume, various data types and high data updating speed when the BIM data processing with standardized arrangement in the power transmission and transformation project site is carried out.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a BIM data processing method for power transmission and transformation project site layout standardization according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating an embodiment of BIM data stream input data;
fig. 3 is a schematic structural diagram of a BIM data processing apparatus standardized for power transmission and transformation project site layout according to an embodiment of the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the examples of this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and "a" and "an" typically include at least two, but do not exclude the presence of at least one.
It should be understood that although the terms first, second, third, etc. may be used to describe XXX in the embodiments of the present application, these XXX should not be limited to these terms. These terms are only used to distinguish XXX from each other. For example, a first XXX may also be referred to as a second XXX, and similarly, a second XXX may also be referred to as a first XXX, without departing from the scope of embodiments of the present application.
The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a monitoring", depending on the context. Similarly, the phrase "if it is determined" or "if it is monitored (a stated condition or event)" may be interpreted as "when determining" or "in response to determining" or "when monitoring (a stated condition or event)" or "in response to monitoring (a stated condition or event)", depending on the context.
It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a commodity or system that includes the element.
More than half of data in the construction process of power transmission and transformation projects are generated in the design stage, and errors in the design stage are amplified continuously along with the progress of projects. In addition, the optimization design is involved in every engineering construction applied to the BIM technology, and the design optimization of the substation must be realized by means of data generated in the design stage. Therefore, the accuracy, uniqueness, reliability and data circulation integrity of the data in the construction and design stage of the transformer substation are very important for the smooth implementation of the whole engineering construction.
The design of power transmission and transformation projects involves a plurality of specialties, including buildings, structures, electrics, heating ventilation air conditioners, rooms and the like, and each specialty is responsible for completing a certain part of the design task. In the traditional substation design process, the design work of different specialties cannot be performed simultaneously, for example, the structure speciality can start to design only after the design of the building speciality is completed. The method prolongs the design period of the project to a certain extent, and greatly reduces the working efficiency in the project design process. And the working idea of the cooperation of the BIM technology changes the design process, and the design work of different specialties can be performed in a crossed or parallel mode.
Fig. 1 is a BIM data processing method for power transmission and transformation project site layout standardization according to an embodiment of the present invention, and the method includes:
BIM data stream input data, and writing the data into a memory after initialization; establishing a plurality of task queues for parallel processing, wherein each task queue is provided with a plurality of nodes including non-leaf nodes and leaf nodes;
judging whether the load of the BIM data exceeds the upper limit of a system, if so, reducing the BIM data, and performing feature selection, feature extraction, instance reduction and data discretization on the data of the BIM data stream;
data reduction aims to reduce the complexity of data and mainly comprises feature selection, feature extraction, instance reduction and data discretization. Assume that there are P data in the dataset, each data consisting of N feature elements. Feature selection is also called attribute selection, and refers to selecting M features from the existing N features, that is, deleting the rest (N-M) features, so that the dimension reduction of the data is realized under the condition that the original data characteristics are ensured as much as possible, and conditions are provided for efficient processing of the data. Feature extraction is similar to feature selection, both of which aim at data dimension reduction, but the feature extraction does not select M features from the original N features, but utilizes an algorithm to process the N features to generate M brand new features to replace the original N features. Example reduction aims to reduce the size of data, and Q data are selected for data analysis processing in a data set consisting of P data. Efficient processing of data is facilitated by reducing the size of the original data set as far as possible without destroying its characteristics. In some data application scenarios, only the relative relationship between data elements is utilized and is irrelevant to the specific values of the data elements, and the magnitude of the values of the data elements is large, so that the data elements can be compressed to reduce the complexity of processing large data, and the original data is replaced by simple values and only the relative relationship is reserved, namely, the data is discretized.
Initializing the size of a sliding window so as to distribute the video memory of the GPU;
the GPU is an SIMT data parallel computing structure, is not good at controlling data buffering, task scheduling, query instruction analysis and the like, can be completed on the CPU, and is very suitable for data intensive and computation intensive parallel computing tasks. According to the characteristics, from the view point of macro coarse granularity, a multithreading technology can be adopted at a host computer end, and two-stage parallel of tasks is realized, namely multithreading is respectively started on a CPU host computer, and different threads can call and start threads on different GPU equipment, so that parallel calculation can be simultaneously carried out on different tasks. For example, buffering of multi-stream data may be performed in parallel in multiple threads in the CPU, and the GPU may calculate summary data structures of data streams in parallel, perform data stream mining algorithms, and so on. This computational structure is very advantageous when the data stream forming a GPU handles coarse-grained collaboration mode, especially for large data sets of high-dimensional multi-streams.
Initializing a control kernel of a calculation algorithm, and copying BIM data in a memory to a video memory of a GPU by using a block copy technology;
initializing and distributing a GPU (graphics processing Unit) video memory for storing a summary data structure, calling a kernel program of an execution equipment end to extract the summary data structure and perform parallel computation, and storing the result in a video memory area for storing the summary data structure;
initializing and allocating a GPU (graphics processing Unit) video memory for storing output results, calling a kernel program of an execution equipment end to calculate a data stream mining algorithm in parallel, and storing processing results in an output result video memory area;
initializing and allocating a CPU memory for storing the exchange output result of the GPU;
copying a result in the GPU video memory to a GPU memory by using a block copying technology;
and the CPU formats and visualizes the output result.
Fig. 2 is a schematic flow chart of BIM data stream input data according to an embodiment of the present invention, where the BIM data stream input data step is:
reading a BIM data stream, writing the BIM data into a buffer area, and establishing a corresponding index;
when the buffer window is filled with the BIM data, the BIM data is exchanged and transmitted to the GPU;
initializing and operating a summary data processing kernel function of a buffer window layer, and calculating and caching a summary data result of a basic child window in a buffer window;
initializing and operating a summary data processing kernel function of a sliding window layer, calculating and caching a summary data result of the sliding window, and updating a statistical data result in an incremental manner;
checking the expired and failed basic sub-window and deleting the expired and failed basic sub-window, and simultaneously storing the latest basic sub-window data into the corresponding position of the memory area;
initializing a parallel kernel function for operating the correlation coefficients of the multiple data streams, and performing parallel calculation on every two correlation coefficients in the BIM data streams based on a multiple data stream summary data matrix in a GPU memory;
running a kernel function of quantile calculation to obtain the overall distribution condition of all correlation coefficients;
running a query kernel function of a correlation coefficient threshold value to obtain BIM data stream pairing information higher than the threshold value;
running a kernel function of an output result, and outputting all paired BIM data streams and a related coefficient set which meet requirements;
returning the result to the CPU;
and continuously scanning the BIM data stream, updating the sliding window in real time until all the data is processed, and quitting from the calculation.
Further, the task queue execution method comprises the following steps:
judging whether the task queue is empty, if so, finishing the algorithm, otherwise, executing the next step;
all nodes calculate the predicted execution time E for executing the next task;
all the non-leaf nodes in the tree detect the bandwidth of each line connected with the child node of the non-leaf node;
all the non-leaf nodes in the tree send instructions to all the child nodes of the non-leaf nodes to obtain the predicted execution time E of the child nodes;
after receiving the instruction, the leaf node returns the predicted execution time E to the father node;
starting from the non-leaf node at the bottommost layer, when the non-leaf node receives the feedback results of all the child nodes, returning a result set to a parent node of the non-leaf node together with the bandwidth of the transmission lines of the non-leaf node and all the child nodes;
recursively calling the previous step until the master node receives the result sets returned by all the son nodes;
the main node calculates coefficients C of all nodes;
and taking one task out of the task queue, distributing the task to the node with the minimum limiting coefficient, and then recycling the steps.
Further, the calculating the coefficient C of all nodes by the master node includes: setting the coefficient C when the S node calculates the k +1 unit tasks(k+1)；
Then
Wherein a and B are the weights of E and T respectively, n is the total node number, E is the predicted execution time of processing a certain unit task by the node s, and T is the predicted transmission time of transmitting the (k + 1) th unit task from the node A to the node B.
The above embodiments provide a BIM data processing method for power transmission and transformation project site layout standardization, which includes establishing a plurality of task queues to perform parallel processing, determining whether a BIM data load exceeds an upper limit of a system, reducing the BIM data, transferring the BIM data from a CPU to a GPU to perform data processing, copying the BIM data in a memory to a video memory of the GPU by using a block copy technology, calling a kernel program of an execution device end to perform summary data structure extraction and parallel computation, calling the kernel program of the execution device end to perform parallel computation on a data stream mining algorithm, and finally returning a CPU output result from the GPU. The technical scheme provided by the embodiment of the invention can effectively solve the technical problem of slow data processing caused by rapid increase of data volume, various data types and high data updating speed when the BIM data processing with standardized arrangement in the power transmission and transformation project site is carried out.
Another embodiment, as shown in fig. 3, provides a BIM data processing apparatus standardized for power transmission and transformation project site layout, having a server side, where the server side performs the following method: BIM data stream input data, and writing the data into a memory after initialization; establishing a plurality of task queues for parallel processing, wherein each task queue is provided with a plurality of nodes including non-leaf nodes and leaf nodes;
judging whether the load of the BIM data exceeds the upper limit of a system, if so, reducing the BIM data, and performing feature selection, feature extraction, instance reduction and data discretization on the data of the BIM data stream;
data reduction aims to reduce the complexity of data and mainly comprises feature selection, feature extraction, instance reduction and data discretization. Assume that there are P data in the dataset, each data consisting of N feature elements. Feature selection is also called attribute selection, and refers to selecting M features from the existing N features, that is, deleting the rest (N-M) features, so that the dimension reduction of the data is realized under the condition that the original data characteristics are ensured as much as possible, and conditions are provided for efficient processing of the data. Feature extraction is similar to feature selection, both of which aim at data dimension reduction, but the feature extraction does not select M features from the original N features, but utilizes an algorithm to process the N features to generate M brand new features to replace the original N features. Example reduction aims to reduce the size of data, and Q data are selected for data analysis processing in a data set consisting of P data. Efficient processing of data is facilitated by reducing the size of the original data set as far as possible without destroying its characteristics. In some data application scenarios, only the relative relationship between data elements is utilized and is irrelevant to the specific values of the data elements, and the magnitude of the values of the data elements is large, so that the data elements can be compressed to reduce the complexity of processing large data, and the original data is replaced by simple values and only the relative relationship is reserved, namely, the data is discretized.
Initializing the size of a sliding window so as to distribute the video memory of the GPU;
the GPU is an SIMT data parallel computing structure, is not good at controlling data buffering, task scheduling, query instruction analysis and the like, can be completed on the CPU, and is very suitable for data intensive and computation intensive parallel computing tasks. According to the characteristics, from the view point of macro coarse granularity, a multithreading technology can be adopted at a host computer end, and two-stage parallel of tasks is realized, namely multithreading is respectively started on a CPU host computer, and different threads can call and start threads on different GPU equipment, so that parallel calculation can be simultaneously carried out on different tasks. For example, buffering of multi-stream data may be performed in parallel in multiple threads in the CPU, and the GPU may calculate summary data structures of data streams in parallel, perform data stream mining algorithms, and so on. This computational structure is very advantageous when the data stream forming a GPU handles coarse-grained collaboration mode, especially for large data sets of high-dimensional multi-streams.
Initializing a control kernel of a calculation algorithm, and copying BIM data in a memory to a video memory of a GPU by using a block copy technology;
initializing and distributing a GPU (graphics processing Unit) video memory for storing a summary data structure, calling a kernel program of an execution equipment end to extract the summary data structure and perform parallel computation, and storing the result in a video memory area for storing the summary data structure;
initializing and allocating a GPU (graphics processing Unit) video memory for storing output results, calling a kernel program of an execution equipment end to calculate a data stream mining algorithm in parallel, and storing processing results in an output result video memory area;
initializing and allocating a CPU memory for storing the exchange output result of the GPU;
copying a result in the GPU video memory to a GPU memory by using a block copying technology;
and the CPU formats and visualizes the output result.
Further, the step of inputting data into the BIM data stream is as follows:
reading a BIM data stream, writing the BIM data into a buffer area, and establishing a corresponding index;
when the buffer window is filled with the BIM data, the BIM data is exchanged and transmitted to the GPU;
initializing and operating a summary data processing kernel function of a buffer window layer, and calculating and caching a summary data result of a basic child window in a buffer window;
initializing and operating a summary data processing kernel function of a sliding window layer, calculating and caching a summary data result of the sliding window, and updating a statistical data result in an incremental manner;
checking the expired and failed basic sub-window and deleting the expired and failed basic sub-window, and simultaneously storing the latest basic sub-window data into the corresponding position of the memory area;
initializing a parallel kernel function for operating the correlation coefficients of the multiple data streams, and performing parallel calculation on every two correlation coefficients in the BIM data streams based on a multiple data stream summary data matrix in a GPU memory;
running a kernel function of quantile calculation to obtain the overall distribution condition of all correlation coefficients;
running a query kernel function of a correlation coefficient threshold value to obtain BIM data stream pairing information higher than the threshold value;
running a kernel function of an output result, and outputting all paired BIM data streams and a related coefficient set which meet requirements;
returning the result to the CPU;
and continuously scanning the BIM data stream, updating the sliding window in real time until all the data is processed, and quitting from the calculation.
Further, the task queue execution method comprises the following steps:
judging whether the task queue is empty, if so, finishing the algorithm, otherwise, executing the next step;
all nodes calculate the predicted execution time E for executing the next task;
all the non-leaf nodes in the tree detect the bandwidth of each line connected with the child node of the non-leaf node;
all the non-leaf nodes in the tree send instructions to all the child nodes of the non-leaf nodes to obtain the predicted execution time E of the child nodes;
after receiving the instruction, the leaf node returns the predicted execution time E to the father node;
starting from the non-leaf node at the bottommost layer, when the non-leaf node receives the feedback results of all the child nodes, returning a result set to a parent node of the non-leaf node together with the bandwidth of the transmission lines of the non-leaf node and all the child nodes;
recursively calling the previous step until the master node receives the result sets returned by all the son nodes;
the main node calculates coefficients C of all nodes;
and taking one task out of the task queue, distributing the task to the node with the minimum limiting coefficient, and then recycling the steps.
Further, the calculating the coefficient C of all nodes by the master node includes: setting the coefficient C when the S node calculates the k +1 unit tasks(k+1)；
Then
Wherein a and B are the weights of E and T respectively, n is the total node number, E is the predicted execution time of processing a certain unit task by the node s, and T is the predicted transmission time of transmitting the (k + 1) th unit task from the node A to the node B.
The above embodiments provide a standardized BIM data processing apparatus for power transmission and transformation project site arrangement, which performs parallel processing by establishing a plurality of task queues, determines whether a BIM data load exceeds an upper limit of a system, reduces the BIM data, transfers the BIM data from a CPU to a GPU for data processing, copies the BIM data in a memory to a video memory of the GPU by using a block copy technology, calls a kernel program of an execution device end to perform summary data structure extraction and parallel computation, calls the kernel program of the execution device end to perform parallel computation on a data stream mining algorithm, and finally returns a CPU output result from the GPU. The technical scheme provided by the embodiment of the invention can effectively solve the technical problem of slow data processing caused by rapid increase of data volume, various data types and high data updating speed when the BIM data processing with standardized arrangement in the power transmission and transformation project site is carried out.
It is also noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a good or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such good or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a commodity or system that includes the element.
The foregoing description shows and describes several preferred embodiments of the invention, but as aforementioned, it is to be understood that the invention is not limited to the forms disclosed herein, but is not to be construed as excluding other embodiments and is capable of use in various other combinations, modifications, and environments and is capable of changes within the scope of the inventive concept as expressed herein, commensurate with the above teachings, or the skill or knowledge of the relevant art. And that modifications and variations may be effected by those skilled in the art without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (5)
1. A BIM data processing method for power transmission and transformation project site layout standardization, which is characterized by comprising the following steps:
BIM data stream input data, and writing the data into a memory after initialization; establishing a plurality of task queues for parallel processing, wherein each task queue is provided with a plurality of nodes including non-leaf nodes and leaf nodes;
judging whether the load of the BIM data exceeds the upper limit of a system, if so, reducing the BIM data, and performing feature selection, feature extraction, instance reduction and data discretization on the data of the BIM data stream;
initializing the size of a sliding window so as to distribute the video memory of the GPU;
initializing a control kernel of a calculation algorithm, and copying the BIM data in a memory to a video memory of the GPU by using a block copy technology;
initializing and distributing a GPU (graphics processing Unit) video memory for storing a summary data structure, calling a kernel program of an execution equipment end to extract the summary data structure and perform parallel computation, and storing the result in a video memory area for storing the summary data structure;
initializing and allocating a GPU (graphics processing Unit) video memory for storing output results, calling a kernel program of an execution equipment end to calculate a data stream mining algorithm in parallel, and storing processing results in an output result video memory area;
initializing and allocating a CPU memory for storing the exchange output result of the GPU;
copying a result in the GPU video memory to a GPU memory by using a block copying technology;
and the CPU formats and visualizes the output result.
2. The method of claim 1, further comprising the step of inputting data into the BIM data stream by:
reading the BIM data stream, writing the BIM data into a buffer area, and establishing a corresponding index;
transmitting the BIM data exchange to the GPU after a buffer window is filled with the BIM data;
initializing and operating a summary data processing kernel function of a buffer window layer, and calculating and caching a summary data result of a basic child window in a buffer window;
initializing and operating a summary data processing kernel function of a sliding window layer, calculating and caching a summary data result of the sliding window, and updating a statistical data result in an incremental manner;
checking the expired and failed basic sub-window and deleting the expired and failed basic sub-window, and simultaneously storing the latest basic sub-window data into the corresponding position of the memory area;
initializing a parallel kernel function for operating the correlation coefficients of multiple data streams, and performing parallel calculation on every two correlation coefficients in the BIM data streams based on a multiple data stream summary data matrix in the GPU memory;
running a kernel function of quantile calculation to obtain the overall distribution condition of all correlation coefficients;
running a query kernel function of a correlation coefficient threshold value to obtain BIM data stream pairing information higher than the threshold value;
running a kernel function of an output result, and outputting all paired BIM data streams and a related coefficient set which meet requirements;
returning a result to the CPU;
and continuously scanning the BIM data stream, updating the sliding window in real time until all the data streams are processed, and quitting from the calculation.
3. The method of claim 1, further wherein the task queue performing method comprises:
judging whether the task queue is empty, if so, finishing the algorithm, otherwise, executing the next step;
all nodes calculate the predicted execution time E for executing the next task;
all the non-leaf nodes in the tree detect the bandwidth of each line connected with the child node of the non-leaf node;
all the non-leaf nodes in the tree send instructions to all the child nodes of the non-leaf nodes to obtain the predicted execution time E of the child nodes;
after receiving the instruction, the leaf node returns the predicted execution time E to the father node;
starting from the non-leaf node at the bottommost layer, when the non-leaf node receives the feedback results of all the child nodes, returning a result set to a parent node of the non-leaf node together with the bandwidth of the transmission lines of the non-leaf node and all the child nodes;
recursively calling the previous step until the master node receives the result sets returned by all the son nodes;
the main node calculates coefficients C of all nodes;
and taking one task out of the task queue, distributing the task to the node with the minimum limiting coefficient, and then recycling the steps.
4. The method of claim 3, further wherein the master node computing coefficients C for all nodes comprises: setting the coefficient C when the S node calculates the k +1 unit tasks(k+1)；
Then
Wherein a and B are the weights of E and T respectively, n is the total node number, E is the predicted execution time of processing a certain unit task by the node s, and T is the predicted transmission time of transmitting the (k + 1) th unit task from the node A to the node B.
5. A BIM data processing device for standardization of power transmission and transformation project site arrangement is characterized by comprising a server side, wherein the server side executes the following method:
BIM data stream input data, and writing the data into a memory after initialization; establishing a plurality of task queues for parallel processing, wherein each task queue is provided with a plurality of nodes including non-leaf nodes and leaf nodes;
judging whether the load of the BIM data exceeds the upper limit of a system, if so, reducing the BIM data, and performing feature selection, feature extraction, instance reduction and data discretization on the data of the BIM data stream;
initializing the size of a sliding window so as to distribute the video memory of the GPU;
initializing a control kernel of a calculation algorithm, and copying the BIM data in a memory to a video memory of the GPU by using a block copy technology;
initializing and distributing a GPU (graphics processing Unit) video memory for storing a summary data structure, calling a kernel program of an execution equipment end to extract the summary data structure and perform parallel computation, and storing the result in a video memory area for storing the summary data structure;
initializing and allocating a GPU (graphics processing Unit) video memory for storing output results, calling a kernel program of an execution equipment end to calculate a data stream mining algorithm in parallel, and storing processing results in an output result video memory area;
initializing and allocating a CPU memory for storing the exchange output result of the GPU;
copying a result in the GPU video memory to a GPU memory by using a block copying technology;
and the CPU formats and visualizes the output result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911164702.2A CN111158898B (en) | 2019-11-25 | 2019-11-25 | BIM data processing method and device aiming at power transmission and transformation project site arrangement standardization |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911164702.2A CN111158898B (en) | 2019-11-25 | 2019-11-25 | BIM data processing method and device aiming at power transmission and transformation project site arrangement standardization |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111158898A true CN111158898A (en) | 2020-05-15 |
| CN111158898B CN111158898B (en) | 2022-07-15 |
Family
ID=70556094
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911164702.2A Active CN111158898B (en) | 2019-11-25 | 2019-11-25 | BIM data processing method and device aiming at power transmission and transformation project site arrangement standardization |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111158898B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461406A (en) * | 2022-04-13 | 2022-05-10 | 麒麟软件有限公司 | DMA OpenGL optimization method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104391679A (en) * | 2014-11-18 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | GPU (graphics processing unit) processing method for high-dimensional data stream in irregular stream |
| CN104965689A (en) * | 2015-05-22 | 2015-10-07 | 浪潮电子信息产业股份有限公司 | Hybrid parallel computing method and device for CPUs/GPUs |
| US20170235671A1 (en) * | 2016-02-15 | 2017-08-17 | MemRay Corporation | Computing device, data transfer method between coprocessor and non-volatile memory, and computer-readable recording medium |
| CN107124286A (en) * | 2016-02-24 | 2017-09-01 | 深圳市知穹科技有限公司 | A kind of mass data high speed processing, the system and method for interaction |
| CN107341507A (en) * | 2017-06-20 | 2017-11-10 | 华中科技大学 | A kind of rapid image SIFT feature matching process based on GPU with cascade Hash |
| CN109213793A (en) * | 2018-08-07 | 2019-01-15 | 泾县麦蓝网络技术服务有限公司 | A kind of stream data processing method and system |
-
2019
- 2019-11-25 CN CN201911164702.2A patent/CN111158898B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104391679A (en) * | 2014-11-18 | 2015-03-04 | 浪潮电子信息产业股份有限公司 | GPU (graphics processing unit) processing method for high-dimensional data stream in irregular stream |
| CN104965689A (en) * | 2015-05-22 | 2015-10-07 | 浪潮电子信息产业股份有限公司 | Hybrid parallel computing method and device for CPUs/GPUs |
| US20170235671A1 (en) * | 2016-02-15 | 2017-08-17 | MemRay Corporation | Computing device, data transfer method between coprocessor and non-volatile memory, and computer-readable recording medium |
| CN107124286A (en) * | 2016-02-24 | 2017-09-01 | 深圳市知穹科技有限公司 | A kind of mass data high speed processing, the system and method for interaction |
| CN107341507A (en) * | 2017-06-20 | 2017-11-10 | 华中科技大学 | A kind of rapid image SIFT feature matching process based on GPU with cascade Hash |
| CN109213793A (en) * | 2018-08-07 | 2019-01-15 | 泾县麦蓝网络技术服务有限公司 | A kind of stream data processing method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114461406A (en) * | 2022-04-13 | 2022-05-10 | 麒麟软件有限公司 | DMA OpenGL optimization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111158898B (en) | 2022-07-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Johansen | Approximate explicit receding horizon control of constrained nonlinear systems | |
| CN108089921B (en) | Server for cloud big data operation architecture and operation resource optimization method thereof | |
| CN116882038B (en) | Electromechanical construction method and system based on BIM technology | |
| US20180329740A1 (en) | System and method of schedule validation and optimization of machine learning flows for cloud computing | |
| JP2010140357A (en) | Stream data processing method, and system | |
| CN107526645A (en) | A kind of communication optimization method and system | |
| JP5245711B2 (en) | Distributed data processing system, distributed data processing method, and distributed data processing program | |
| CN114862656A (en) | Method for acquiring training cost of distributed deep learning model based on multiple GPUs | |
| KR102613367B1 (en) | Method and apparatus for automatically reducing model weight for deep learning model serving optimization, and a method for providing cloud inference services usin the same | |
| CN111752691A (en) | AI (artificial intelligence) calculation graph sorting method, device, equipment and storage medium | |
| CN111158898B (en) | BIM data processing method and device aiming at power transmission and transformation project site arrangement standardization | |
| CN109783033B (en) | Data storage method and electronic equipment suitable for heterogeneous system | |
| CN116361567B (en) | Data processing method and system applied to cloud office | |
| CN115186738B (en) | Model training method, device and storage medium | |
| CN113504966B (en) | GPU cluster scheduling strategy simulation method and GPU cluster simulator | |
| JP6142878B2 (en) | Information system performance evaluation apparatus, method and program | |
| JP4322094B2 (en) | Information processing method, service time deriving method, and processing unit number adjusting method | |
| Wang et al. | An adaptive elasticity policy for staging based in-situ processing | |
| CN112596901A (en) | Cloud platform automation deployment and operation method, electronic equipment and storage medium | |
| CN113821313A (en) | Task scheduling method and device and electronic equipment | |
| US10970430B2 (en) | Computer-readable recording medium, computing machine resource allocation method, and particle simulation apparatus | |
| US11209784B1 (en) | Smart grid controller | |
| CN117667606B (en) | High-performance computing cluster energy consumption prediction method and system based on user behaviors | |
| US10969758B1 (en) | Smart grid controller | |
| KR102613366B1 (en) | Artificial intelligence-based cloud service device and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |