CN115373850A - Residual curve data filtering and screening method and related equipment - Google Patents
Residual curve data filtering and screening method and related equipment Download PDFInfo
- Publication number
- CN115373850A CN115373850A CN202211058837.2A CN202211058837A CN115373850A CN 115373850 A CN115373850 A CN 115373850A CN 202211058837 A CN202211058837 A CN 202211058837A CN 115373850 A CN115373850 A CN 115373850A
- Authority
- CN
- China
- Prior art keywords
- data
- filtering
- residual
- screening
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001914 filtration Methods 0.000 title claims abstract description 43
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012216 screening Methods 0.000 title claims abstract description 42
- 230000000903 blocking effect Effects 0.000 claims abstract description 6
- 238000004590 computer program Methods 0.000 claims description 12
- 238000000638 solvent extraction Methods 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 abstract description 21
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 7
- 238000004088 simulation Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a residual error curve data filtering and screening method and related equipment, wherein the method comprises the following steps: (1) Data filtering, namely separating residual data from the obtained data; (2) Screening data, namely screening various residual error values in the iteration step range according to the iteration step range aiming at the separated residual error data; (3) Data blocking is carried out according to the screened total number of the iterative steps; (4) Calculating the value of the data block, and regarding each determined data block, taking the iteration step of the first data point of each data block as the horizontal coordinate value of the data block, and taking the point with the maximum absolute value in each data block as the vertical coordinate value of the data block; (5) And (4) returning the result, and returning the numerical values of all the data blocks based on the horizontal and vertical coordinates determined in the step (4). The scheme provided by the invention can realize the simplification of data generated by calculation of some common simulation software, and can ensure that the simplified curve and the original curve have the same shape.
Description
Technical Field
The invention relates to an industrial simulation calculation technology, in particular to a residual error curve data filtering technology.
Background
In the field of industrial simulation calculation, many calculation examples have large scale and long calculation time, and engineers usually observe intermediate results of calculation in the calculation process to determine whether the calculation deviates from a preset target, so as to avoid wasting time and calculation resources.
The residual curve is an effective indicator of the deviation between the observed calculated value and the estimated value. However, because the scales of many examples are large, the residual data amount generated in the calculation process is also large, and is limited by the limitations of resources such as memory, CPU and the like, so that many times, such data cannot be completely rendered, and even if the data can be rendered, a large amount of time is often consumed, thereby affecting the user experience.
Therefore, how to effectively filter and screen residual error curve data, improve the industrial simulation calculation efficiency and save the calculation resources is a problem to be solved urgently in the field.
Disclosure of Invention
Aiming at the problem of large computing time and resource consumption in the existing industrial simulation computing, the invention aims to provide a residual error curve data filtering and screening scheme, so that the computing data volume is reduced, and a large amount of time and resources are saved.
In order to achieve the above object, a first aspect of the present invention provides a residual curve data filtering and screening method, including:
(1) The data is filtered, and the data is filtered,
separating residual data from the acquired data;
(2) The data is screened out, and the data is filtered out,
screening out various residual error values in the iteration step range according to the iteration step range aiming at the separated residual error data;
(3) The data is divided into blocks, and the data is divided into blocks,
partitioning according to the total amount of data to form a plurality of data blocks to be processed;
(4) The value of the data block is calculated,
regarding each data block determined in the step (3), taking the iteration step of the first data point of each data block as the abscissa value of the data block, and taking the point with the maximum absolute value in each data block as the ordinate value of the data block;
(5) The result is returned back to the server in the form of a file,
and (4) returning the numerical values of all the data blocks based on the horizontal and vertical coordinates determined in the step (4).
Further, different data filtering modes are matched for different output data in the step (1).
Further, when the data is filtered in step (2), firstly, the processing range of the original data needs to be determined, and then the required data is filtered based on the determined processing range of the original data.
Furthermore, when the data is blocked in the step (3),
if the screened total number N of the iterative steps is less than or equal to N, dividing the data into N data blocks, wherein each block comprises data of one iterative step;
if the screened total number N > of the iterative steps needs to be divided into N blocks, dividing the data into N data blocks, wherein the mth data block contains N m And (5) iteration step data.
In order to achieve the above object, a second aspect of the present invention provides a computer-readable storage medium having a program stored thereon, where the program is executed by a processor to implement the steps of the above residual curve data filtering and screening method.
In order to achieve the above object, a third aspect of the present invention provides a processor for executing a program, where the program is executed to perform the steps of the above residual curve data filtering and screening method.
In order to achieve the above object, a fourth aspect of the present invention provides a terminal device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, where the program code is loaded and executed by the processor to implement the steps of the above residual curve data filtering and screening method.
In order to achieve the above object, a fifth aspect of the present invention provides a computer program product adapted to perform the steps of the above residual curve data filtering and screening method when executed on a data processing apparatus.
The residual curve data filtering and screening scheme provided by the invention can realize the simplification of data generated by calculation of some common simulation software, and can ensure that the simplified curve has the same shape as the original curve.
The residual curve data filtering and screening scheme provided by the invention greatly reduces the data volume after simplifying the original data generated by the simulation software calculation compared with the original data, and can save a large amount of time and resources when a curve is drawn on equipment which needs network transmission or has a tense memory.
Drawings
The invention is further described below in conjunction with the appended drawings and the detailed description.
Fig. 1 is a process flow diagram of a residual curve data filtering and screening method according to an embodiment of the present invention.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further explained below by combining the specific drawings.
Aiming at the problems of low calculation efficiency and high resource consumption caused by large calculation data amount in the existing industrial simulation calculation process, the invention provides a residual curve data filtering and screening scheme to simplify the original data generated by the simulation software, so that the calculation data amount can be reduced in reply, the calculation efficiency is improved, and simultaneously the simplified curve and the original curve can be ensured to have the same shape.
Accordingly, the residual curve data filtering and screening scheme provided by the invention is mainly realized by the mutual cooperation of 5 functional steps of data filtering, data screening, data blocking, data block value calculation and result return.
Referring to fig. 1, the specific processing flow of the residual curve data filtering and screening scheme is as follows:
step (1): filtering data;
original data generated by simulation software calculation is filtered, and residual data is separated out.
Step (2): screening data;
and (2) screening out various residual error numerical values in the iteration step range according to the iteration step range aiming at the residual error data separated in the step (1).
And (3): data is partitioned;
and partitioning according to the total amount of the data to form a plurality of data blocks to be processed.
And (4): calculating the value of the data block;
and (4) regarding each data block determined in the step (3), taking the iteration step of the first data point of each data block as the abscissa value of the data block, and taking the point with the maximum absolute value in each data block as the ordinate value of the data block.
And (5): returning a numerical value;
and (4) returning the numerical values of all the data blocks based on the horizontal and vertical coordinates determined in the step (4) for drawing a residual error curve.
Specific embodiments of the residual curve data filtering and screening scheme provided by the present invention are further described below.
Specifically, when the residual curve data filtering and screening scheme is implemented, in the step (1) of the scheme, a corresponding data filtering algorithm is constructed in advance according to different software and is stored in a corresponding database in advance aiming at the characteristic that data output by different software has a large difference when data filtering is performed.
According to the scheme, for the acquired data, firstly, the data is analyzed to determine the software characteristic information for outputting the data, the corresponding data filtering algorithm is called from the database based on the determined software characteristic information, and finally, the acquired data is filtered through the called filtering algorithm to separate out corresponding residual data.
In the scheme, when data screening is performed on residual data separated in the step (1) in the step (2), an iteration step range is determined at first.
By way of example, where an iterative step range is determined, it may be supported that the range of iterative steps is specified, and if not, all iterative steps are defaulted.
Then, screening out various residual error values in the range based on the determined iteration walking range.
Furthermore, when the data is screened, the processing range of the original data needs to be determined first, and then the required data is screened based on the determined processing range of the original data. For example, if the software currently calculates 10000 iteration step data in total, if the user only needs to check the data of 2000 to 3000 iteration steps, the user can screen out various residual error values in the range from the processed original data based on the iteration step range (2000 to 3000 iteration steps) selected by the user, and the subsequent processing is performed based on the data.
In order to control the total amount of the processed data, the scheme firstly carries out blocking on the data through the step (3). By way of example, if the total amount of data is at most 500 (the maximum value is determined by combining the network environment and the front-end rendering capability), the processed data is at most 500 groups, that is, at most 500 data blocks are divided.
On this basis, when data blocking is performed in step (3) in this scheme, it is preferable that data blocking be performed as follows.
The total number of screened iterative steps is set to be N, and the iterative steps are divided into N blocks (default N = 500).
1. If N is less than or equal to N, dividing the data into N data blocks, wherein each block contains data of one iteration step.
2. If N is present>N, divided into N data blocks, the mth data block containing N m Iteration step data.
Further, for the case of incomplete division, the number of iteration steps of each actual data block is:
after the block division, N iterative steps are basically and uniformly distributed into each data block, so that the value calculated in the subsequent step can reflect the original residual variation trend as accurately as possible.
In the scheme, when the data block value is calculated in the step (4), the iteration step of the first data point of each data block is taken as the value of the abscissa of the data block, and the point with the maximum absolute value in each data block is taken as the value of the ordinate of the data block.
According to the scheme, each data block corresponds to a residual value of one coordinate point on the result, and the value of each data block is calculated based on the mode, so that the result data can reflect the change trend of the original residual error as accurately as possible, the wave peak value on the residual error curve can be effectively reflected, and a user can conveniently and quickly know the extreme value in the residual error data.
When the formed residual curve data filtering and screening scheme is implemented, original data generated by industrial simulation calculation can be effectively simplified, and the data volume can be greatly reduced compared with the original data, so that the calculation efficiency can be effectively improved, and the simplified curve and the original curve can be ensured to have the same shape.
When the scheme provided by the invention is applied specifically, a corresponding software program can be formed to form a corresponding system for filtering and screening the residual curve data. When the software program runs, the residual curve data filtering and screening method is executed, and the residual curve data filtering and screening method is stored in a corresponding storage medium for being called and executed by a processor.
The embodiment of the invention also provides a computer readable storage medium, on which a program is stored, and the program, when executed by a processor, implements the steps of the residual curve data filtering and screening method.
The embodiment of the invention also provides a processor, which is used for running a program, wherein the step of the residual error curve data filtering and screening method is executed when the program runs.
The embodiment of the invention also provides terminal equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein the program code is loaded and executed by the processor to realize the steps of the residual curve data filtering and screening method.
The invention also provides a computer program product adapted to perform the steps of the above-described residual curve data filtering screening method when executed on a data processing apparatus.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the system, the apparatus and the module described above may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both permanent and non-permanent, removable and non-removable media, may implement the information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Meanwhile, it should be understood that portions not described in detail in this specification belong to the prior art.
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (8)
1. A residual curve data filtering and screening method is characterized by comprising the following steps:
(1) The data is filtered out, and then the data is filtered,
separating residual data from the acquired data;
(2) The data is screened out, and the data is filtered out,
screening out various residual error values in the iteration step range according to the iteration step range aiming at the separated residual error data;
(3) The data is divided into blocks, and the data is divided into blocks,
partitioning according to the total data amount to form a plurality of data blocks to be processed;
(4) The value of the data block is calculated,
regarding each data block determined in the step (3), taking the iteration step of the first data point of each data block as the abscissa value of the data block, and taking the point with the maximum absolute value in each data block as the ordinate value of the data block;
(5) The result is returned back to the user,
and (4) returning the numerical values of all the data blocks based on the horizontal and vertical coordinates determined in the step (4).
2. The residual curve data filtering and screening method of claim 1, wherein different data filtering manners are matched for different output data in the step (1).
3. The residual curve data filtering and screening method according to claim 1, wherein in the step (2), when screening the data, a processing range of the original data needs to be determined first, and then the data needed for screening is screened based on the determined processing range of the original data.
4. The residual curve data filtering and screening method of claim 1, wherein in the step (3), when data blocking is performed,
if the screened total number N of the iterative steps is less than or equal to N, dividing the data into N data blocks, wherein each block contains the data of one iterative step;
if the total number N of the screened iterative steps is larger than the required numberWhen the data is N blocks, the data is divided into N data blocks, and the mth data block contains N m Iteration step data.
5. A computer-readable storage medium, on which a program is stored, which program, when being executed by a processor, is adapted to carry out the steps of the residual curve data filtering and screening method of any one of claims 1 to 4.
6. A processor for running a program, wherein the program is run to perform the steps of the residual curve data filtering method of any one of claims 1 to 4.
7. A terminal device comprising a processor, a memory and a program stored on and executable on the memory, characterized in that the program code is loaded and executed by the processor to implement the steps of the residual curve data filtering method of any of claims 1-4.
8. A computer program product characterized by being adapted to perform the steps of the residual curve data filtering screening method of any one of claims 1-4 when executed on a data processing device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211058837.2A CN115373850A (en) | 2022-08-31 | 2022-08-31 | Residual curve data filtering and screening method and related equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211058837.2A CN115373850A (en) | 2022-08-31 | 2022-08-31 | Residual curve data filtering and screening method and related equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115373850A true CN115373850A (en) | 2022-11-22 |
Family
ID=84070622
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211058837.2A Pending CN115373850A (en) | 2022-08-31 | 2022-08-31 | Residual curve data filtering and screening method and related equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115373850A (en) |
-
2022
- 2022-08-31 CN CN202211058837.2A patent/CN115373850A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20160246852A1 (en) | Systems and Methods for Quantile Estimation in a Distributed Data System | |
| CN110768912B (en) | API gateway current limiting method and device | |
| CN111310784B (en) | Resource data processing method and device | |
| CN106648839B (en) | Data processing method and device | |
| CN111709415B (en) | Target detection method, device, computer equipment and storage medium | |
| CN110737717A (en) | database migration method and device | |
| CN118193410A (en) | Execution method, equipment and storage medium of memory handling operator | |
| CN117593464A (en) | Construction method and system of electric power digital twin model library | |
| CN117196850A (en) | Method and device for determining financial product combination, electronic equipment and storage medium | |
| CN110928941A (en) | Data fragment extraction method and device | |
| CN115373850A (en) | Residual curve data filtering and screening method and related equipment | |
| CN112712125A (en) | Event stream pattern matching method and device, storage medium and processor | |
| CN114818458A (en) | System parameter optimization method, device, computing equipment and medium | |
| CN114679335B (en) | Power monitoring system network security risk assessment training method, assessment method and equipment | |
| CN115827028A (en) | System upgrading method and device, storage medium and electronic equipment | |
| CN115544011A (en) | Data query method, electronic equipment and computer readable storage medium | |
| CN114817209A (en) | Monitoring rule processing method and device, processor and electronic equipment | |
| CN115442262A (en) | Resource evaluation method and device, electronic equipment and storage medium | |
| CN111079390B (en) | Method and device for determining selection state of check box list | |
| CN112540979A (en) | Expression storage method and device | |
| CN116048978B (en) | Software service performance self-adaptive test method, system, terminal and medium | |
| CN117092698A (en) | Seismic data reconstruction method, processor and storage medium based on binary tree structure | |
| CN113986727B (en) | Function coverage rate detection method, system, terminal and medium based on thermodynamic diagram | |
| CN111753949B (en) | Data block processing method and device and electronic equipment | |
| CN110866033B (en) | Feature determination method and device for predicting query resource occupancy |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Country or region after: China Address after: 201306 Room 105A36, Main Building, No. 99, Huanhu West 1st Road, Lingang New Area, China (Shanghai) Pilot Free Trade Zone, Pudong New Area, Shanghai Applicant after: Zhongke Shuguang Information Industry (Shanghai) Co.,Ltd. Address before: 201306 Room 105A36, Main Building, No. 99, Huanhu West 1st Road, Lingang New Area, China (Shanghai) Pilot Free Trade Zone, Pudong New Area, Shanghai Applicant before: SHANGHAI SUPERCOMPUTER TECHNOLOGY CO.,LTD. Country or region before: China |
|
| CB02 | Change of applicant information |