CN112269610A - Method and device for executing batch model algorithm - Google Patents
Method and device for executing batch model algorithm Download PDFInfo
- Publication number
- CN112269610A CN112269610A CN202011152232.0A CN202011152232A CN112269610A CN 112269610 A CN112269610 A CN 112269610A CN 202011152232 A CN202011152232 A CN 202011152232A CN 112269610 A CN112269610 A CN 112269610A
- Authority
- CN
- China
- Prior art keywords
- model algorithm
- model
- algorithm
- module
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004422 calculation algorithm Methods 0.000 title claims abstract description 420
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000012163 sequencing technique Methods 0.000 claims abstract description 7
- 238000012545 processing Methods 0.000 claims description 7
- 238000013480 data collection Methods 0.000 claims description 5
- 230000009467 reduction Effects 0.000 claims description 4
- 238000011946 reduction process Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 14
- 230000006870 function Effects 0.000 description 64
- 238000001514 detection method Methods 0.000 description 39
- 238000005457 optimization Methods 0.000 description 21
- 238000012423 maintenance Methods 0.000 description 17
- 238000000605 extraction Methods 0.000 description 10
- 238000012544 monitoring process Methods 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 230000002085 persistent effect Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000012856 packing Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4482—Procedural
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a method and a device for executing a batch model algorithm. The method comprises the steps of firstly, wholly initializing a model algorithm, constructing a quick function for the called model algorithm through model algorithm calling relation analysis in the initialization process, and replacing an original model algorithm in other model algorithms with the quick function. When the batch model algorithm is executed, firstly, sequencing is carried out according to the calling relation of the model algorithm, then the sequenced model algorithms are executed in sequence, when a quick function is executed, an intermediate data set is searched according to an interface name and an input parameter, and if corresponding intermediate data can be found in the intermediate data set, the intermediate data is returned; otherwise, executing the original model algorithm, and keeping the execution result to the intermediate data set according to the interface name and the input parameter. The invention greatly improves the execution speed of the batch model algorithm by changing the time in space.
Description
Technical Field
The invention relates to execution optimization of batch model algorithms.
Background
The intelligent operation and maintenance platform relates to a producer side and a client side. The manufacturer produces and provides equipment; the customer side purchases and uses the device; the intelligent operation and maintenance platform provides maintenance service for the use of the equipment for the client side. Each customer party may have purchased a number of different kinds of equipment from a number of manufacturer parties. The manufacturer may produce different kinds of equipment. When the intelligent operation and maintenance platform provides maintenance service for equipment for a client, various operation data of each equipment produced by each manufacturer needs to be collected, and then the state of each equipment is analyzed according to the various operation data. The intelligent operation and maintenance platform analyzes the state of the equipment through various operation data and relates to a model and an algorithm. Since the intelligent operation and maintenance platform involves numerous devices of numerous manufacturers, the number of involved model algorithms is also very large. When the state of the equipment is monitored each time, a plurality of model algorithms need to be executed in batches, and the consumed calculation amount is very large. Particularly, when the model algorithm needs to analyze historical operating data, the amount of calculation required to be consumed is more huge, and even the operation and maintenance platform cannot be burdened.
Disclosure of Invention
The problems to be solved by the invention are as follows: and the method has the problem of large calculation consumption when the model algorithm is executed in batches.
In order to solve the problems, the invention adopts the following scheme:
the invention relates to a method for executing a batch model algorithm, which comprises the following steps:
s11: acquiring a model algorithm information set; the model algorithm information is information of a model algorithm and at least comprises an interface name, input parameter information and a program instruction set;
s12: obtaining the calling relation of the model algorithm by analyzing the program instruction set of each model algorithm in the model algorithm information set;
s13: according to the calling relation of the model algorithm, constructing a corresponding fast function for each called model algorithm, and replacing the calling of the called model algorithm in the program instruction set of the model algorithm calling the called model algorithm with the calling of the corresponding fast function;
s21: acquiring a model algorithm set to be executed;
s22: according to the calling relation of the model algorithms, sequencing each model algorithm in the model algorithm set to be executed to obtain a sequenced model algorithm queue, so that the called model algorithms are arranged before the model algorithms calling the called model algorithms;
s31: acquiring an input source data set;
s32: sequentially executing each model algorithm after the rapid function replacement in the ordered model algorithm queue according to the input source data set; when the model algorithm is executed, for the model algorithm with the called situation, storing output result data after the model algorithm is executed into an intermediate data set according to the interface name of the model algorithm and input parameter data of the model algorithm; when the quick function is called, searching output result data from the intermediate data set according to the interface name and the input parameter data of the model algorithm corresponding to the quick function; if the output result data can be found, returning the found output result data as the output result data of the quick function; and if the output result data cannot be found, executing the model algorithm corresponding to the quick function, and storing the output result data after execution into an intermediate data set according to the interface of the model algorithm and the input parameter data of the model algorithm.
Further, according to the execution method of the batch model algorithm of the present invention, the step S31 further includes performing a reduction processing on the collection cycle of the periodically collected data in the input source data set, so that the collection cycles of the periodically collected data are the same; and step S32, sequentially executing each model algorithm in the ordered model algorithm queue according to the data set after the collection period reduction processing.
Further, the method for executing the batch model algorithm according to the present invention further includes step S14; the step S14: persistently storing the constructed fast function, the model algorithm information set replaced by the step S13 and the model algorithm calling relation.
Further, the method for executing the batch model algorithm according to the present invention further includes step S23; the step S23: constructing a first program instruction set according to the ordered model algorithm queue; in step S32, a first set of program instructions is executed according to the input source data set.
The invention relates to an execution device of a batch model algorithm, which comprises a model algorithm integral initialization module, a model algorithm to be executed initialization module and a sequential execution module;
the model algorithm overall initialization module comprises the following modules:
m11, used for: acquiring a model algorithm information set; the model algorithm information is information of a model algorithm and at least comprises an interface name, input parameter information and a program instruction set;
m12, used for: obtaining the calling relation of the model algorithm by analyzing the program instruction set of each model algorithm in the model algorithm information set;
m13, used for: according to the calling relation of the model algorithm, constructing a corresponding fast function for each called model algorithm, and replacing the calling of the called model algorithm in the program instruction set of the model algorithm calling the called model algorithm with the calling of the corresponding fast function;
the model algorithm to be executed initialization module comprises the following modules:
m21, used for: acquiring a model algorithm set to be executed;
m22, used for: according to the calling relation of the model algorithms, sequencing each model algorithm in the model algorithm set to be executed to obtain a sequenced model algorithm queue, so that the called model algorithms are arranged before the model algorithms calling the called model algorithms;
the in-order execution module comprises the following modules:
m31, used for: acquiring an input source data set;
m32, used for: sequentially executing each model algorithm after the rapid function replacement in the ordered model algorithm queue according to the input source data set; when the model algorithm is executed, for the model algorithm with the called situation, storing output result data after the model algorithm is executed into an intermediate data set according to the interface name of the model algorithm and input parameter data of the model algorithm; when the quick function is called, searching output result data from the intermediate data set according to the interface name and the input parameter data of the model algorithm corresponding to the quick function; if the output result data can be found, returning the found output result data as the output result data of the quick function; and if the output result data cannot be found, executing the model algorithm corresponding to the quick function, and storing the output result data after execution into an intermediate data set according to the interface of the model algorithm and the input parameter data of the model algorithm.
Further, according to the apparatus for executing the batch model algorithm of the present invention, the module M31 further includes a data collection cycle reduction process for periodically collecting data in the input source data set, so that the data collection cycles of the periodically collected data are the same; and the module M32 executes each model algorithm in the ordered model algorithm queue in turn according to the data set subjected to the normalization processing of the acquisition period.
Further, according to the apparatus for executing the batch model algorithm of the present invention, the model algorithm whole initialization module further includes a module M14; the module M14 is configured to: and persistently storing the constructed quick function, the model algorithm information set replaced by the module M13 and the model algorithm calling relation.
Further, according to the apparatus for executing the batch model algorithm of the present invention, the module for initializing the model algorithm to be executed further includes a module M23; the module M23 is configured to: constructing a first program instruction set according to the ordered model algorithm queue; in the module M32, the first set of program instructions is executed according to the input source data set.
The invention has the following technical effects: the invention greatly improves the execution speed of the batch model algorithm by changing the time in space.
Drawings
Fig. 1 is a schematic block diagram of an embodiment of the present invention.
Wherein the content of the first and second substances,
100 is a device condition monitoring system;
111 is a manufacturer model algorithm package and equipment information access module, 112 is a model algorithm optimization package access module, 113 is an equipment operation data access module, 114 is an intermediate data access module, and 115 is an equipment state information access module;
121 is a device operation data receiving module;
131 is a device operation data input interface module;
141 is a model algorithm calling relation analysis module, 142 is a model algorithm optimization module;
151 is a to-be-executed model algorithm extraction module, and 152 is a to-be-executed model algorithm sorting module;
160 is a device status analysis execution module;
dev _1 to Dev _ n are n detection devices, respectively.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
The embodiment relates to an intelligent operation and maintenance platform system. The intelligent operation and maintenance platform system, as shown in fig. 1, includes an equipment status monitoring system 100. The device condition monitoring system 100 is a system implemented by a machine executing a software program. The machine for realizing the equipment state monitoring system 100 is connected with detection equipment Dev _ 1-Dev _ n for operation and maintenance management. The intelligent operation and maintenance platform system is used for providing operation and maintenance services for the connected detection devices, wherein the device state detection system 100 is used for analyzing the device states of the detection devices through analyzing the operation data of the detection devices. Wherein the detection devices Dev _ 1-Dev _ n may be provided by a plurality of vendors and comprise various different types of devices.
The equipment state detection system 100 is a specific embodiment of an apparatus for executing a batch model algorithm according to the present invention, and includes: the system comprises a manufacturer model algorithm package and equipment information access module 111, a model algorithm optimization package access module 112, an equipment operation data access module 113, an intermediate data access module 114, an equipment state information access module 115, an equipment operation data receiving module 121, a model algorithm calling relation analysis module 131, a model algorithm optimization module 132, a model algorithm to be executed extraction module 141, a model algorithm to be executed sorting module 142, an equipment state analysis execution module 150, an equipment operation data input interface module 161 and an equipment state data output interface module 162. The interaction process between the modules of the equipment state detection system 100 is a specific embodiment of the execution method of the batch model algorithm referred to in the present invention.
The manufacturer model algorithm package and device information access module 111 is used for accessing the manufacturer model algorithm package and the device information of the detection device for operation and maintenance management. The manufacturer model algorithm package is usually represented by a file, and contains model algorithm information for analyzing the equipment state of various types of detection equipment of each manufacturer. The model algorithm information is information of the model algorithm. In this embodiment, the model algorithm is implemented by a function, and therefore, the information of the model algorithm, that is, the function information corresponding to the model algorithm, at least includes an interface name, input parameter information, and a program instruction set. The function for defining the model algorithm can be realized by adopting a script language, and can also be realized by adopting languages such as java and Python. In this embodiment, the function for defining the model algorithm is java, and the vendor model algorithm package is java class file package. The device information is information for detecting the device, and includes information such as a device type, a vendor, a device serial number, and a device status parameter. The detection devices for operation and maintenance management in this embodiment are the aforementioned connected detection devices Dev _1 to Dev _ n for operation and maintenance management. In this embodiment, the manufacturer model algorithm package is preferably stored in a file manner, and the device information is preferably stored in a database manner.
The model algorithm optimization package accessing module 112 is used for accessing the optimized model algorithm package and the calling relationship of the model algorithm. For the optimized model algorithm package and the calling relationship of the model algorithm, see the description of the model algorithm calling relationship analysis module 141 and the model algorithm optimization module 142.
The device operation data access module 113 is used for accessing operation data of the detection device. The operation data of the detection device in this embodiment is the input source data referred to in the present invention; the input source data set referred to by the present invention is in this embodiment a set of operational data of the detection device.
The intermediate data access module 114 is used to access intermediate data generated during the execution of the model algorithm. The set of intermediate data is the intermediate data set. In this embodiment, the intermediate data access module 114 provides two interfaces: StoreMFVallue and FindMFValue. The StoreMFValue is used for storing result data executed by the model algorithm into an intermediate data set as intermediate data according to the interface name of the model algorithm and input parameter data of the model algorithm, and the calling method is as follows: StoreMFValue (MF _ Name, MF _ ParaList, MF _ rValue). The FindMFValue is used for searching whether corresponding intermediate data exist in the intermediate data set according to the interface name of the model algorithm and the input parameter data of the model algorithm, and the calling method is as follows: MF _ rValue = finmfvalue (MF _ Name, MF _ ParaList).
In the above-described method for calling StoreMFValue and finmfvalue, MF _ Name represents an interface Name of a model algorithm, MF _ ParaList represents a list of input parameter data of the model algorithm, and MF _ rValue represents result data returned when the model algorithm represented by MF _ Name is called according to the input parameter data MF _ ParaList. If the MF _ rValue returned by the findmavalue (MF _ Name, MF _ paramist) is empty, it indicates that the intermediate data corresponding to the MF _ Name and MF _ paramist cannot be found in the intermediate data set.
The device status information access module 115 is configured to access the analyzed device status information. In this embodiment, the device status information of each device analyzed by the device status monitoring system 100 is stored by the device status information access module 115. When other modules of the intelligent operation and maintenance platform system need to acquire the device state information of the detection device, the device state information is extracted through the device state information access module 115.
The device operation data receiving module 121 is configured to receive operation data sent by the detection device, or collect operation data of the detection device at regular time. These operational data are related to the device status. Such as the optical fiber signal strength of the transmission equipment and the error rate of data transmission, the processor temperature and the fan speed of the computer equipment, the energy consumption of the energy-saving equipment, and the like. The device operation data receiving module 121 continuously stores the operation data of the detection device via the device operation data access module 113. The storage of the operating data of the detection device preferably uses a database.
The device operation data input interface module 131 is configured to provide an interface service for the operation data of the detection device stored in the model algorithm extraction device operation data access module 113. The set of the operation data of the detection device stored by the device operation data access module 113, that is, the acquisition input source data set referred to by the module M31 of the present invention, is extracted by the device operation data input interface module 131.
It should be noted that, the device status monitoring system 100 of the present embodiment has a plurality of detection devices connected thereto, and therefore, the time periods of the operation data sent by the different detection devices may be different, that is, the time periods of the operation data of the detection devices received by the device operation data receiving module 121 may be different. Thus, the time periods of the operation data of the respective detection devices stored by the device operation data access module 113 may be different.
In order to facilitate the processing of the model algorithm, in this embodiment, in the device operation data input interface module 131, the data collected periodically is subjected to a collection period reduction process, so that the data collection periods collected periodically are the same. Thus, the model algorithm extracts the operation data of the detection device through the device operation data input interface module 131 for the same time period. The data collection period is the time period of the operation data of the detection device received by the device operation data receiving module 121. The acquisition period reduction treatment is specifically as follows:
firstly, acquiring the running data acquisition periods of all detection devices, then calculating the greatest common divisor of the acquisition periods, and finally, taking the greatest common divisor as the acquisition period to perform period compensation on the running data of the detection devices. For example, the operating data acquisition period of the detection device Dev _1 is 3 minutes, and the operating data acquisition period of the detection device Deve _2 is 5 minutes.
For example, in a model algorithm, the latest 24 hours of operation data of the detecting device Dev _1 needs to be acquired, and the latest 480 operation data of the detecting device Dev _1 need to be extracted from the device operation data access module 113 according to the original 3 minutes of acquisition period. After periodic compensation is performed by the device operation data input interface module 131, 480 operation data are compensated into 1440 operation data.
The method of running data compensation can simply employ linear compensation. For example, the operation data of the detection device Dev _1 includes processor temperature, and in the 480 operation data queues, the first processor temperature is 37, and the second processor temperature is 40, that is, the 480 processor temperature data are: 37,40,...... The compensation of the 480 operational data into 1440 operational data means that two compensated processor temperature data are inserted between the first processor temperature and the second processor temperature, the two compensated processor temperatures being 38 and 39 in a linear compensation manner, whereby the 1440 compensated processor temperatures are: 37,38,39,40,...... Where 38 and 39 are to compensate for the inserted processor temperature.
The model algorithm calls the relational analysis module 141, namely the aforementioned module M12, for: and analyzing the program instruction set of each model algorithm in the model algorithm information set to obtain the calling relation of the model algorithm. The set of model algorithm information is from the manufacturer model algorithm package and the manufacturer model algorithm package stored by the device information access module 111. The manufacturer model algorithm package and the manufacturer model algorithm package stored in the device information access module 111 are combined into a model algorithm information set, that is, the model algorithm information set obtained in the aforementioned module M11.
In this embodiment, the model algorithm is implemented by a java function, and the manufacturer model algorithm package is a class file package of java, so that when the module M11 obtains the model algorithm information set, the method further includes a processing procedure of performing reverse compilation on the class file package of java. Therefore, in this embodiment, the program instruction set of the model algorithm in the model algorithm information set is a java function definition expressed by java language. Since the model algorithm is implemented by a function, the essence of the model algorithm call relation analysis module 141 is function call relation analysis. The analysis of the function call relationship is familiar to those skilled in the art, and the description thereof is omitted.
The model algorithm optimization module 142, i.e. the aforementioned module M13, is configured to: and according to the calling relation of the model algorithms, constructing a corresponding quick function for each called model algorithm, and replacing the calling of the called model algorithm in the program instruction set of the model algorithm calling the called model algorithm with the calling of the corresponding quick function.
For example, assume that the current pair of model algorithm information sets includes ten model algorithms, MF 1-MF 10. Where a model algorithm, such as MF5, calls model algorithm MF3, then MF3 is the called model algorithm. The construction of the corresponding fast function definition for the called model algorithm MF3 is expressed as follows:
MF3_f(MF3ParaList)
{
return Valueset rValue// define a return value;
rValue = FindMFValue(“MF3”, MF3ParaList);
v/find intermediate dataset
If (rValue! = NULL) return rValue;/If intermediate data is found in the intermediate dataset
rValue = MF3(MF3 ParaList); if not found, directly calling the original model algorithm;
StoreMFValue(“MF3”, MF3ParaList,rValue);
// store the results to an intermediate dataset
Return rValue; // return;
}
of the fast functions represented by the pseudo code, MF3_ f is the fast function constructed by MF 3. MF3 is the original model algorithm corresponding to fast function MF3_ f. MF3ParaList is the input parameter to the model algorithm. FindMFValue and StoreMFValue are interfaces provided by the intermediate data access module 114.
"replace all calls to the called model algorithm in the set of program instructions for the model algorithm calling the called model algorithm with calls to its corresponding fast function", that is, replace MF3 in the set of program instructions for MF5 with the fast function MF3_ f for the aforementioned model algorithm MF 5. For example, the original model algorithm MF5 is defined as follows:
MF5(MF5ParaList)
{
……
T3 = MF3(MF3ParaList)；
……
}
the model algorithm MF5 after the fast function replacement is defined as follows:
MF5(MF5ParaList)
{
……
T3 = MF3_f(MF3ParaList)；
……
}
the fast function constructed by the model algorithm optimization module 142 and the model algorithm information set formed after the fast function replacement by the model algorithm optimization module 142 constitute the "optimized model algorithm package".
In this embodiment, the calling relationship of the model algorithm obtained by the model algorithm calling relationship analysis module 141, the fast function constructed by the model algorithm optimization module 142, and the model algorithm information set formed after the fast function replacement is performed by the model algorithm optimization module 142 are continuously stored by the model algorithm optimization package access module 112. This persistent storage is the module M14 described above.
The to-be-executed model algorithm extraction module 151 is configured to: the required executed model algorithm is extracted according to the manufacturer model algorithm package and the device information in the device information access module 111. Specifically, the model algorithm is corresponding to the device state parameter in the device information. Device status parameters such as a certain device type include: a1, A2 and A3, wherein the corresponding model algorithms are MF3, MF4 and MF5 respectively. Therefore, the to-be-executed model algorithm extracted by the to-be-executed model algorithm extraction module 151 is: MF3, MF4, MF 5. MF3, MF4 and MF5 constitute a set of model algorithms to be executed.
The correspondence between the model algorithms and the device state parameters is preferably defined in a definition file in some XML format in the manufacturer model algorithm package.
Obviously, the model algorithms extracted by the model algorithm to be executed extraction module 141 are a subset of the model algorithms defined by the manufacturer model algorithm package. For example, ten model algorithms, MF 1-MF 10, are defined by the manufacturer model algorithm package, and the model algorithms extracted by the to-be-executed model algorithm extraction module 151 are only three: MF3, MF4, MF 5. The module 151 for extracting model algorithms to be executed, i.e. the aforementioned module M21, extracts a set of model algorithms to be executed. The model algorithm set is generally a set composed of interface names of model algorithms.
The to-be-executed model algorithm ordering module 152 is configured to: according to the calling relation of the model algorithms, sequencing each model algorithm in the model algorithm set to be executed to obtain a sequenced model algorithm queue, so that the called model algorithms are arranged before the model algorithms calling the called model algorithms; a first set of program instructions is then constructed from the ordered array of model algorithms. In the model algorithm to be executed sequencing module 152, the calling relationship of the model algorithm is taken from the calling relationship of the model algorithm persistently stored by the model algorithm optimization package accessing module 112. The model algorithm optimization package access module 112 persistently stores the calling relationships of the model algorithms from the output of the model algorithm calling relationship analysis module 141. Therefore, the calling relationship of the model algorithm needed to be used by the to-be-executed model algorithm sorting module 152 is ultimately output from the model algorithm calling relationship analysis module 141. The set of model algorithms to be executed in the model algorithm to be executed ranking module 152 comes from the output of the model algorithm to be executed extraction module 151.
The functions of the model algorithm ordering module 152 to be executed are the functions of the module M22 and the module M23. The output of the module M22 is a sorted model algorithm queue, which is a queue of model algorithms to be executed. Module M23 is the process of packing the ordered queue of model algorithms to be executed into a program instruction set. Specifically, the program instruction set of each model algorithm in the model algorithm information set formed after the model algorithm optimization module 142 performs the fast function replacement, which is continuously stored by the model algorithm optimization package access module 112, and the program instruction set of the fast function constructed by the model algorithm optimization module 142 are first extracted, and then the first program instruction set is formed by combining the summary interface of the to-be-executed module algorithms.
The summary interface of the to-be-executed module algorithm is a calling sequence composed of the ordered to-be-executed model algorithms, for example, the ordered to-be-executed model algorithms are MF3, MF4, and MF 5; wherein, MF5 calls MF 3; the summary interface of the module algorithm to be executed is defined as follows:
Main_Interface()
{
……
initialization of input parameters M3ParaList, M4ParaList, and M5ParaList for model Algorithm
……
MF3_rValue = MF3_f(M3ParaList);
MF4_rValue = MF4(M4ParaList);
MF5_rValue = MF5(M5ParaList);
……
V/will model Algorithm execution results MF3_ rValue, MF4_ rValue, MF5_ rValue
// persistent storage via device status information access module 115.
}
It should be noted that, in the summary interface of the module algorithm to be executed, the model algorithm to be called is replaced by its corresponding fast function.
In this embodiment, the fast functions stored in the model algorithm optimization packet access module 112 and the program instruction sets of the model algorithms are java source codes. Therefore, when the to-be-executed model algorithm sorting module 152 forms the first program instruction set, the compiling of the java source code is further included, and the finally obtained first program instruction set is a class file package of java.
The device state analysis executing module 160 is configured to execute a first program instruction set, that is, in the module M32, sequentially execute each model algorithm after fast function replacement in the ordered model algorithm queue according to the input source data set. The first program instruction set in this embodiment may be regarded as "each model algorithm after replacing the fast functions in the ordered model algorithm queue is executed in sequence" represented by the program instruction set.
When the first program instruction set is executed, the first program instruction set replaces the corresponding original model algorithm by the fast function. Therefore, for the model algorithm with the called condition, the output result data after the execution of the model algorithm is stored into the intermediate data set according to the interface name of the model algorithm and the input parameter data of the model algorithm. In this embodiment, the process of outputting the model algorithm with the called condition to the intermediate data set is implemented by a program instruction set defined by a summary interface Main _ invocation interface of the module algorithm to be executed.
Taking the model algorithm to be executed after the sorting in the Main _ Interface as an example, the model algorithm MF3 is a model algorithm with a called situation, and therefore the original model algorithm MF3 is replaced by the fast function MF3_ f. Therefore, when the fast function MF3_ f is executed, findmavalue ("MF 3", MF3ParaList) is called first, since MF3 has not been called before, the returned rValue is empty, that is, the corresponding result cannot be found from the intermediate data set, then the original model algorithm, that is, rValue = MF3(MF3ParaList), is executed, and then the output result data rValue after the execution of the original model algorithm MF3 is stored into the intermediate data set by calling function StoreMFValue.
In the above process, since FindMFValue always returns to empty. Therefore, the summary interface Main _ inference of the module algorithm to be executed may be further optimized as follows:
Main_Interface()
{
……
initialization of input parameters M3ParaList, M4ParaList, and M5ParaList for model Algorithm
……
MF3_rValue = MF3(M3ParaList);
StoreMFValue(“MF3”, MF3ParaList, MF3_rValue);
MF4_rValue = MF4(M4ParaList);
MF5_rValue = MF5(M5ParaList);
……
V/will model Algorithm execution results MF3_ rValue, MF4_ rValue, MF5_ rValue
// persistent storage via device status information access module 115.
}
Because the original model algorithm MF5 calls the original model algorithm MF3, in the program instruction set defined by the summary interface Main _ inference of the module algorithm to be executed, the model algorithm MF5 is an optimized model algorithm, and the fast function MF3_ f of the model algorithm MF3 is called. Therefore, when the program instruction set defined by the summary interface Main _ referrface of the module algorithm to be executed is executed to the model algorithm MF5, the fast function MF3_ f of the model algorithm MF3 is called.
Refer to the program instruction set definition of the fast function MF3_ f described above. When the fast function MF3_ f is called, the findfalue is called for the input parameter by using the interface name "MF 3" of the model algorithm corresponding to the fast function MF3_ f and the input parameter data MF3ParaList, that is, a statement: rValue = finmfvalue ("MF 3", MF3 ParaList). The findfalue calling process is the aforementioned "searching output result data from the intermediate data set according to the interface name and the input parameter data of the model algorithm corresponding to the fast function". Wherein, rValue is the output result data obtained by searching. Then judging whether the returned rValue of the FindMFValue is empty, if the rValue is not empty, returning the rValue, namely: "If (rValue! = NULL) return rValue". The procedure of the statement "If (rValue | = NULL) return rValue" is to perform the aforementioned "If the output result data can be found, return the found output result data as the output result data of the fast function". If the output result data cannot be found, executing the model algorithm corresponding to the fast function, and storing the output result data after execution into an intermediate data set according to the interface of the model algorithm and the input parameter data of the model algorithm, namely, a process in which statements "rValue = MF3(MF3 ParaList)" and "StoreMFValue (" MF3 ", MF3ParaList, rValue)" are executed.
That is, "when the fast function is called," in the module M32, the output result data is searched from the intermediate data set according to the interface name and the input parameter data of the model algorithm corresponding to the fast function; if the output result data can be found, returning the found output result data as the output result data of the quick function; if the output result data can not be found, executing the model algorithm corresponding to the quick function, and storing the output result data after being executed into the intermediate data set according to the interface of the model algorithm and the input parameter data of the model algorithm, wherein the execution is realized by executing a program instruction set defined by the quick function.
When the program instruction set defined by the summary interface Main _ referrface of the module algorithm to be executed is executed, firstly, the model algorithm input parameter data M3ParaList, M4ParaList and M5ParaList are initialized. Inputting parameter data M3ParaList, M4ParaList and M5ParaList, including operation data of the detection device; that is, when initializing the model algorithm input parameters, the operation data of the detection device needs to be acquired. The operation data of the acquisition detection device is the "acquisition input source data set" referred to by the aforementioned module M31. After the model algorithms MF3, MF4, MF5 are executed, the results MF3_ rValue, MF4_ rValue, MF5_ rValue returned by each model algorithm MF3, MF4, MF5 will be stored persistently by calling the interface function provided by the device state information access module 115. That is, in this embodiment, the module M31 and the module M32 are implemented by a program instruction set of the summary interface Main _ inference of the module algorithm to be executed.
It should be noted that, in the present embodiment, executing the first program instruction set is a periodic process, and is generally driven by a timer, that is, the device state analysis execution module 160 is driven by a timer. The process processed by the model algorithm call relation analysis module 141 and the model algorithm optimization module 142 is a process of model algorithm overall initialization, that is, the model algorithm call relation analysis module 141 and the model algorithm optimization module 142 constitute the aforementioned model algorithm overall initialization module. The process of model algorithm global initialization occurs when a change occurs to the manufacturer model algorithm package. The processes processed by the to-be-executed model algorithm extraction module 151 and the to-be-executed model algorithm ranking module 152 are processes of initializing to-be-executed model algorithms. That is, the to-be-executed model algorithm extraction module 151 and the to-be-executed model algorithm sorting module 152 constitute the aforementioned to-be-executed model algorithm initialization module. The process of model algorithm initialization to be performed occurs when a change occurs in the manufacturer model algorithm package or when a change occurs in the detection equipment. Therefore, the execution method of the batch model algorithm corresponding to the execution device of the batch model algorithm in this embodiment includes two initialization steps and a step of executing the first program instruction set, where the two initialization steps are respectively: the method comprises a model algorithm overall initialization step and a to-be-executed model algorithm initialization step. The model algorithm overall initialization step is the step corresponding to the model algorithm overall initialization module, that is, the combination of the above steps S11, S12, S13 and S14. The model algorithm to be executed is initialized, that is, the steps corresponding to the model algorithm to be executed initialization module, that is, the combination of the aforementioned steps S21, S22 and S23. And after the initialization is completed, executing the first program instruction set by a timer. In this embodiment, when the first program instruction set is executed, the aforementioned step S31 and step S32 are executed by calling the summary interface Main _ referrface of the execution module algorithm.
It should be noted that the present embodiment is only an application of the method and apparatus for executing batch model algorithm of the present invention in the device status detection system 100. The method and the device for executing the batch model algorithm are applied to different application occasions and are adjusted within the protection scope of the claims of the invention.
Claims (8)
1. A method for executing a batch model algorithm is characterized by comprising the following steps:
s11: acquiring a model algorithm information set; the model algorithm information is information of a model algorithm and at least comprises an interface name, input parameter information and a program instruction set;
s12: obtaining the calling relation of the model algorithm by analyzing the program instruction set of each model algorithm in the model algorithm information set;
s13: according to the calling relation of the model algorithm, constructing a corresponding fast function for each called model algorithm, and replacing the calling of the called model algorithm in the program instruction set of the model algorithm calling the called model algorithm with the calling of the corresponding fast function;
s21: extracting a model algorithm set to be executed;
s22: according to the calling relation of the model algorithms, sequencing each model algorithm in the model algorithm set to be executed to obtain a sequenced model algorithm queue, so that the called model algorithms are arranged before the model algorithms calling the called model algorithms;
s31: acquiring an input source data set;
s32: sequentially executing each model algorithm after the rapid function replacement in the ordered model algorithm queue according to the input source data set; when the model algorithm is executed, for the model algorithm with the called situation, storing output result data after the model algorithm is executed into an intermediate data set according to the interface name of the model algorithm and input parameter data of the model algorithm; when the quick function is called, searching output result data from the intermediate data set according to the interface name and the input parameter data of the model algorithm corresponding to the quick function; if the output result data can be found, returning the found output result data as the output result data of the quick function; and if the output result data cannot be found, executing the model algorithm corresponding to the quick function, and storing the output result data after execution into an intermediate data set according to the interface of the model algorithm and the input parameter data of the model algorithm.
2. The method for executing a batch model algorithm according to claim 1, wherein the step S31 further comprises performing a collection period reduction process on the periodically collected data in the input source data set, so that the collection periods of the periodically collected data are the same; and step S32, sequentially executing each model algorithm in the ordered model algorithm queue according to the data set after the collection period reduction processing.
3. The method of performing a batch model algorithm of claim 1 further comprising step S14; the step S14: persistently storing the constructed fast function, the model algorithm information set replaced by the step S13 and the model algorithm calling relation.
4. The method of performing a batch model algorithm of claim 1 further comprising step S23; the step S23: constructing a first program instruction set according to the ordered model algorithm queue; in step S32, a first set of program instructions is executed according to the input source data set.
5. The execution device of the batch model algorithm is characterized by comprising a model algorithm integral initialization module, a model algorithm to be executed initialization module and a sequential execution module;
the model algorithm overall initialization module comprises the following modules:
m11, used for: acquiring a model algorithm information set; the model algorithm information is information of a model algorithm and at least comprises an interface name, input parameter information and a program instruction set;
m12, used for: obtaining the calling relation of the model algorithm by analyzing the program instruction set of each model algorithm in the model algorithm information set;
m13, used for: according to the calling relation of the model algorithm, constructing a corresponding fast function for each called model algorithm, and replacing the calling of the called model algorithm in the program instruction set of the model algorithm calling the called model algorithm with the calling of the corresponding fast function;
the model algorithm to be executed initialization module comprises the following modules:
m21, used for: acquiring a model algorithm set to be executed;
m22, used for: according to the calling relation of the model algorithms, sequencing each model algorithm in the model algorithm set to be executed to obtain a sequenced model algorithm queue, so that the called model algorithms are arranged before the model algorithms calling the called model algorithms;
the in-order execution module comprises the following modules:
m31, used for: acquiring an input source data set;
m32, used for: sequentially executing each model algorithm after the rapid function replacement in the ordered model algorithm queue according to the input source data set; when the model algorithm is executed, for the model algorithm with the called situation, storing output result data after the model algorithm is executed into an intermediate data set according to the interface name of the model algorithm and input parameter data of the model algorithm; when the quick function is called, searching output result data from the intermediate data set according to the interface name and the input parameter data of the model algorithm corresponding to the quick function; if the output result data can be found, returning the found output result data as the output result data of the quick function; and if the output result data cannot be found, executing the model algorithm corresponding to the quick function, and storing the output result data after execution into an intermediate data set according to the interface of the model algorithm and the input parameter data of the model algorithm.
6. The apparatus for executing batch model algorithm according to claim 5, wherein said module M31 further comprises a data collection period reduction process for periodically collected data in the input source data set, so that the collection periods of the periodically collected data are the same; and the module M32 executes each model algorithm in the ordered model algorithm queue in turn according to the data set subjected to the normalization processing of the acquisition period.
7. The apparatus for performing a batch modeling algorithm of claim 5 wherein said model algorithm global initialization module further comprises module M14; the module M14 is configured to: and persistently storing the constructed quick function, the model algorithm information set replaced by the module M13 and the model algorithm calling relation.
8. The apparatus for executing batch model algorithms according to claim 1, wherein said model algorithm to be executed initialization module further comprises a module M23; the module M23 is configured to: constructing a first program instruction set according to the ordered model algorithm queue; in the module M32, the first set of program instructions is executed according to the input source data set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011152232.0A CN112269610A (en) | 2020-10-26 | 2020-10-26 | Method and device for executing batch model algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011152232.0A CN112269610A (en) | 2020-10-26 | 2020-10-26 | Method and device for executing batch model algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112269610A true CN112269610A (en) | 2021-01-26 |
Family
ID=74341883
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011152232.0A Pending CN112269610A (en) | 2020-10-26 | 2020-10-26 | Method and device for executing batch model algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112269610A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790778A (en) * | 1996-08-07 | 1998-08-04 | Intrinsa Corporation | Simulated program execution error detection method and apparatus |
| US20090307654A1 (en) * | 2008-06-06 | 2009-12-10 | Hiroshi Ishikawa | System, method and computer program for generating sequence diagram |
| JP2012104065A (en) * | 2010-11-12 | 2012-05-31 | Fuji Electric Co Ltd | Language conversion apparatus and language conversion program |
| CN108241543A (en) * | 2016-12-30 | 2018-07-03 | 深圳壹账通智能科技有限公司 | Method, service server and the system that business operation breakpoint performs |
| US20190278607A1 (en) * | 2016-11-18 | 2019-09-12 | V12 Technology Limited | Event handling instruction processing |
| CN111339375A (en) * | 2020-03-19 | 2020-06-26 | 中国海洋石油集团有限公司 | Universal big data model configuration and analysis method |
-
2020
- 2020-10-26 CN CN202011152232.0A patent/CN112269610A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5790778A (en) * | 1996-08-07 | 1998-08-04 | Intrinsa Corporation | Simulated program execution error detection method and apparatus |
| US20090307654A1 (en) * | 2008-06-06 | 2009-12-10 | Hiroshi Ishikawa | System, method and computer program for generating sequence diagram |
| JP2012104065A (en) * | 2010-11-12 | 2012-05-31 | Fuji Electric Co Ltd | Language conversion apparatus and language conversion program |
| US20190278607A1 (en) * | 2016-11-18 | 2019-09-12 | V12 Technology Limited | Event handling instruction processing |
| CN108241543A (en) * | 2016-12-30 | 2018-07-03 | 深圳壹账通智能科技有限公司 | Method, service server and the system that business operation breakpoint performs |
| CN111339375A (en) * | 2020-03-19 | 2020-06-26 | 中国海洋石油集团有限公司 | Universal big data model configuration and analysis method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Canfora et al. | An improved algorithm for identifying objects in code | |
| CN110083436A (en) | A kind of business datum real-time monitoring system and method based on Java bytecode enhancing technology | |
| CN112394942B (en) | Distributed software development compiling method and software development platform based on cloud computing | |
| CN113935497A (en) | Intelligent operation and maintenance fault processing method, device and equipment and storage medium thereof | |
| CN110795455A (en) | Dependency relationship analysis method, electronic device, computer device and readable storage medium | |
| CN112328671A (en) | Data format conversion method, system, storage medium and equipment | |
| CN111897548B (en) | Front-end resource issuing method and device, electronic equipment and computer storage medium | |
| CN113626241B (en) | Abnormality processing method, device, equipment and storage medium for application program | |
| US6374261B1 (en) | Expert system knowledge-deficiency reduction through automated database updates from semi-structured natural language documents | |
| CN112529528B (en) | Workflow monitoring and warning method, device and system based on big data flow calculation | |
| CN113407801A (en) | Cloud computing index recommendation method and system | |
| CN115981902A (en) | Fine-grained distributed micro-service system abnormal root cause positioning method and device | |
| CN112269610A (en) | Method and device for executing batch model algorithm | |
| Barros et al. | Static analysis for improved modularity of procedural web application programming interfaces | |
| CN111752927B (en) | Clone-based data form generation method, device, terminal equipment and medium | |
| CN109033196A (en) | A kind of distributed data scheduling system and method | |
| CN114021759A (en) | Method for managing digital factory equipment based on AIOT and MR technology | |
| CN112130933A (en) | Method and device for constructing and calling operator set | |
| Butt et al. | Towards Ontology Driven Provenance in Scientific Workflow Engine. | |
| CN110737707A (en) | Multi-system log searching method and device and computer readable storage medium | |
| CN117081666B (en) | Fault prediction method, device, electronic equipment, storage medium and program product | |
| CN114596011B (en) | Enterprise data processing method based on artificial intelligence and related device | |
| CN109117381A (en) | The adjusting, measuring method and device of processing task | |
| CN112507013B (en) | Industrial equipment data storage method and device | |
| CN115242613B (en) | Target node determining method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |