CN115099060A

CN115099060A - Simulation method and device for efficiency evaluation system

Info

Publication number: CN115099060A
Application number: CN202210864627.6A
Authority: CN
Inventors: 陈西选; 李帼伟; 秦斌; 蔡磊; 曲凯; 冯金金; 樊志强
Original assignee: Clp Taiji Group Co ltd
Current assignee: Clp Taiji Group Co ltd
Priority date: 2022-07-22
Filing date: 2022-07-22
Publication date: 2022-09-23

Abstract

The application discloses a simulation method and a device for an efficiency evaluation system, which comprise the following steps: determining an evaluation demand model corresponding to the target demand according to the target evaluation demand, wherein the evaluation demand model is used for definitely defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model; generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model; operating the simulation model to obtain evaluation data of efficiency evaluation; determining a comprehensive efficiency evaluation model according to the evaluation data; the comprehensive efficiency evaluation model is operated to obtain an efficiency evaluation result, and the problems that a complex system lacks systematic efficiency evaluation method guidance and can be described, quantized, analyzed and executed are solved.

Description

Simulation method and device for efficiency evaluation system

Technical Field

The present application relates to the field of computer technologies, and in particular, to a simulation method and apparatus, a terminal device, and a storage medium for an efficiency evaluation system.

Background

In the related research and research summary of system performance evaluation, the system performance is evaluated based on the architecture model, but mainly from the architecture model itself, the influence of the operation task and the operation environment on the system performance is less concerned, and meanwhile, the architecture elements are not effectively combined with the functions and behaviors of simulation entities such as the operation unit and the weapon platform in the simulation process, so that the influence of the self-capability of the operation unit on the system performance cannot be truly embodied.

The currently developed system performance evaluation research is system structure verification or system performance evaluation developed aiming at specific evaluation requirements in specific fields, and the formed method and tool have low expansibility, so that the method and the tool are difficult to be reused in other fields, and particularly do not have the capability of constructing an evaluation test environment according to the evaluation requirements.

Disclosure of Invention

The invention aims to provide a simulation method, a simulation device, terminal equipment and a storage medium for an efficiency evaluation system, so as to solve the defects in the prior art.

In a first aspect, an embodiment of the present invention provides a simulation method for a performance evaluation system, where the method includes:

determining an evaluation demand model corresponding to the target demand according to the target evaluation demand, wherein the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model;

generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model;

operating the simulation model to obtain evaluation data of efficiency evaluation;

determining a comprehensive efficiency evaluation model according to the evaluation data;

and operating the comprehensive efficiency evaluation model to obtain an efficiency evaluation result.

Optionally, the system architecture model includes at least a plurality of an activity model of an operation or system perspective, node connection information, state transition information, event tracking information, organizational relationship information, and data information.

Optionally, the evaluation requirement model at least comprises an evaluation target sub-model, an evaluation index system sub-model, an evaluation object sub-model and an evaluation scenario sub-model, and the evaluation requirement model is used for generating a simulation script, simulating a system and evaluating the system;

wherein the evaluation target submodel is used for characterizing the target which is expected to be achieved by the evaluation;

the evaluation index system submodel is used for representing analysis and measuring evaluation indexes and the mutual relation of the evaluation indexes;

the evaluation object sub-model is used for identifying related elements required by evaluation and related constraints concerned by the evaluation on the system architecture model in a reference or labeling mode;

the evaluation scenario submodel is used for characterizing system deployment situations and action sequences required for executing simulation reaching the evaluation target.

Optionally, the evaluation scenario decider model at least comprises an operation task, an operation organization, a parameter pool, an external event set and an adaptive adjustment activity; the operation tasks at least comprise subtasks and activities, and associations exist between the subtasks and the activities;

the operation organization at least comprises a sub-organization and nodes, and an association relationship exists between the sub-organization and the nodes;

the parameter pool at least comprises a system parameter pool and a system environment parameter pool, and each parameter pool at least comprises all parameter name value pairs corresponding to the parameter pool;

the set of external events includes at least a plurality of external events, each external event having an event name and a probability of occurrence.

Optionally, the simulation model at least comprises a network sub-model, a basic sub-model, an environment sub-model and simulation data; the network submodel is used for representing the dynamic structure of the system and at least comprises an executive body and an executive body model corresponding to the executive body, wherein the executive body model at least comprises input, output, a state set, an internal transfer function, an external transfer function and a time advance function.

Optionally, the comprehensive efficiency evaluation model includes an evaluation data specification submodel, an index system submodel, an evaluation comprehensive calculation submodel and an evaluation result submodel; wherein the content of the first and second substances,

the evaluation data specification submodel is used for representing evaluation data in the efficiency evaluation;

the index system submodel is used for representing an index system in the efficiency evaluation;

and the evaluation comprehensive calculation sub-model is used for describing efficiency evaluation operation.

Optionally, the evaluation data specification submodel includes system data information, evaluation data information, and a mapping of the system data information to evaluation data;

the index system submodel at least comprises an evaluation index system and an evaluation index, wherein the evaluation index system comprises a plurality of evaluation indexes, and the evaluation indexes are used for representing index items of performance evaluation;

the evaluation comprehensive calculation sub-model at least comprises a plurality of evaluation operator models, and the evaluation operator models comprise index mapping, normalization calculation, weight calculation, index comprehensive calculation and result analysis; wherein, the first and the second end of the pipe are connected with each other,

the index mapping is used for calculating and finishing mapping of evaluation indexes and evaluation data specification items of each leaf node of the lower layer, and the mapping scheme is calculated by a mapping operator;

the normalization calculation is used for completing the normalization processing of the evaluation index value of each leaf node of the lower layer;

the weight calculation is used for carrying out weight assignment on all leaf nodes of the lower layer;

the index comprehensive calculation is used for completing the segmentation of an index system according to different comprehensive calculation methods and the comprehensive calculation of each sub-evaluation index system;

the result analysis is used for completing the analysis of the comprehensive calculation result of the evaluation index, wherein the result analysis indication comprises range analysis and contribution degree analysis;

the evaluation result submodel is used for representing an evaluation result specification item of a single result and an evaluation result specification set consisting of the evaluation result specification items; the evaluation result specification item is used for describing the standard of the evaluation result, and comprises a result name, an evaluation index related to the result and an evaluation result value; the evaluation result specification set is used for describing a result set of one-time performance evaluation.

In a second aspect, an embodiment of the present invention provides a simulation apparatus for a performance evaluation system, where the apparatus includes:

the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for determining an evaluation demand model corresponding to a target demand according to the target evaluation demand, the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model;

the conversion module is used for generating a simulation model corresponding to the evaluation demand model by adopting an application model conversion method for the evaluation demand model;

the simulation module is used for operating the simulation model to obtain evaluation data of efficiency evaluation;

the determining module is used for determining a comprehensive efficiency evaluation model according to the evaluation data;

and the evaluation module is used for operating the comprehensive efficiency evaluation model to obtain an efficiency evaluation result.

wherein, the evaluation target submodel is used for characterizing the target expected to be reached by the evaluation;

the evaluation index system submodel is used for representing analysis and measuring evaluation indexes and the correlation thereof;

the evaluation object sub-model is used for identifying related elements required by evaluation and related constraints concerned by evaluation on the system architecture model in a reference or labeling mode;

Optionally, the evaluation scenario submodel at least comprises an operation task, an operation organization, a parameter pool, an external event set and an adaptive adjustment activity; the operation tasks at least comprise subtasks and activities, and associations exist between the subtasks and the activities;

Optionally, the simulation model at least comprises a network submodel, a basic submodel, an environment submodel and simulation data; the network submodel is used for representing the dynamic structure of the system and at least comprises an executive body and an executive body model corresponding to the executive body, wherein the executive body model at least comprises input, output, a state set, an internal transfer function, an external transfer function and a time advance function.

the evaluation data specification sub-model is used for representing evaluation data in the efficiency evaluation;

the normalization calculation is used for completing the normalization processing of the evaluation index values of all leaf nodes of the lower layer;

the evaluation result submodel is used for representing an evaluation result specification item of a single result and an evaluation result specification set consisting of the evaluation result specification items; the evaluation result specification item is used for describing the standard of an evaluation result, and comprises a result name, an evaluation index related to the result and an evaluation result value; the evaluation result specification set is used for describing a result set of one-time performance evaluation.

In a third aspect, an embodiment of the present invention provides a terminal device, including: at least one processor and memory;

the memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the performance evaluation system-oriented simulation method provided in the first aspect.

In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed, the simulation method for a performance evaluation system provided in the first aspect is implemented.

The embodiment of the invention has the following advantages:

according to the simulation method and device, the terminal device and the storage medium for the efficiency evaluation system, an evaluation demand model corresponding to a target demand is determined according to the target evaluation demand, the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model; generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model; operating the simulation model to obtain evaluation data of efficiency evaluation; determining a comprehensive efficiency evaluation model according to the evaluation data; the comprehensive efficiency evaluation model is operated to obtain an efficiency evaluation result, and the problems that a complex system lacks systematic efficiency evaluation method guidance and can be described, quantized, analyzed and executed are solved.

Drawings

In order to more clearly illustrate the embodiments or prior art solutions of the present application, the drawings used in the description of the embodiments or prior art will be briefly described below, it is obvious that the drawings in the description below are only some embodiments described in the present application, and that other drawings can be obtained by those skilled in the art without inventive labor.

Fig. 1 is a flowchart of a simulation method for a performance evaluation system according to an embodiment of the present application;

FIG. 2 is a flow chart of an adaptive simulation method for performance evaluation according to another embodiment of the present application;

FIG. 3 is a block diagram of a performance evaluation model architecture according to an embodiment of the present application;

FIG. 4 is a diagram illustrating a meta-model of an activity model according to an embodiment of the present application;

FIG. 5 is a diagram illustrating a meta-model of operation node connection description according to an embodiment of the present application;

FIG. 6 is a diagram illustrating a meta-model for state transition description in an embodiment of the present application;

FIG. 7 is a diagram of a meta-model for event tracking description in an embodiment of the present application;

FIG. 8 is a diagram illustrating a meta-model of an organizational relationship model in an embodiment of the present application;

FIG. 9 is a diagram illustrating a meta-model of a logical data model according to an embodiment of the present application;

FIG. 10 is a diagram illustrating a meta-model for evaluating a demand model according to an embodiment of the present application;

FIG. 11 is a diagram illustrating a meta-model of a proposed model in an embodiment of the present application;

FIG. 12 is a diagram illustrating meta-models of a system simulation model according to an embodiment of the present application

FIG. 13 is a diagram illustrating an embodiment of a meta model for evaluating a data specification model

FIG. 14 is a diagram illustrating meta-models of the index system model according to an embodiment of the present application

FIG. 15 is a diagram illustrating a meta-model of a performance evaluation comprehensive computation model according to an embodiment of the present application

FIG. 16 is a diagram illustrating a meta model of an evaluation result model according to an embodiment of the present application

FIG. 17 is a diagram illustrating a heterogeneous simulation data unified description framework according to an embodiment of the present application;

FIG. 18 is a diagram of a heterogeneous simulation data unified adaptation framework in yet another embodiment of the present application;

FIG. 19 is a schematic illustration of a parallel assembly according to yet another embodiment of the present application;

FIG. 20 is a schematic view of a sequential combination in yet another embodiment of the present application;

FIG. 21 is a schematic view of an embedding assembly in a further embodiment of the present application;

FIG. 22 is a schematic illustration of a selection combination in a further embodiment of the present application;

FIG. 23 is a schematic illustration of a hybrid combination according to yet another embodiment of the present application;

FIG. 24 is a schematic structural diagram of a simulation apparatus for a performance evaluation system according to an embodiment of the present application;

fig. 25 is a schematic configuration diagram of a terminal device according to the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the technical solutions of the present application will be described in detail and completely with reference to the following embodiments and accompanying drawings. It should be apparent that the described embodiments are only a few embodiments of the present application, and not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort belong to the protection scope of the present application.

An embodiment of the present invention provides a simulation method for an efficiency evaluation system, which is used for performing performance evaluation on the efficiency evaluation system to be evaluated. The execution subject of the embodiment is a simulation apparatus oriented to the performance evaluation system, and is disposed on a terminal device, for example, the terminal device at least includes a computer terminal and the like.

Referring to fig. 1, a flowchart of steps of an embodiment of a simulation method for a performance evaluation system according to the present invention is shown, and the method may specifically include the following steps:

s101, determining an assessment demand model corresponding to a target demand according to the target assessment demand, wherein the assessment demand model is used for clearly defining and analyzing an assessment object, a range, a target, an assessment index system and an assessment scenario, and the assessment object is a system structure model;

specifically, the model-driven performance evaluation method first establishes an evaluation requirement model by evaluating requirement analysis activities according to a system architecture model and an evaluation target. The assessment requirement model can clearly define and analyze assessment objects, ranges, targets, assessment index systems, assessment ideas and the like. Wherein, the evaluation object indicates the system to be evaluated and represents a system architecture model; the evaluation scope indicates the relevant information scope in the architecture model which needs to be used and involved in the evaluation; the evaluation target is used for clearly evaluating the performance index of the target system and the quantification mode of the performance index; the evaluation index system decomposes an evaluation target, divides an upper evaluation index into a series of sub-indexes, and can relatively easily perform direct measurement and calculation on leaf indexes; finally, the assessment scenario indicates the task scenario and external environment under which the target system is to be simulated and evaluated for performance, and the scenario also describes the operational tasks, the configuration of the operational units, and the changes in the external environment. Through the assessment requirement modeling activity, assessment personnel can comprehensively describe assessment requirements in a modeling mode.

S102, generating a simulation model corresponding to the assessment demand model by the assessment demand model through an application model conversion method;

after the evaluation requirement model is established, an executable simulation model is generated by applying a model conversion method, and simulation operation is carried out to obtain data required by efficiency evaluation. The designed model conversion technology can automatically convert the system architecture model according to the evaluation requirement to obtain a simulation model, and assists an evaluator in completing the final simulation system model. The model conversion technology takes static structure and dynamic behavior description in a system architecture model as input, combines evaluation scenario in an evaluation demand model and information, such as system parameters, and the like, which is supplemented by an evaluator and is necessary for simulation execution, and generates a complete and executable simulation model through the conversion technology based on multi-model fusion.

S103, operating the simulation model to obtain evaluation data of efficiency evaluation;

s104, determining a comprehensive efficiency evaluation model according to the evaluation data;

and S105, operating the comprehensive efficiency evaluation model to obtain an efficiency evaluation result.

After the simulation data required for evaluation is obtained, the performance evaluation phase is entered. This stage involves multiple modeling activities, including building an index mapping model, building an index calculation model, and building an efficiency synthesis model, with the output product of each modeling activity being the corresponding model. The index mapping model maps the evaluation indexes in the evaluation index system with the simulation data, and the index calculation model calculates the values and data normalization of the indexes through the characteristics of the indexes and the data provided by the index mapping model. And finally, the efficiency comprehensive model realizes the evaluation calculation and analysis of the whole efficiency. Through the three modeling activities, a final performance evaluation result is obtained.

(1) The model-driven system efficiency evaluation method is provided, the model supports the whole efficiency evaluation process, and the consistency and reusability of information conversion at different stages are ensured; the simulation model can be automatically generated by applying a model conversion technology, so that the simulation model can correctly reflect system design factors, the workload of simulation model development is reduced, and the credibility of simulation and evaluation results is improved; and the evaluation result can be used for tracing the evaluation requirement and the system design.

(2) A multi-view and multi-level model system is designed for efficiency evaluation, the description of concerned problems from different perspectives of designers, simulators, evaluators and the like is supported, the strict association relationship between model semantics and model elements is established, the consistency between model views and layers is ensured, and data can be obtained from various sources and conversion to a plurality of platforms can be supported.

(3) Configurable, extensible and traceable evaluation calculation is designed, storage, access and calculation of simulation data are managed in a configurable and extensible mode based on metadata and knowledge elements, flexible model combination is supported, and evaluation results can be traced back to a simulation model through the evaluation model and the simulation metadata.

According to the simulation method for the efficiency evaluation system, an evaluation demand model corresponding to a target demand is determined according to the target evaluation demand, the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model; generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model; operating the simulation model to obtain evaluation data of efficiency evaluation; determining a comprehensive efficiency evaluation model according to the evaluation data; the comprehensive efficiency evaluation model is operated to obtain an efficiency evaluation result, and the problems that a complex system lacks systematic efficiency evaluation method guidance and can be described, quantized, analyzed and executed are solved.

The present invention further provides a simulation method for a performance evaluation system provided in the above embodiments.

Optionally, the evaluation requirement model at least comprises an evaluation target sub-model, an evaluation index system sub-model, an evaluation object sub-model and an evaluation scenario sub-model, and the evaluation requirement model is used for simulation script generation, system simulation and evaluation;

the evaluation index system submodel is used for representing evaluation indexes of analysis and measurement and the mutual relation of the evaluation indexes;

the organization at least comprises a sub-organization and nodes, and the sub-organization and the nodes have an association relation;

Optionally, the simulation model at least comprises a network sub-model, a basic sub-model, an environment sub-model and simulation data; the network submodel is used for characterizing the dynamic structure of the system and at least comprises an executive body and an executive body model corresponding to the executive body, wherein the executive body model at least comprises input, output, a state set, an internal transfer function, an external transfer function and a time advance function.

Optionally, the comprehensive efficiency evaluation model comprises an evaluation data specification sub-model, an index system sub-model, an evaluation comprehensive calculation sub-model and an evaluation result sub-model; wherein the content of the first and second substances,

the evaluation comprehensive calculation sub-model at least comprises a plurality of evaluation operator models, and the evaluation operator models comprise index mapping, normalization calculation, weight calculation, index comprehensive calculation and result analysis; wherein the content of the first and second substances,

Fig. 2 is a flowchart of an adaptive simulation method for performance evaluation according to another embodiment of the present application, as follows:

the model-driven efficiency evaluation method for the information system, provided by the embodiment of the invention, emphasizes that modeling activities run through the evaluation process, and relates to multiple stages of system architecture modeling, evaluation requirement modeling, system simulation, system evaluation and the like. The work product or action object of each stage is a model, and the grammar and the semantics of the model are defined by a uniform meta-model. The model and modeling activities have clear dependency relationship, and a model conversion technology is provided to assist the construction of the model.

The model-driven efficiency evaluation method firstly establishes an evaluation demand model through evaluation demand analysis activities according to a system architecture model and an evaluation target. The assessment requirement model can clearly define and analyze assessment objects, ranges, targets, assessment index systems, assessment ideas and the like. Wherein, the evaluation object indicates the system to be evaluated and represents a system architecture model; the evaluation scope indicates the relevant information scope in the architecture model which needs to be used and involved in the evaluation; the evaluation target is used for determining the performance index of the evaluation target system and the quantification mode thereof; the evaluation index system decomposes an evaluation target, divides an upper evaluation index into a series of sub-indexes, and can relatively easily carry out direct measurement and calculation on leaf indexes; finally, the assessment scenario indicates the task scenario and external environment under which the target system is to be simulated and evaluated for performance, and the scenario also describes the operational tasks, the configuration of the operational units, and the changes in the external environment. Through the assessment requirement modeling activity, assessment personnel can comprehensively describe assessment requirements in a modeling mode.

After the evaluation requirement model is established, an executable simulation model is generated by applying a model conversion method, and simulation operation is carried out to obtain data required by efficiency evaluation. The designed model conversion technology can automatically convert the system architecture model according to the evaluation requirement to obtain a simulation model, and assists an evaluator to complete the final simulation system model. The model conversion technology takes static structure and dynamic behavior description in a system architecture model as input, combines evaluation scenario in an evaluation demand model and information, such as system parameters, and the like, which is supplemented by an evaluator and is necessary for simulation execution, and generates a complete and executable simulation model through the conversion technology based on multi-model fusion.

Based on the activities of the three stages, the performance evaluation of the information system is supported to be carried out in an iterative increment mode. By analyzing the evaluation model, data and results obtained in each iteration cycle, guidance can be provided for the improvement of the model or the calculation method in the later cycle. This process can be repeated until the assessment results have been obtained to the confidence level required by the assessor (or other stopping criteria).

Multi-level, multi-view, quantifiable and analyzable performance evaluation model system

The efficiency evaluation model system defines models and the interrelation thereof used by the information system-oriented efficiency evaluation method, and in practice, the models can be tailored or customized according to factors such as the characteristics and the evaluation requirements of a specific system, the technical means adopted for evaluation and the like. Therefore, the study of the performance evaluation model system will focus on the completeness and versatility of the model. Fig. 2 shows the hierarchy and relationship of the performance evaluation model system proposed in the application embodiment.

FIG. 3 is a block diagram of a performance evaluation model architecture according to an embodiment of the present application; the efficiency evaluation model system comprises 4 layers which are respectively a system structure model, an evaluation demand model, a system simulation model and a system evaluation model. Each level of the model comprises a plurality of model views or products, and the model views or products support describing elements, semantics and constraints of the model from different angles. The system architecture model is the input to the model-driven performance evaluation method and is also an important component of the evaluation model system. In the embodiment of the exploration research application, the activity model, the node connection description, the state transition description, the event tracking description, the organizational relationship model, the data model and the like of the operation or system view are focused on at the system architecture level.

The evaluation demand model specifically comprises 4 models of an evaluation target, an evaluation index system, an evaluation object and an evaluation scenario, and has important significance for guiding the generation of a simulation script, the system simulation and the development of evaluation in the subsequent links of the model-driven efficiency evaluation method. Wherein the evaluation target model describes a target to be evaluated and expected to be achieved, and the evaluation index system describes evaluation indexes and mutual relations thereof which are analyzed and measured in order to achieve the evaluation target; the evaluation object model describes relevant elements required by evaluation and relevant constraints concerned by evaluation on an input model, namely an architecture model, in a reference or labeling mode; the evaluation intent describes the system deployment and sequence of actions required to simulate to achieve the evaluation goals. Under the guidance of the evaluation demand model, a simulation script generation technology is applied, the system architecture model can be automatically converted to generate a simulation model, and simulation data required by system evaluation is generated through the operation of the simulation model. The simulation model describes the operation flow of the system, relevant uncertainty factors in the operation process and relevant data labels needing to be collected. The simulation data model defines the relevant data and the interrelationship thereof which need to be provided in the operation process of the simulation model.

Under the support of simulation data and evaluation requirements, system evaluators can analyze and process the data according to the evaluation model, and define, calculate and synthesize indexes, thereby implementing efficiency evaluation. The evaluation model comprises a detailed evaluation index system, an evaluation data specification model, an evaluation comprehensive calculation model and an evaluation result model. The evaluation index system is a refinement result of the evaluation index system proposed in the evaluation demand model, and is required to have definite dimension, index-to-data mapping and calculation mode definition; the evaluation comprehensive calculation model defines the hierarchical relationship and the comprehensive mode among various indexes, so that the efficiency evaluation can be carried out according to an evaluation index system and simulation data; the evaluation result model displays the evaluation and calculation results in an intuitive mode matched with the evaluation index dimension, so that an evaluator can observe and analyze conveniently; the evaluation data specification model describes the specifications of all data depending on evaluation calculation and result display which are developed according to an evaluation index system, including the types of the data, the relationship among the data, the value restriction of the data and the like.

All of the above models in the performance assessment model system comply with the grammatical, semantic and relational constraints specified by the uniformly defined meta-model. The meta-model is used for defining the evaluation model system in a standard and strict way, so that the standardization of a plurality of models and modeling activities in the model system is ensured, the theoretical foundation is laid for quantitative and systematic efficiency evaluation, and the technical foundation is laid for realizing rapid and strict conversion between models by applying a model conversion theory.

Meta-model design for performance evaluation model

The meta model itself is also a model, which is an example of a meta model. The meta-model itself must also be expressed in a precisely defined language so that semantically unambiguous and rigorous model instances can be constructed therefrom. The model and the meta model have relative relationship, the meta model is abstraction and definition of the model, and the syntax, the semantics and the constraint of the model are defined; the model is an externalization and instantiation of the meta-model, which must comply with the syntactic, semantic, and related constraints defined in the meta-model. The embodiment of the application conforms to MOF (Meta Object facility) meta-modeling specification of OMG (Object Management group) organization in terms of meta-model definition. The MOF specification suggests a model description system of four-layer structure, with the top layer being the meta-model layer, the M3 layer model. The M3 model is the language in which MOFs build meta-models (called M2 models). M2 is a meta-model layer that describes the elements of the M1 layer, and the most typical example of the M2 model is the UML meta-model, which defines UML syntax and semantics. M1 is a model layer, such as a UML model, and the last layer is an M0 or data layer model, which often describes real-world entities and their interrelationships. The above hierarchical relationships are relative, and the elements in any one hierarchy are defined by the upper layer, and the upper layer model can be regarded as the meta model of the lower layer. Meanwhile, the MOF is also a closed structure, and the meta-model of the M3 layer is itself, that is, the M3 model can define not only the M2 model but also itself. The meta-model design of each type of model in the performance evaluation model system is described in detail below.

Meta-model of system architecture model

In the aspect of system architecture modeling, the description and definition of elements such as system activities, node connection, state transition, event tracking, organizational relationships, logical data and the like are mainly concerned. For space-limited, the detailed explanation of each meta-model element and its constraints is referred to the relevant documents such as meta-model design.

The activity model corresponds to the design guide of the architecture of the electronic information system in China and OV-5 products in DoDAF, describes various activities (tasks) normally carried out in the process of executing missions or acquiring business capacity and input and output (I/O) flows among the activities, further describes input and output flows among activities outside the system architecture, and is designed as a meta model shown in FIG. 4. Wherein, the operation activity (operation activity) represents a series of actions or behaviors for realizing mission task, the input (In) identifies the input of the outer layer system, the output (Out) identifies the output of the layer, and the input/output information flow (DataFlow) identifies the input and output of information between the operation activities; FIG. 4 is a diagram illustrating a meta-model of an activity model according to an embodiment of the present application;

FIG. 5 is a diagram illustrating a meta-model of operation node connection description according to an embodiment of the present application; the node connection description corresponds to the structural design guide of the electronic information system in China and OV-2 and SV-2 in DoDAF, the structure and the mutual relation of the system are described from the operation view and the system view respectively, and FIG. 5 shows the meta-model design of the node connection description of the operation view. The operation node (OperationNode) describes a node for executing a command or a task, the operation activity (operationactivity) represents a series of actions or behaviors for realizing a mission task, the external node (ExternalNode) represents an operation node which is not in the system architecture, a requirement line (needleline) is a logic representation for information interaction needs of a pair of nodes, and information exchange (informationexchange) is an explanation of information interacted between operation nodes.

FIG. 6 is a diagram illustrating a meta-model for state transition description in an embodiment of the present application; the state transition description corresponds to an electronic information system architecture design guide and OV-6b and SV-11b products in DoDAF, describes an operation activity or clear action sequence inside the system from the operation viewpoint and the system viewpoint respectively, and takes a state machine diagram as the basis, and the meta model design of the state machine diagram is shown in FIG. 6. Wherein, State represents a State element, Transition represents Transition between states and Transition conditions thereof, and Start and End represent Start and End states, respectively.

FIG. 7 is a diagram of a meta-model for event tracking description in an embodiment of the present application; the event tracking description corresponds to the architecture design guide of the electronic information system in China and OV-6c and SV-11c in DoDAF, information exchange between operation/system nodes (participating in the operation/system nodes) is described according to the time sequence from the operation view and the system view respectively, and the meta model design is as shown in FIG. 7 on the basis of a sequence diagram. The system node (system node) represents a node for executing a task or a mission, PLine represents a lane line, CallMessage represents that required information is obtained through calling, SendMessage represents that the node sends out information, and Message represents information transfer between nodes.

FIG. 8 is a diagram illustrating a meta-model of an organizational relationship model in an embodiment of the present application; the organizational relationship corresponds to the structural design guide of the electronic information system in China and OV-4 in DoDAF, explains the command structure or relationship among various roles, organizations and organizational types of people (main participants in the architecture), and the meta-model design is shown in FIG. 8. Wherein Organization (Organization) represents a task-related Organization, Organization type is the type of Organization, operation role (Organization role) represents an operator with specific skills in the Organization, and Organization relationship (relationship) identifies the Organization with which the association exists.

FIG. 9 is a diagram illustrating a meta-model of a logical data model according to an embodiment of the present application; the logical data model corresponds to the structural design guide of the electronic information system architecture in China and OV-7 in DoDAF, describes the system data type structure in the architecture field and the structured business rule for managing system data in the operation view, is a key element for supporting the interoperation between architectures, and has SV-11 corresponding to OV-7 in the system view, and the design of the meta model is shown in FIG. 9. Wherein, Entity represents an Entity, relationship represents the relationship between entities, and Attribute represents the attribute of the Entity.

FIG. 10 is a diagram illustrating a meta-model for evaluating a demand model according to an embodiment of the present application; the meta-model evaluation demand model of the evaluation demand model comprises an evaluation target model, an evaluation index system, an evaluation object model and an evaluation scenario model. The evaluation target model describes a target expected by the performance evaluation, and the refinement and decomposition of the target, the index system model describes indexes required to be evaluated for achieving the evaluation target and the relation between the indexes, the evaluation object model describes the system structure model elements involved in model conversion and performance evaluation, and the evaluation scenario model describes external configuration and action sequence driving simulation operation. The meta-model design for evaluating the demand model is shown in FIG. 10. The evaluation demand model (EvaRequirement) comprises an evaluation target model (EvaGoal model), an evaluation index system (EvaIndex), an evaluation object model (EvaObjectmodel) and an evaluation Scenario (EvaCenarrio model), wherein the evaluation target model is composed of a series of gradually-refined evaluation targets (EvaGoal), the evaluation index system is composed of a series of hierarchical Hoard evaluation indexes (indicators), the evaluation object model is described by evaluation object (EvaObject) elements, and the evaluation Scenario is described by Scenario (Scenario) elements.

FIG. 11 is a diagram illustrating a meta-model of a proposed model in an embodiment of the present application; the evaluation scenario (evaluationscene) includes the operation task (CombatTask), the operation Organization (Organization), the parameter pool (ParameterPool), the external event set (ExternalEventSet), and the adaptive adjustment activity (adaptiveadjust). The operation task can be divided into subtasks and activities, and the subtasks can be composed of subtasks and activities, and there is an association between the subtasks and the activities. The organization can be further refined into sub-organizations and nodes, the sub-organizations can be composed of the sub-organizations and the nodes, and the association relationship exists between the sub-organizations and the nodes. The parameter pool comprises a system parameter pool and a system environment parameter pool, and each parameter pool comprises all parameter name value pairs corresponding to the parameter name value pair. The external event set is composed of external events, each having its event name and probability of occurrence.

Fig. 12 is a schematic diagram of a meta-model of a system simulation model according to an embodiment of the present application, which proposes a design of the meta-model of the system simulation model based on a dynamic structure discrete event system specification (DEVS), as shown in fig. 12. The system simulation model comprises a network model (NetworkModel), a base model (BasicModel), an environment model (environmental model) and simulation data (simullationsData). The network model is used for describing a dynamic structure of the system and comprises an executive body (network executive) and a corresponding model (executive model) of the executive body, wherein the executive body model comprises an Input (Input), an Output (Output), a state set (StateSet), an internal transfer function (InternalTransition), an external transfer function (ExternalTransition) and a time advance function (TimeAdvance). The state set is composed of a series of states, each state is associated with a structural model (structural model), and the change of the dynamic structure of the system is realized through the change of the state. Each structural model is composed of a series of basic models and their mutual influence relations. Each basic model is similar to an executive model, but each state is a basic state and no corresponding structural model exists. The environment model describes the external environment of the system, and the adaptive adjustment rule describes how the system model dynamically changes according to the environmental change, which will be explained in detail in the following research on the adaptive simulation method. The simulation data is simpler and is the name-value pair of the simulation object.

The system evaluation model comprises an evaluation data specification model, an index system model, an evaluation comprehensive calculation model and an evaluation result model. The evaluation data specification model describes evaluation data in the efficiency evaluation, the index system model describes an index system in the efficiency evaluation, the system efficiency evaluation model describes specific efficiency evaluation operations including index mapping, normalization, index comprehensive calculation, result analysis and the like, specific evaluation tasks are completed through specific combinations of the evaluation operations, and the results of the calculation and the analysis of the efficiency evaluation are stored through the evaluation model and are displayed in different modes.

FIG. 13 is a diagram illustrating a meta-model for evaluating a data specification model according to an embodiment of the present application; the evaluation data specification model mainly comprises a description of system data, a description of evaluation data and a mapping of system data to evaluation data, and the meta model design thereof is shown in fig. 13. The system data describes various system simulation data collected in simulation execution, and describes attribute value pairs and a set of system specifications (SysDataFormat) through a SysDataFormat item; the evaluation data describes evaluation data used in the performance evaluation, describes attribute value pairs by an evaluation data specification item (EvaDataItem), and evaluates a set of data specifications (evadataformat set) description; since the system data and the evaluation data are not completely in one-to-one correspondence or need to be obtained through calculation, a data analysis (dataperse) model is adopted to describe the association relationship between the two.

Fig. 14 is a schematic diagram of a meta-model of an index system model in an embodiment of the present application, in which two items, namely, an evaluation index system (EvaIndexSystem) and an evaluation index (EvaIndex), are mainly used. Wherein the evaluation index describes specific index items of the performance evaluation, and the evaluation indexes are combined to form an evaluation index system. Meanwhile, the evaluation index system forms an evaluation index.

FIG. 15 is a diagram illustrating a meta-model of a performance evaluation comprehensive computation model according to an embodiment of the present application; the method mainly supports the completion of performance evaluation calculation and comprises a plurality of evaluation operator models, including index mapping (IndexMapping), normalization calculation (IndexCommCalculate), weight calculation (Calweight), index comprehensive calculation (IndexCalculate) and result analysis (ResultParse).

The index mapping calculation completes the mapping of the evaluation indexes of each leaf node of the lower layer and the evaluation data specification items, and the mapping scheme is supported by a mapping calculation operator, such as an average difference operator;

normalization calculation is completed to normalize the evaluation index values of all leaf nodes of the lower layer, and a normalization scheme is supported by a normalization operator, such as linear normalization, segmented normalization and the like;

the weight calculation supports unified weight assignment for all leaf nodes of the lower layer, and the weight calculation scheme is supported by weight operators, such as a dichotomy, an artificial assignment method and the like;

the index comprehensive calculation support completes the segmentation of an index system and the comprehensive calculation of each sub-evaluation index system according to different comprehensive calculation methods, wherein the comprehensive calculation scheme is supported by index comprehensive calculation operators, such as a fuzzy evaluation method, an analytic hierarchy process and the like;

and the result analysis completes the analysis of the comprehensive calculation result of the evaluation index, wherein the result analysis scheme is supported by a result analysis model, such as range analysis, contribution degree analysis and the like. And describing corresponding identification, name, interface description, function description and concrete operator implementation in evaluation operators (mapping operator (MappingProcess), normalization operator (CommProcess), weight operator (WeightCalProcess), index comprehensive calculation operator (IndexCalProcess) and result analysis operator (ResultParseProProcess)).

FIG. 16 is a diagram illustrating a meta-model of an assessment results model according to an embodiment of the present application; mainly comprises an evaluation result specification item (EvaresultFormatItem) for describing a single result and an evaluation result specification set (EvaresultFormatSet) consisting of the evaluation result specification items. The evaluation result specification item is used for describing the standard of the evaluation result, and comprises a result name, an evaluation index related to the result and an evaluation result value; the evaluation result specification set describes a result set of a performance evaluation. By defining the evaluation result model, different angle display of the evaluation result can be supported.

FIG. 17 is a diagram illustrating a heterogeneous simulation data unifying description framework according to an embodiment of the present application; the model combination method based on the ontology meta-model description is a key technology for realizing the construction of a model-driven efficiency evaluation system based on the evaluation model combination of the ontology meta-model. And the component integration of the evaluation system is completed in a mode of model combination. The method describes the structure of an evaluation system such as simulation data, an evaluation index system, an evaluation calculation method, an evaluation analysis method, an evaluation display method and the like by using models, and the models are constructed by using ontology meta-models as conventions and have syntactic information and semantic information. The model combination method based on ontology meta-model description is used for constructing an evaluation system from the existing evaluation model, takes an evaluation task as guidance, adopts a top-down and layer-by-layer decomposition method, firstly retrieves and selects a corresponding model from a model library according to the evaluation requirement and the task, analyzes according to the functions and combination properties of the model, selects a proper model combination mode, finally strictly matches the model on the structure, the behavior, the interface, the semantics and the dynamic interaction behavior according to a model grammar matching rule and a semantic matching rule, adapts a reasonable model and correctly assembles the model, realizes the expected function of the system, and finally completes the construction of the evaluation system.

According to the embodiment of the application, the semantic ambiguity problem in model driving is solved in a mode of the local voxel model, so that the on-demand dynamic construction of a more effective evaluation system is supported. Therefore, the technology of combining the semantic-based heterogeneous data processing, the ontology meta-model-based evaluation calculation model description and the semantic-based evaluation model is a key technology of the embodiment of the application, and each key technology is explained in detail below.

Semantic-based heterogeneous data processing technology

Because the simulation data generated by different simulation systems have great differences in type and structure, how to uniformly describe, organize and manage heterogeneous simulation data is achieved, the heterogeneity of various simulation data in various layers such as type, format, content and the like is solved, a uniform simulation data acquisition mode is provided for various evaluation models, and the method is a basis and a precondition for realizing rapid and dynamic combination of models.

In order to solve the influence of the heterogeneity of simulation data on model combination, the core is to design a unified and extensible simulation data unified description framework, so that all layers (such as description information, format information, access information, content information and semantic information) of heterogeneous simulation data can be uniformly described in a unified standard manner, and support is provided for automatic filtering, dynamic conversion and flexible combination of models of subsequent simulation data.

The heterogeneous simulation data unified description framework mainly comprises two parts, namely a core layer and an extension layer. The core layer mainly comprises four types of metadata information, namely simulation data description metadata, simulation data format metadata, simulation data access control metadata and simulation data content related metadata. The simulation data description metadata mainly describes summary information of the simulation data, such as information including which type of simulation system the simulation data is generated from, a publisher of the data, keywords of the content, and the like; the simulation data format metadata is mainly used for describing the type and format of simulation data, for example, for simulation data stored in a database, information such as a related library table structure and field meaning needs to be described; the simulation data content metadata is mainly used for describing contained core content and can provide support for simulation data filtering; the simulation data access metadata mainly comprises information such as access modes, interfaces, parameters and the like of the simulation data.

On one hand, the extension layer comprises simulation data semantic metadata based on the knowledge ontology, and in addition, an extension function of the metadata is provided, so that a user can extend metadata required by the user according to needs, such as the knowledge ontology and the like.

The heterogeneous simulation data unified description framework is an important component of a model combination method based on ontology meta-model description, is used for uniformly describing heterogeneous simulation data generated by different simulation systems, is used for solving the heterogeneous foundation of the simulation data, and provides support for realizing automatic analysis, on-demand filtering and dynamic conversion of various heterogeneous simulation data.

On the basis of the heterogeneous simulation data uniform description framework, a uniform extensible heterogeneous simulation data adaptation framework needs to be designed to realize the analysis of various types of simulation data (such as XML types, data message types, database types and text types), and the simulation data obtained by analysis is converted into a uniform standard format for storage and management. Fig. 18 is a schematic diagram of a heterogeneous simulation data unified adaptation framework in another embodiment of the present application.

The heterogeneous simulation data unified adaptation framework adopts an open service-oriented technical system, and in order to realize unified scheduling of various adapters, an access adapter unified management framework is designed so as to realize unified control and centralized management of various heterogeneous simulation data adapters. In addition, by defining the uniform interface specification of the adapter, a user is allowed to develop the required adapter in a self-defining way, and the adapter is allowed to be added into the adapter uniform management framework for uniform management and scheduling, so that the framework has higher integration flexibility, adaptability and expandability.

Evaluation calculation model description technology based on ontology meta-model

Due to the objective existence of a large number of evaluation operators, although the consistency of grammar is achieved through the construction of the models, semantic information does not exist among the models, so that the problems of mutual non-comprehension, repeated definition, inaccurate use method and the like exist among a large number of models, and therefore the ontology meta-model technology is applied to semantically guide the models.

Because the combinability of the grammar cannot guarantee that the meanings of the interactive data and the information are understood consistently by both parties, the specific meanings of each element in the grammar model need to be further defined strictly, namely, semantic hierarchy description (hereinafter referred to as semantic model) of the evaluation model is established. The semantic model mainly refers to static semantic description of the model, namely, the meaning of each syntactic element of the model in the actual use or used process is not involved, the semantic information is implied by the model and is irrelevant to whether the model interacts with the outside world or not, when the model interacts with the outside world and the interaction sequence, so that the semantic information is also the naming reason of the static semantics of the model, namely, the semantic information of the model has a static retention characteristic. The semantic model mainly comprises semantic constraints of each input and output interface and model state variables (attributes) of the model. Based on the semantic model, a simulation model combination property analysis mechanism of a semantic level can be established, and the combinability of the semantic model is analyzed.

The semantic information is mainly divided into: concept semantics and functional semantics. Concept semantics mainly focuses on the description of the relationship between concept classifications and concepts, such as the hierarchy of concepts, the inclusion, division, causal relationship between concepts, and the like. The semantic model is an extension of the model, which is distinguished by the explicit description of the meaning of various resources in the model library, making it possible for these models to be understood by machines. Under the supporting action of the ontology and the description logic, the model is easier to retrieve and match, so that the rapid and efficient combination is convenient to realize. From the perspective of the ontology, the concept semantics can be regarded as associating the concepts described by the ontology with symbolic representations of the model, thereby realizing formalization of the model concept semantics. Functional semantics generally refer to the functional description of a model, i.e., the behavior laws or behavior constraints implied by the model interface.

And for the evaluation operator model, an ontology language is adopted to describe the meta model. The ontology language description meta-model is a heavyweight ontology meta-modeling method, and in the process of constructing the meta-model, the concept of a certain descriptive ontology is directly introduced as the meta-class of the ontology. The meta-model defined by the method is directly connected with the ontology model, and the evolution of the ontology model can directly influence the meta-model. Therefore, a language (OEL) similar to OWL-s is defined to describe the evaluation operators, including a set of ontologies, providing a vocabulary of evaluation operators to describe, which can be inferred from the Inputs (Inputs), outputs (outputs), preconditions (preconditions) and results (Effects) of the operators. OEL has machine understandability and ease of use, supporting the discovery of intelligent agent automated, dynamic evaluation operators.

OEL divides the description of one Evaluation Operator (Evaluation Operator) into three parts: operator contour, operator base point and operator model.

1) Operators (Evaluation Operator) are abstractions of Operator concepts, with one Operator top-level ontology instance corresponding to one existing Evaluation Operator entity.

2) The operator outline describes what the operator can do, and the search agent determines whether to find a satisfactory evaluation operator according to the content provided by the operator outline. The operator profile includes three parts: firstly, registration information of an operator provider, such as a contact way of the operator provider; secondly, the functional information of the operator mainly refers to the IOPR (Input, Output, Precondition, Result) of the operator; finally, the operator profile may provide a classification, parameters, quality information, etc. to which the operator belongs. The operator outline has the biggest characteristic of two-way, an operator provider can describe the function of the operator by the operator outline, and an operator using requester can also describe the requirement by the operator outline.

3) The operator model tells the user how to use this operator. To describe in detail how operators operate, operator models treat each evaluation operator as a Process (Process). Therefore, the operator model is also referred to as a process model. There are three processes in the process model: atomic processes (Atomic processes), Simple processes (Simple processes), and Composite processes (Composite processes). The atomic process can be directly called, and can be completed in one step from the perspective of an operator requester; simple procedures may not be directly invoked, nor have a corresponding grouping. Simple procedures are used for abstraction, which can be used to provide a view of an atomic procedure or a simplified representation of a compound procedure; the composition process may be composed of other atomic or composition processes through some control constructors that are typical in programming languages, such as Sequence (Sequence), selection (If-Then-Else), loop (Repeat-Until), and so on.

4) The operator base points describe how the operator should be invoked. Operator outlines and operator models are abstract descriptions of operators, and operator base points are concrete specifications relating to operators. Briefly, it describes how operators are accessed. Specifically, it needs to specify the calling interface, communication mode, port, etc. of the operator.

Evaluation model combination technology based on semantics

A user can combine evaluation models (index mapping, normalization, weighting, comprehensive evaluation calculation, result analysis and the like) through a visual modeling platform to form a process combined model. The semantic combination model is used for guiding operations such as refinement, interface semantic matching query, operator binding and performability analysis of the process combination model, and in order to support the behaviors, the model combination mode and the model combination rule are analyzed.

1) Model combination mode

Model assembly is the process of assembling models to create a system, and assembly is not a simple operation involving the mutual integration of models and model infrastructure. The essential characteristic of the evaluation model combination is to establish correlation between the models and further coordinate the behaviors of the models to organize the models into an organic whole. The mode of constructing the evaluation system by assembling the evaluation model provides a plurality of combination modes, and facilitates the model combination process. According to the interdependence and interaction relation of the evaluation model in the combination process, the combination model comprises a parallel combination mode, a sequential combination mode, an embedded combination mode, a selection combination mode, a mixed combination mode and the like, and various choices are provided for the model assembly process.

FIG. 19 is a schematic diagram of a parallel combination in a further embodiment of the present application; the parallel combination mode means that the models participating in combination respectively have independent operation control threads, and the cooperative work is realized through an interaction protocol in the combination process.

FIG. 20 is a schematic view of sequential combinations in a further embodiment of the present application; the sequential combination mode means that the evaluation models cooperate with each other in a manner similar to two processing procedures before and after the production line. The model of sequential combination has sequential operational relationships that execute sequentially as shown in FIG. 20. The sequential combination of a and B is noted: and (A) and (B). The execution characteristics are as follows: when the input event occurs, A is called to execute, and the external input event can also be used as the input of B at the moment, but B does not participate in the operation at the moment. And (4) outputting the result after the calculation of the A is finished as the input of the B, simultaneously handing over the operation control to the B, then operating the B until the output result is obtained, and transferring the operation control out. The sequential combination pattern is similar to a workflow, with different models being used to complete problems at different stages in the evaluation task, and then the complete task is achieved by sequential combination.

FIG. 21 is a schematic view of an embedding assembly in a further embodiment of the present application; embedded combination patterns are typically used to describe functional or structural "outsourced" or "delegated" relationships between evaluation models, and may also describe "whole-part" relationships between models. The operation mechanism of the embedding combination is shown in fig. 21.

The operation mechanism is as follows: a is called to be executed when an input event occurs, A can delegate a part of a calculation task to B in the running process, at the moment, A provides input parameters of B, B outputs a result to A after calculation is completed, and A then carries out the rest calculation tasks. In the embedded combination mode, B is transparent to the caller of A, i.e. the caller of A does not know whether B is present or not, and B only works in the context provided by A, so logically B appears to be part of A as a whole.

FIG. 22 is a schematic illustration of a selection combination in a further embodiment of the present application; selecting a combination mode refers to performing the function of one of the evaluation models according to the context parameters, thereby forming a more functional model, as shown in fig. 22. The operation mechanism is as follows: and selecting A or B to execute according to the context environment parameters, the input event type and other information, and completely transferring the operation control right to the model selected to execute.

FIG. 23 is a schematic illustration of a hybrid combination according to yet another embodiment of the present application; the hybrid combination mode refers to that different combination modes are mixed for use so as to meet more complex combination requirements, and then a more powerful evaluation model is constructed. There are a variety of hybrid combination modes, one typical hybrid mode being sequential and embedded hybrid modes.

Different model combination types can be adopted according to different standards and actual requirements. In the process of one-time evaluation model combination, one combination mode can be adopted, and multiple combination modes can also be adopted. The model combination aims to meet the variable evaluation requirements, namely, the models are quickly reconstructed or combined as the evaluation requirements change.

2) Rules of composition

The selection and combination of model operators are the key for realizing the combination of the models. And establishing an analysis mechanism of model combination by constructing a combination matching rule between the model grammar and the model semantics. The specific combination rules are described in table 1:

TABLE 1 combination rules

Compatibility and replaceability are two main aspects of the combinability of the investigation model, and an analysis mechanism of semantic combination property is established by constructing compatibility rules and replaceability rules among the models.

It should be noted that for simplicity of description, the method embodiments are shown as a series of combinations of acts, but those skilled in the art will recognize that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.

Another embodiment of the present invention provides a simulation apparatus for a performance evaluation system, which is used to execute the simulation method for the performance evaluation system provided in the above embodiment.

Referring to fig. 24, a block diagram of an embodiment of a simulation apparatus for a performance evaluation system according to the present invention is shown, and the apparatus may specifically include the following modules: an obtaining module 2401, a converting module 2402, a simulating module 2403, a determining module 2404 and an evaluating module 2405, wherein:

the obtaining module 2401 is configured to determine, according to a target evaluation requirement, an evaluation requirement model corresponding to the target requirement, where the evaluation requirement model is used to explicitly define and analyze an evaluation object, a range, a target, an evaluation index system, and an evaluation scenario, and the evaluation object is a system architecture model;

the conversion module 2402 is configured to generate a simulation model corresponding to the evaluation requirement model by using an application model conversion method for the evaluation requirement model;

the simulation module 2403 is used for operating the simulation model to obtain evaluation data of efficiency evaluation;

the determining module 2404 is used for determining a comprehensive efficiency evaluation model according to the evaluation data;

the evaluation module 2405 is configured to run the performance evaluation comprehensive model to obtain a performance evaluation result.

According to the simulation device for the efficiency evaluation system, provided by the embodiment of the invention, the evaluation demand model corresponding to the target demand is determined according to the target evaluation demand, the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model; generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model; operating the simulation model to obtain evaluation data of efficiency evaluation; determining a comprehensive efficiency evaluation model according to the evaluation data; the comprehensive efficiency evaluation model is operated to obtain an efficiency evaluation result, and the problems that a complex system lacks systematic efficiency evaluation method guidance and can be described, quantized, analyzed and executed are solved.

The present invention further provides a simulation apparatus for a performance evaluation system provided in the above embodiments.

Optionally, the comprehensive efficiency evaluation model comprises an evaluation data specification sub-model, an index system sub-model, an evaluation comprehensive calculation sub-model and an evaluation result sub-model; wherein, the first and the second end of the pipe are connected with each other,

and the evaluation comprehensive computation submodel is used for describing efficiency evaluation operation.

Optionally, the evaluation data specification sub-model includes system data information, evaluation data information, and a mapping of the system data information to the evaluation data;

the index system submodel at least comprises an evaluation index system and evaluation indexes, wherein the evaluation index system comprises a plurality of evaluation indexes, and the evaluation indexes are used for representing index items of performance evaluation;

For the device embodiment, since it is basically similar to the method embodiment, the description is simple, and for the relevant points, refer to the partial description of the method embodiment.

The simulation device for the performance evaluation system provided by the embodiment of the invention obtains the evaluation requirements to be evaluated for the performance evaluation system, wherein the evaluation requirements at least comprise an evaluation task, an evaluation condition and an evaluation target to be achieved; determining a simulation acquisition list according to the evaluation requirement, wherein the simulation acquisition list at least comprises parameter information of a plurality of acquisition points; determining simulation models corresponding to the plurality of acquisition points according to the parameter information of the plurality of acquisition points, so that the simulation models can simulate at the acquisition points to obtain simulation results; receiving a simulation result returned by the simulation model, comparing the simulation result with an evaluation requirement, performing self-adaptive adjustment on the simulation model according to the comparison result, establishing the simulation model with dynamic adjustment behavior description capability, supporting a simulation data acquisition technology of self-adaptive adjustment acquisition points and a self-adaptive construction technology of simulation excitation, continuously improving an output result, and further realizing optimization of the performance of the simulation system.

Still another embodiment of the present invention provides a terminal device, configured to execute the simulation method for a performance evaluation system provided in the foregoing embodiment.

Fig. 25 is a schematic structural diagram of a terminal device of the present invention, and as shown in fig. 25, the terminal device includes: at least one processor 2501 and memory 2502;

the memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the simulation method for the performance evaluation system provided by the above embodiments.

According to the terminal device provided by the embodiment, the evaluation demand model corresponding to the target demand is determined according to the target evaluation demand, the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model; generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model; operating the simulation model to obtain evaluation data of efficiency evaluation; determining a comprehensive efficiency evaluation model according to the evaluation data; the comprehensive efficiency evaluation model is operated to obtain an efficiency evaluation result, and the problems that a complex system lacks systematic efficiency evaluation method guidance and can be described, quantized, analyzed and executed are solved.

Yet another embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the simulation method oriented to the performance evaluation system provided in any of the above embodiments is implemented.

According to the computer-readable storage medium of the embodiment, an evaluation demand model corresponding to a target demand is determined according to the target evaluation demand, the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system architecture model; generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model; operating the simulation model to obtain evaluation data of efficiency evaluation; determining a comprehensive efficiency evaluation model according to the evaluation data; the comprehensive efficiency evaluation model is operated to obtain an efficiency evaluation result, and the problems that a complex system lacks systematic efficiency evaluation method guidance and can be described, quantized, analyzed and executed are solved.

It should be noted that the above detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present application. As used herein, the singular is intended to include the plural unless the context clearly indicates otherwise. Furthermore, it will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein are capable of operation in other sequences than those illustrated or otherwise described herein.

Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, article, or apparatus.

Spatially relative terms, such as "above … …," "above … …," "above … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". The device may also be oriented in other different ways, such as by rotating it 90 degrees or at other orientations, and the spatially relative descriptors used herein interpreted accordingly.

In the foregoing detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, like numerals typically identify like components, unless context dictates otherwise. The illustrated embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented here.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A simulation method for a performance evaluation system, the method comprising:

determining an evaluation demand model corresponding to a target demand according to the target evaluation demand, wherein the evaluation demand model is used for clearly defining and analyzing an evaluation object, a range, a target, an evaluation index system and an evaluation scenario, and the evaluation object is a system structure model;

and operating the efficiency evaluation comprehensive model to obtain an efficiency evaluation result.

2. The method of claim 1, wherein the system architecture model includes at least a plurality of activity models of operational or system perspectives, node connection information, state transition information, event tracking information, organizational relationship information, and data information.

3. The method of claim 2, wherein the evaluation requirement model at least comprises an evaluation target sub-model, an evaluation index system sub-model, an evaluation object sub-model and an evaluation scenario sub-model, and the evaluation requirement model is used for simulation script generation, system simulation and evaluation;

4. The method of claim 3, wherein the evaluation scenario submodel includes at least an operational task, an operational organization, a parameter pool, a set of external events, and an adaptive adjustment activity; the operation tasks at least comprise subtasks and activities, and associations exist between the subtasks and the activities;

5. The method of claim 1, wherein the simulation model includes at least a network submodel, a base submodel, an environment submodel, and simulation data; the network submodel is used for characterizing the dynamic structure of the system and at least comprises an executive body and an executive body model corresponding to the executive body, wherein the executive body model at least comprises input, output, a state set, an internal transfer function, an external transfer function and a time advance function.

6. The method of claim 1, wherein the performance evaluation composite model comprises an evaluation data specification sub-model, an index system sub-model, an evaluation composite calculation sub-model, and an evaluation results sub-model; wherein the content of the first and second substances,

7. The method of claim 6, wherein the evaluation data specification submodel includes system data information, evaluation data information, and a mapping of system data information to evaluation data;

the index mapping is used for calculating and completing the mapping of the evaluation indexes and the evaluation data specification items of each leaf node of the lower layer, and the mapping scheme is calculated by a mapping calculation operator;

8. A simulation apparatus for a performance evaluation system, the apparatus comprising:

the conversion module is used for generating a simulation model corresponding to the evaluation demand model by using an application model conversion method for the evaluation demand model;

9. A terminal device, comprising: at least one processor and memory;

the memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the performance evaluation system oriented simulation method of any of claims 1-7.

10. A computer-readable storage medium, in which a computer program is stored, which, when executed, implements the performance evaluation system-oriented simulation method of any one of claims 1 to 7.