CN112883527B - Simulation evaluation method for network system - Google Patents
Simulation evaluation method for network system Download PDFInfo
- Publication number
- CN112883527B CN112883527B CN202110279983.7A CN202110279983A CN112883527B CN 112883527 B CN112883527 B CN 112883527B CN 202110279983 A CN202110279983 A CN 202110279983A CN 112883527 B CN112883527 B CN 112883527B
- Authority
- CN
- China
- Prior art keywords
- information
- simulation
- timeliness
- evaluation
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/18—Network design, e.g. design based on topological or interconnect aspects of utility systems, piping, heating ventilation air conditioning [HVAC] or cabling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/02—CAD in a network environment, e.g. collaborative CAD or distributed simulation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Geometry (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computational Mathematics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computer Networks & Wireless Communication (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention relates to the technical field of information system architecture design, in particular to a simulation evaluation method for a network system, which comprises the following steps: s1, analyzing the evaluation requirement of the system structure design scheme; s2, establishing a system structure simulation evaluation index system and a calculation model; s3, establishing a system structure simulation system; s4, designing a simulation evaluation test scheme; s5, developing a simulation evaluation test; and S6, acquiring a simulation evaluation result. The scheme has the advantages that the evaluation on the network system is more detailed, the detail reliability of the evaluation data is high, the statistical significance is realized, the specific requirements of optimization and upgrading of the network system are met, the result obtained based on the scheme evaluation has better universality, and the method is suitable for popularization and application.
Description
Technical Field
The invention relates to the technical field of information system architecture design, in particular to a simulation evaluation method for a network system.
Background
The system structure verification is to check the correctness of the system structure design and determine whether the system structure description meets the requirements of the system and the degree of meeting the requirements. Unreasonable and even contradictory designs can be discovered and timely improved through the verification of the system structure, and the design quality is improved. At present, the research of the architecture verification theory and method mainly focuses on two directions: firstly, verifying the content, namely researching and verifying what, and verifying the architecture from which level and which aspects; the second is a verification method: i.e. how to verify, what way and means should be used for verifying the architectural data for different levels and different aspects.
At present, the structural design of an information system mostly adopts an architecture framework, and the architecture framework can standardize the design of the architecture and is easy to understand and communicate. However, since the architecture framework only specifies the design content and form of the architecture, how to design the architecture reasonably mainly depends on the experience of the designer. For a complex information system, because the system has many constituent units, complex functions and structures, and complex dynamic processes and information flows, the quality of the system structure is difficult to ensure only by the personal experience of designers, therefore, the system structure must be verified and evaluated on the basis of the system structure design, and the system structure must be continuously optimized according to the evaluation result, so as to ensure the correctness and feasibility of the system structure design, meet the military requirements of the system, reduce the design risk, and reduce the development cost.
The existing evaluation method, such as the literature 'command information system survivability simulation research' is based on an ANP network analysis method, and has strong subjectivity and poor result reliability. The literature, namely the military communication network efficiency simulation evaluation method based on the SEA evaluation operator, adopts the SEA method, so that the historical data is large in demand, and the universality of the result is not enough based on a deduction scheme.
Disclosure of Invention
The invention provides a simulation evaluation method for a network system, which solves the technical problems of low simulation precision and low reliability of the traditional network system.
The invention provides a simulation evaluation method facing to a network system for solving the technical problem, which comprises the following steps:
s1, analyzing the evaluation requirement of the system structure design scheme;
s2, establishing a system structure simulation evaluation index system and a calculation model;
s3, establishing a system structure simulation system;
s4, designing a simulation evaluation test scheme;
s5, developing a simulation evaluation test;
and S6, acquiring a simulation evaluation result.
Optionally, the indicators in S2 include high efficiency, robustness, agility, and task performance
Optionally, the high efficiency specifically includes information guarantee timeliness, information sharing timeliness and command control timeliness;
the timeliness of the information guarantee is used for measuring whether the information processing unit, the decision control unit and the terminal combat unit can obtain corresponding information guarantee information in time;
the information sharing timeliness is particularly information subscription/distribution timeliness and situation sharing timeliness;
the information subscription/distribution timeliness is used for measuring whether an information user in an information system can acquire related information services in time through an information subscription request;
the situation sharing timeliness is used for measuring the time length of the situation shared by each decision control unit and the terminal combat unit;
the command control timeliness is specifically control information timeliness and action synergy timeliness;
the timeliness of the control information is used for measuring whether the decision control unit and the terminal combat unit can obtain the control information in time;
the action cooperative timeliness is used for measuring that the information acquisition unit and the terminal combat unit can acquire cooperative information in time.
Optionally, the S3 specifically includes: when the high-efficiency effect of the system structure is simulated and evaluated, the simulation model is used for simulating the information flow model process of transmission and processing between systems, the simulation is mainly focused on the time delay effect generated in the information processing and interaction process, but not on the real and specific information processing and transmission activity, and a computer simulation method is specifically adopted, and the functional performance simulation model of the system unit and the information transmission is built and operated.
Optionally, when the robustness effect of the system structure is simulated and evaluated, a prototype or semi-physical simulation model is used to simulate the implementation process and technical principle of the system unit damage, succession and resource borrowing functions, and the success rate and timeliness indexes are tested and evaluated.
Optionally, when the agility effect of the system structure is simulated and evaluated, the dynamic access, dynamic combination and system structure adjustment function implementation processes and technical principles of the system unit are simulated by using prototype and semi-physical simulation modeling methods, and success rate and timeliness indexes are tested and evaluated.
Optionally, when the task performance of the architecture is simulated and evaluated, the simulation model first needs to simulate the task environment and conditions of the system for resisting the task, and also needs to simulate the functional performance of each unit and the relationship between the units under the driving of the task environment, and finally simulates the state and behavior of the system in the process of completing the task, so as to obtain the experimental data for evaluating the task performance.
Optionally, the S3 specifically includes: determining a system composition unit and a relation simulation model, determining a system operation environment simulation model, designing an engagement rule and an operation process simulation model, and constructing a simulation system based on the models.
Optionally, the S4 specifically includes: designing test scenario, selecting collected test data, designing test times, and ensuring simulation confidence level.
Optionally, the S5 specifically includes: and (4) loading a test scenario, and collecting test data.
Optionally, after S6, the method further includes: and adjusting the design scheme of the architecture according to the evaluation result.
Has the advantages that: the invention provides a simulation evaluation method facing to a network system, which comprises the following steps: s1, analyzing the evaluation requirement of the system structure design scheme; s2, establishing a system structure simulation evaluation index system and a calculation model; s3, establishing a system structure simulation system; s4, designing a simulation evaluation test scheme; s5, developing a simulation evaluation test; and S6, acquiring a simulation evaluation result. The scheme has the advantages that the evaluation of the network system is more detailed, the detail reliability of the evaluation data is high, the statistical significance is higher, the specific requirements of optimization and upgrading of the network system are met, the result obtained based on the scheme evaluation has better universality, and the method is suitable for popularization and application.
The foregoing description is only an overview of the technical solutions of the present invention, and in order to make the technical solutions of the present invention more clearly understood and to implement them in accordance with the contents of the description, the following detailed description is given with reference to the preferred embodiments of the present invention and the accompanying drawings. The detailed description of the present invention is given in detail by the following examples and the accompanying drawings.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention and do not constitute a limitation of the invention. In the drawings:
FIG. 1 is a schematic flow chart of a simulation evaluation method for a network architecture according to the present invention;
fig. 2 is a simulation evaluation index system diagram of the simulation evaluation method for the network system of the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth to illustrate, but are not to be construed to limit the scope of the invention. The invention is described in more detail in the following paragraphs by way of example with reference to the accompanying drawings. Advantages and features of the present invention will become apparent from the following description and from the claims. It is to be noted that the drawings are in a very simplified form and are not to precise scale, which is merely for the purpose of facilitating and distinctly claiming the embodiments of the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When a component is referred to as being "connected" to another component, it can be directly connected to the other component or intervening components may also be present. When a component is referred to as being "disposed on" another component, it can be directly on the other component or intervening components may also be present. The terms "vertical," "horizontal," "left," "right," and the like are used herein for purposes of illustration only.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
As shown in fig. 1, the present invention provides a simulation evaluation method for a network architecture, which comprises the following steps: s1, analyzing the evaluation requirement of the system structure design scheme; s2, establishing a system structure simulation evaluation index system and a calculation model; s3, establishing a system structure simulation system; s4, designing a simulation evaluation test scheme; s5, developing a simulation evaluation test; and S6, acquiring a simulation evaluation result. The scheme has the advantages that the evaluation of the network system is more detailed, the detail reliability of the evaluation data is high, the statistical significance is higher, the specific requirements of optimization and upgrading of the network system are met, the result obtained based on the scheme evaluation has better universality, and the method is suitable for popularization and application.
The basic principle of the simulation evaluation of the architecture is to evaluate the advantages and disadvantages of the system units and the relationships by simulating the behaviors, the states and the task completion effects of the system units and the relationships under specific confrontation tasks and environments, and the specific flow of the simulation evaluation method for the network architecture adopted by the embodiment of the invention is shown in fig. 1. Similar to a general simulation evaluation process, the basic process of the architecture simulation evaluation includes: the method comprises the steps of evaluating requirement analysis of a system structure design scheme, establishing a simulation evaluation index system and a calculation model, establishing a system structure simulation system, designing a simulation evaluation test scheme, developing a simulation evaluation experiment, obtaining a simulation evaluation result and the like.
Optionally, the analysis of the requirement of the system structure design scheme comprises analysis of system composition units and related relations, analysis of system functional structure and task completion efficiency, and analysis of characteristics of the system such as high efficiency, robustness and agility; establishing a system structure simulation evaluation index system and a calculation model, wherein the method comprises the steps of establishing a system structure simulation evaluation index hierarchical structure, determining high efficiency, robustness, agility and task efficiency indexes, and determining each index calculation model and weight distribution; establishing a system structure simulation system, wherein the system structure simulation system comprises a system composition unit and a relationship simulation model, a system operational environment simulation model, a design engagement rule and an operational process simulation model, and a simulation system is established based on the model; designing a simulation evaluation experiment scheme, wherein the design simulation evaluation experiment scheme comprises designing an experiment, selecting collected experiment data and design experiment times, and ensuring the confidence level of a simulation result; the developing of the simulation evaluation experiment comprises loading experiment scenario, collecting experiment data and monitoring the experiment process; and acquiring a simulation evaluation result comprises acquiring test data, calculating an evaluation result and carrying out comparative analysis on the evaluation result. After the simulation evaluation result is obtained, the design scheme of the architecture can be optimized and adjusted according to the evaluation result.
Alternatively, the architecture simulation index system is built from 4 aspects of high efficiency, robustness, agility and task efficiency, as shown in fig. 2. The high efficiency comprises information guarantee timeliness, information sharing timeliness, command control timeliness and the like; the robustness comprises the structural survivability of the system, the information guarantee variation degree, the information sharing variation degree, the command control variation degree and the like; agility includes configuration capability, access capability, adjustment capability, and the like; the task performance includes situation awareness performance, command decision performance, and action control performance.
According to the optional scheme, the high efficiency index can be evaluated through the time characteristics of information flow motifs such as simulation information, control and cooperation in the process of completing tasks by the system. In the specific evaluation process, statistical calculation can be carried out based on multiple experimental results, and the reliability of the index evaluation result is improved.
The specific calculation model of the high efficiency index is as follows:
(1) information guarantee timeliness
The timeliness of the information guarantee is used for measuring whether information sink units such as the information processing unit, the decision control unit and the terminal combat unit can obtain related information guarantee information in time, is defined as the average time interval from the information source units such as the information acquisition unit and the information processing unit to the information sink unit to receive the information, and is the average value of all the information sink units in a certain evaluation time period in the system. The timeliness of the information guarantee can be specifically divided into timeliness of respectively acquiring information by an information processing unit, a decision control unit and a terminal combat unit, and an evaluation model is as follows:
take the timeliness of the information processing unit for obtaining information as an example, NInformation processing unitIndicating the total number of intelligence handling units in the system, MInformation processing unitIndicating the number of times the intelligence processing unit i receives the intelligence information in a specified evaluation period,
the time of the ith information processing unit receiving the jth information in the selected evaluation time period is shown, and the time can be obtained by stamping a time stamp of the receiving time on each received information message by the information processing unit;
the information source sends the jth information message to the information source, and the time can be obtained by the information source by stamping the time stamp of the sending time on each sent information message. In the simulation experiment process, the evaluation period may be set as the whole experiment process. Therefore, the timeliness of the information processing units for acquiring the information is the averaging of all the information acquisition timeliness of the information processing units in a certain period of time, and the timeliness of the decision control unit and the terminal combat unit for acquiring the information is similar to the timeliness of the information acquisition of the decision control unit and the terminal combat unit.
The timeliness of the information guarantee information of the whole system structure is the average of the timeliness of the information obtained by all the information processing units, the decision control unit and the terminal combat unit, and the evaluation model is as follows:
(2) timeliness of information sharing
The information sharing timeliness is composed of two parts of information subscription/distribution timeliness and situation sharing timeliness.
Information subscription/distribution timeliness
The timeliness of information subscription/distribution is used for measuring whether information users in an information system can acquire related information services in time through information subscription requests, wherein the information users comprise an information processing unit, a decision control unit, a terminal combat unit and the like, and the information users are not further classified here for simplifying an evaluation model. The timeliness of information subscription/distribution is defined as the time interval from the information user sending an information subscription request to an information provider to receiving information service feedback, wherein the information provider can be an information service center in military information infrastructure, or can be an information acquisition, information processing, decision control unit and the like capable of providing information to the outside. The timeliness of information subscription/distribution is the average value of all information users in the system in a certain evaluation period, and the evaluation model is as follows:
in the formula: n is a radical of hydrogenInformation userThe total number of information users in the system structure comprises an information processing unit, a decision control unit, a terminal combat unit and the like; mInformation user iThe total times of information subscription requests sent out by the information user i in the evaluation period;
the moment when the jth information subscription request is sent out for the ith information user in the selected evaluation period;
indicating the time when the ith information user receives the jth information service feedback.
(ii) situation-sharing timeliness
The situation sharing timeliness is used for measuring the time length of the situation shared by each decision control unit and the terminal operation unit, and is defined as averaging the time intervals from the moment information which is locally formed by one decision control unit to the moment information which is received by other decision control units or the terminal operation units. The situation sharing timeliness is that all situation information sharing behaviors are averaged in a certain evaluation period, and an evaluation model is as follows:
in the formula:
the moment of distributing situation information in the ith situation sharing behavior in the selected evaluation period;
the time when the situation information is received in the ith situation sharing behavior.
(3) Command control timeliness
The command and control timeliness consists of two parts of command and control information timeliness and action coordinated timeliness.
Firstly, the timeliness of the information is controlled
The timeliness of the control information is used for measuring whether the decision control unit, the terminal combat unit and the like can obtain the control information in time, wherein the control information refers to information such as a countermeasure scheme, a plan, an instruction and the like. The timeliness of the control information is defined as the average time interval from the time when the decision control unit sends out the control information to the time when the lower-level decision control unit or the terminal combat unit receives the control information. The timeliness evaluation model for the decision control unit and the terminal combat unit to obtain the control information is shown as the following formula:
taking the timeliness of the control information obtained by the end combat unit as an example, NResponsive execution unitRepresenting the total number of end combat units in the system; m is a group ofIn response to execution Unit iIndicating the number of times the end combat unit i has received control information in common during the specified evaluation period,
the time of the ith terminal combat unit receiving the jth control information in the selected evaluation period is shown, and the time can be obtained by the terminal combat unit marking the time stamp of the receiving time on each received control information message;
the time indicating the jth control information message is sent from the superior decision control unit can be obtained by the decision control unit by stamping the time stamp of the sending time on each sent control information message. In the simulation experiment process, the evaluation period may be set as the whole experiment process. It can be seen that the timeliness of the control information acquired by the terminal combat units is the average of the timeliness of the control information acquired by all the terminal combat units within a certain period of time, and the timeliness of the control information acquired by the decision control unit is similar to the timeliness.
The timeliness of the control information of the whole system structure is the average of the timeliness of the control information obtained by all the decision control units and the terminal combat units, and an evaluation model is as follows:
(ii) action synergy timeliness
The action cooperative timeliness is used for measuring whether the autonomous mechanism cooperative task information can be generated between the information acquisition unit and the terminal combat unit. The action cooperative timeliness is defined as the average time interval from the time when any one information acquisition unit or terminal combat unit sends out cooperative request information to the time when the cooperative response information of another information acquisition unit or terminal combat unit is successfully received. The evaluation model is as follows:
wherein is defined similarly as above: n is a radical ofEnd combat unitRepresenting the total number of end combat units in the system; mEnd combat unit iIndicating the number of times the end combat unit i sends the collaborative request within the specified evaluation period,
the time (indicating successful cooperation) when the ith terminal combat unit receives the jth cooperative response in the selected evaluation period is shown, and the time can be obtained by stamping a time stamp of the receiving time on each received information message by the terminal combat unit;
and the time when the jth cooperation request message is sent is shown, and the time can be obtained by the time stamp of the sending time of the unit.
Optionally, the robustness index may be evaluated by simulating the degree of change in system capability and the effect of completing a task in the case of failure of some system units. In the specific evaluation process, statistical calculation can be carried out based on multiple experimental results, and the reliability of the index evaluation result is improved.
The robustness index specific calculation model is as follows:
(1) system architecture survivability
The system structure survivability measures whether measures such as backup succession or resource borrowing can be taken to effectively maintain the operation efficiency of the structure under the condition that part of system units fail or degrade, and the system structure survivability measures are composed of a backup succession success rate and a resource borrowing success rate.
Backup succeed rate
The backup replacing means that the backup unit can successfully replace the original system unit to work through means such as data synchronization, communication network recombination and the like under the condition that the system unit works abnormally. For example, for a key system, corresponding preparation systems can be established in different regions, and when an original system works abnormally, task succession is completed through starting of a backup system, directory updating, data synchronization, information distribution relation updating and the like. The backup take-over success rate is defined as the proportion of the times of successfully completing the backup take-over to the total times of generating the backup take-over, and the evaluation model is as follows:
second success rate of resource borrowing
The resource borrowing means that the system unit borrows resources of other units to complete the tasks of the unit through resource scheduling under the condition that the efficiency of the system unit is reduced or the tasks exceed the expected capacity, and the resources comprise computing resources, software service resources and the like. The resource borrowing success rate is defined as the proportion of the times of successfully completing the resource borrowing to the total times of borrowing requests, and the evaluation model is as follows:
(2) information guarantee variation degree
The intelligence guarantee variation degree measures the variation degree of intelligence guarantee capability under the condition that part of system units fail or degrade. The method comprises two parts of target detection capability and information processing capability variation degree.
Target detection capability variation degree
The target detection capability is specifically measured through a system detection coverage area and a target coverage coefficient of a system responsibility area.
The system detection coverage is a joint detection coverage of all information acquisition units in the system, that is, a union of detection coverage, which can be expressed as:
in the formula: cInformation acquisition unit iIn the mood of changesThe coverage of the detection side of the information acquisition unit i, and N is the number of the information acquisition units.
The coverage factor of an object refers to the number of intelligence acquisition units that can simultaneously detect the object in space. The target coverage coefficient refers to the average of the coverage coefficients of all targets in space, and can be expressed as
In the formula: lijPresentation information acquisition unit OiTo target MjDistance of (L)iPresentation information acquisition means OiMaximum detection distance of whenij≤LiPresentation information acquisition unit OiCapable of detecting an object MjThen S isi(Mj) 1, otherwise Si(Mj) Where 0, O is the set of all information acquisition units, M is the set of all targets, and n is the number of targets.
The degree of change in the system detection coverage can be defined as: when a plurality of information acquisition units lose suburb/efficiency, the system detects the descending degree of the coverage area. The calculation model is
The degree of change of the target coverage coefficient within the system responsibility area can be defined as: and when the information acquisition units fail/degrade, the degree of degradation of the target coverage coefficient of the system responsibility area. The calculation model is
The target detection capability variation degree can be calculated by adopting a weighting method, and the evaluation model is as follows:
Cdegree of change in detectivity＝W1×αDegree of coverage variation+W2×αDegree of coverage coefficient change
Information processing capability variation degree
The intelligence processing capability includes capacity and time delay of system processing intelligence. The information processing capacity variation degree can adopt a calculation method similar to the target detection capacity variation degree, and the calculation model is
(3) Degree of change in information sharing
The information sharing variation measure the variation degree of the information sharing capability under the condition that part of the system units fail or degrade, and is composed of the variation degree of the information subscribing/distributing capability and the variation degree of the situation information sharing capability.
Information subscription/distribution capability change degree
The information subscription/distribution capability variation degree is defined as: and when a plurality of system units fail/degrade, the system subscribes/distributes the degradation degree of timeliness.
Situation information sharing capability change degree
The situation information sharing capability variation degree is defined as: when several system units fail/degrade, the situation shares the degradation degree of timeliness.
(4) Degree of variation of command control
The command control change degree measures the change degree of command control ability under the condition that part of system units fail or degrade, and can be measured from the aspects of situation perception ability change degree, control information timeliness change degree, action synergy timeliness change degree and the like.
Firstly, the change degree of the situation perception capability
The degree of change in situational awareness is defined as: when a plurality of system units fail/lose effectiveness, the degree of decline of the overall situation perception range, situation sharing effectiveness and other indexes of the system can be expressed as:
second degree of change of the control information with time
The time-dependent change degree of the command information is defined as follows: when a plurality of system units fail/degrade, the degradation degree of the timeliness of the system control information can be expressed as
Thirdly, action synergy timeliness change degree
The degree of change of action-synergy timeliness is defined as: when a plurality of system units fail/degrade, the information acquisition unit or the terminal combat unit acts cooperatively and the degradation degree of timeliness can be expressed as
Alternatively, the agility index may be evaluated by adapting the simulated system architecture to changes in the task and environment. The specific task and environment change can represent the change in various aspects such as dynamically accessing a new unit, dynamically combining units according to tasks, adjusting the functions of the units, and processing the business process flow and information interaction relationship among the units. In the specific evaluation process, the condition change stimuli (access new unit, random combination unit, unit function adjustment and the like) can be designed based on a simulation experiment system, and phenomena and behavior characteristic data (time, success identification and the like) in the agility aspect of the system adapting to task and environment change are collected and recorded.
The agility index is specifically calculated as follows:
(1) configuration capability
The configuration capability measures the capability of flexibly configuring the system unit and the relationship between the units, and consists of two parts, namely the success rate of system configuration and the configuration time.
(I) configuration success rate
The configuration success rate refers to the proportion of the times of completing the unit capacity or unit relation configuration and realizing the normal operation of the system to the total times of the system configuration, and the evaluation model is as follows:
(ii) allocation time
The configuration time comprises configuration time of system units and configuration time of relationships among the units, and the evaluation model is as follows:
(2) access capability
Whether the access capacity measuring unit can successfully access the system is automatically identified by the system and quickly integrated into the system to work, and the access capacity measuring unit consists of two parts, namely access success rate and access time.
First, access success rate
The access success rate is defined as the proportion of the times of successful access to the total times of access requests, and the evaluation model is as follows:
access time-
The access time refers to the average time consumed by a unit to successfully access the system, and the evaluation model is
In the formula: t is ti(request access) is the access request time, tiThe (successful access) is the moment of successful access.
(3) Combining ability
The combining capability measures whether various types of resources (resources in system units) distributed on the information system can be dynamically combined into a task community according to task requirements. The task community is a temporary system formed by various resources such as battlefield perception, information processing, confrontation command, weapon control and the like on an information system dynamically organized around a task target. The combination capability consists of two parts, namely combination success rate and combination time.
Combining to form power
The combination success rate is defined as the proportion of the times of successful combination to the total times of request combination, and the evaluation model is as follows:
② combination time
The combination time is defined as the time for constructing a task community by various resources on various information infrastructures, the processes of task decomposition, business process arrangement, resource query and matching, task community operation parameter configuration and generation and the like are involved from the beginning of task receiving, and the evaluation model is
TDynamic combination time＝ΔtTask decomposition+ΔtBusiness process orchestration+ΔtResource query matching+ΔtOperating parameter configuration
(4) Capability of adjustment
The adjustment capability measures whether the system structure unit and the relation can respond in time according to the change of the external operation environment and the internal operation state, and relates to the adaptive adjustment of the operation parameters, the working mode, the behavior rules and the like of the system unit. The adjusting capability consists of two parts of adaptive adjusting power and adjusting time.
Firstly, the power is adjusted
The adjustment success rate is defined as the proportion of the times of successful adjustment to the total times of requested adjustment, and the evaluation model is as follows:
② adjusting time
The adjustment time is defined as the time interval from the change of the system task or the external environment to the completion of the targeted adjustment of the system structure, and the evaluation model is as follows:
Tadjusting time＝tAdjusting the end of a behavior-tTask/environment changes
On the basis of the four agility evaluations, the agility index of the system can be comprehensively evaluated. Adopting a weighted comprehensive evaluation method, and calculating models respectively
PSuccess rate＝W1×αConfiguration success rate+W2×αSuccess rate of access+W3×αSuccess rate of combination+W4×αAdjusting success rate
TAging property＝W1×TConfiguring time+W2×TTime of access+W3×TTime of combination+W4×TAdjusting time
In the selectable scheme, the task efficiency is a comprehensive index for evaluating whether the system structure meets the design requirements from the aspect of task completion, and is also a key index for integrally evaluating the quality of the design scheme of the system structure. According to the definition of the efficiency, the following conclusion can be obtained that aiming at different systems, different task efficiency evaluation indexes need to be established, and a universal task efficiency evaluation index and a universal task efficiency evaluation model do not exist; ② for the same system. The degree to which a system completes a task or function can be measured in terms of time, quality, efficiency, and the like; many information systems are integrated systems formed by integrating multiple functional systems, and there are multiple indexes for representing their efficiency.
Without loss of generality, the task of the military information system is to provide accurate situation information, efficient command and decision and smooth action control. Therefore, the task performance of the scheme is defined as the combination of situation perception performance, command decision performance and action control performance.
(1) Situational awareness
The situation perception efficiency comprehensively measures the capability and effect of the system in the aspects of detecting, tracking and identifying the battlefield daily standard, and generally can be measured by the completeness, correctness, accuracy and timeliness of a perception target.
Situation integrity
The integrity is used for measuring the degree of real targets contained in the perceived target set, and t is defined as the proportion of the number of the targets perceived in the battlefield to the total number of the real targets. The integrity of the system perception target at time t, i (t), is expressed as:
in the formula: btRepresenting the number of real objects contained in the set of objects perceived by the system at time t, BtRepresenting the true target total at time t.
The perception situation is updated for K times in the whole evaluation period, and the duration of each time is tkThen the integrity of the average can be expressed as
Situation correctness
The situation correctness is used for measuring the degree of the objects with the correctly recognized identities in the sensing object set, and is defined as the proportion of the number of the sensing objects with the correctly recognized identities to the total number of the real objects. The correctness of the system perception target at the time t is expressed as V (t)
In the formula：ctNumber of sensing targets C representing correct identification of identity in sensing situation at time ttRepresenting the total number of real targets in the unified situation at the moment t of the system.
The perception situation is updated for K times in the whole evaluation time period, and the duration of each time is tkThen the correctness of the average can be expressed as
Situation accuracy
The situation accuracy is used for measuring the degree that the target measurement accuracy of battlefield perception meets the use requirements of users, and is divided into position accuracy and speed accuracy, wherein the speed accuracy can be subdivided into speed magnitude accuracy and speed direction accuracy. The accuracy index can also adopt a multi-threshold method to determine the index value, and the key point is to determine the upper threshold value and the lower threshold value of the acceptable measurement granularity according to the user requirement. The position accuracy P of the target i at the time ti(t) is represented by
In the formula: xi,tIndicates the position measurement accuracy (deviation of the position measurement value from the true position of the target) of the target i at time t, X0And XMRespectively preset upper and lower thresholds, 0 < XM＜XM-1＜…＜X1The accuracy of all target positions at the moment of < 1 is calculated by
In the formula: n is the target total number at time t. The perception situation is updated for K times in the whole evaluation time period, and the duration of each time is tkThen the accuracy of the average can be expressed as
Situation timeliness
The situation timeliness is used for measuring the degree of time required by the sensor to detect the target and form the situation target. Timeliness S of target i at time t of target feature p of interest to systemi,p(t) can be expressed as:
in the formula: i. t and p represent the time interval between the characteristic p of the concerned battlefield target i at the moment T and the acquisition of the perception information, T0And TMRespectively preset upper and lower thresholds, 0 < TM＜TM-1＜…＜T1Is less than 1. the calculation formula of the effectiveness of the system on the characteristics p of all targets at the time t is
In the formula: n is the total number of targets at the moment t.
The perception situation is updated for K times in the whole evaluation time period, and the duration of each time is tKThen the average timeliness for the target feature p can be expressed as
(2) Efficiency of command decision
The comprehensive performance of the command and decision measures the capability and effect of the system in aspects of situation analysis, task solution, countermeasure scheme/plan generation and the like, and can be generally measured by a command control cycle and a scheme integrity rate.
Command control period
The time sum of tracking, identification, task allocation and countermeasure plan/instruction generation is measured by averaging all enemy target OODA (detection, control, fighting and evaluation ring) periods. The command control period is calculated by the formula
In the formula: t isiOODA period against enemy target i; and N is the total number of enemy targets.
(ii) completeness of scheme
The scheme integrity is defined as the integration of the completeness and the availability of documents such as a combat scheme, a combat plan and the like generated by the decision control unit, and the calculation model is as follows:
(3) action control efficiency
The comprehensive action control efficiency measures the capability and effect of the system in aspects of weapon platform (system) control, terminal combat unit autonomous combat and the like, and can be generally measured by weapon control success rate and autonomous cooperative success rate.
Weapon control success rate
The weapon control success rate refers to the ratio of the number of successful weapon control times to the total number of weapon control times, the successful weapon control refers to the control of the weapon platform (system) state to the designated activity state, and the calculation formula is
In the formula: m is total weapon control times; m is a group ofsControls the success times for the weapon.
(ii) autonomous cooperative success rate
The autonomous cooperative success rate refers to the ratio of the number of times of autonomous initiation success of cooperative fighting units to the total number of applications, the autonomous system success refers to sending the applications from the cooperative initiator, and within the time limit of the cooperative fighting window, the terminal unit receives or rejects all tasks properly, and the calculation formula is
In the formula: x is the total number of autonomous synergies; xsThe number of autonomous cooperative success times.
In the optional scheme, a system structure simulation system is established, and the function and the advantage of simulation can be fully played only by selecting a proper simulation modeling method and technology according to a simulation task, an object and a target. Therefore, aiming at different simulation evaluation contents and data requirements of high efficiency, robustness, agility and task performance of the system structure scheme, the requirements on the fidelity of the simulation model and the construction method of the simulation system are different.
(1) When the high efficiency evaluation of the system structure is simulated, the simulation model needs to simulate the process of transmitting and processing various information flow models between systems, and the simulation focuses on the time delay effect generated in the information processing and interaction process rather than the real and specific information processing and transmission activities. Therefore, a computer simulation method can be adopted, and the process and the time delay of unit processing time delay and information interaction between units can be simulated more realistically by establishing and operating a functional performance simulation model of the system unit and information transmission.
(2) When the robustness evaluation of the system structure is simulated, the indexes such as success rate, timeliness and the like of the system can be tested and evaluated only by simulating the excitation, the specific implementation process and the like of the survivability technology by a system unit model. Therefore, a prototype or semi-physical simulation model is required to meet the requirements of the experiment. On the other hand, however, when the effect of the crash countermeasure is evaluated by simulation, i.e. the fluctuation of the system capability or performance, the system unit is focused on simulating the functional performance thereof, and a computer simulation modeling method can be adopted.
(3) When the agility evaluation of the architecture is simulated, the simulation system needs to be able to simulate the function implementation processes and technical principles of system unit dynamic access, dynamic combination, system structure adjustment, and the like. Therefore, when the realization principle of the system structure flexibility is subjected to simulation evaluation, prototype and semi-physical simulation modeling methods are required.
(4) When the task efficiency is simulated and evaluated, the simulation model firstly simulates the task environment and conditions of the system for resisting the task, and also simulates the functional performance of each unit and the relationship among the units under the driving of the task environment, and finally simulates the state and the behavior of the system in the process of completing the task, so as to obtain the experimental data for evaluating the structural efficiency of the system. Therefore, the system structure performance simulation evaluation mainly relates to the simulation of the system task environment, conditions and functional performance, and a computer simulation modeling method can be adopted.
The foregoing is illustrative of the preferred embodiments of the present invention, and is not to be construed as limiting the invention in any way; the present invention may be readily implemented by those of ordinary skill in the art as illustrated in the accompanying drawings and described above; however, those skilled in the art should appreciate that they can readily use the disclosed conception and specific embodiments as a basis for designing or modifying other structures for carrying out the same purposes of the present invention without departing from the scope of the invention; meanwhile, any changes, modifications, and evolutions of the equivalent changes of the above embodiments according to the actual techniques of the present invention are still within the protection scope of the technical solution of the present invention.
Claims (6)
1. A simulation evaluation method for a network system is characterized by comprising the following steps:
s1, analyzing the evaluation requirement of the system structure design scheme;
s2, establishing a system structure simulation evaluation index system and a calculation model; the indexes comprise high efficiency, robustness, agility and task efficiency;
the high efficiency is particularly intelligence guarantee timeliness, information sharing timeliness and command control timeliness;
the timeliness of the information guarantee is used for judging whether the information processing unit, the decision control unit and the terminal combat unit can obtain corresponding information guarantee information in time;
the information sharing timeliness is specifically information subscription/distribution timeliness and situation sharing timeliness;
the information subscription/distribution timeliness is used for measuring whether an information user in an information system can acquire related information services in time through an information subscription request;
the situation sharing timeliness is used for measuring the time length of the situation shared by each decision control unit and the terminal combat unit;
the command control timeliness is specifically control information timeliness and action synergy timeliness;
the timeliness of the control information is used for measuring whether the decision control unit and the terminal combat unit can obtain the control information in time;
the action cooperative timeliness is used for measuring that the information acquisition unit and the terminal combat unit can acquire cooperative information in time;
when the robustness effect of the system structure is simulated and evaluated, a prototype or semi-physical simulation model is adopted to simulate the realization process and the technical principle of the damage, succession and resource borrowing functions of the system unit, and the success rate and the timeliness index are tested and evaluated;
when the agility effect of the system structure is simulated and evaluated, the dynamic access, dynamic combination and system structure adjustment function realization processes and technical principles of the system unit are simulated by using a prototype and semi-physical simulation modeling method, and success rate and timeliness indexes are tested and evaluated;
s3, establishing a system structure simulation system;
s4, designing a simulation evaluation test scheme;
s5, developing a simulation evaluation test;
and S6, acquiring a simulation evaluation result.
2. The simulation evaluation method for a network architecture according to claim 1, wherein the S3 specifically includes: when the high-efficiency effect of the system structure is simulated and evaluated, the simulation model is used for simulating the information flow simulation process of transmission and processing between systems, the simulation is mainly focused on the time delay effect generated in the information processing and interaction process, but not on the real and specific information processing and transmission activity, and a computer simulation method is specifically adopted, and the functional performance simulation model of the system unit and the information transmission is built and operated.
3. The network system-oriented simulation evaluation method according to claim 1, wherein when performing simulation evaluation on the task performance of the architecture, the simulation model firstly simulates the task environment and conditions of the system for confronting tasks, and also simulates the functional performance of each unit and the relationship between units under the driving of the task environment, and finally simulates the state and behavior of the system in the process of completing tasks, so as to obtain the experimental data for evaluating the task performance.
4. The network architecture-oriented simulation evaluation method according to claim 1, wherein the S4 specifically includes: designing test scenario, selecting collected test data, designing test times, and ensuring simulation confidence level.
5. The network architecture-oriented simulation evaluation method according to claim 1, wherein the S5 specifically includes: and (4) loading test scenarios and collecting test data.
6. The simulation evaluation method for a network architecture oriented system according to claim 1, wherein after the S6, the method further comprises: and adjusting the design scheme of the architecture according to the evaluation result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110279983.7A CN112883527B (en) | 2021-03-16 | 2021-03-16 | Simulation evaluation method for network system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110279983.7A CN112883527B (en) | 2021-03-16 | 2021-03-16 | Simulation evaluation method for network system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112883527A CN112883527A (en) | 2021-06-01 |
| CN112883527B true CN112883527B (en) | 2022-07-15 |
Family
ID=76042681
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110279983.7A Active CN112883527B (en) | 2021-03-16 | 2021-03-16 | Simulation evaluation method for network system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112883527B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114826967B (en) * | 2022-06-29 | 2022-10-11 | 中国人民解放军国防科技大学 | Information sharing capability evaluation method and device |
| CN115130332A (en) * | 2022-08-30 | 2022-09-30 | 中电太极(集团)有限公司 | Simulation method and device for efficiency evaluation system |
| CN115963731B (en) * | 2023-03-16 | 2023-05-30 | 南京信息工程大学 | Command control system network structure optimization method based on improved genetic algorithm |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101620645A (en) * | 2009-08-17 | 2010-01-06 | 王钰 | Method and system of large-scale simulation electronic information system architecture |
| CN103278828A (en) * | 2013-05-23 | 2013-09-04 | 桂林电子科技大学 | Satellite navigation novel signal system simulation system and method and satellite navigation novel signal system performance evaluation system and method |
| CN104865844A (en) * | 2015-03-26 | 2015-08-26 | 中国电子科技集团公司第五十四研究所 | Distributed platform-based spatial information network comprehensive evaluation demonstration method |
| CN105825022A (en) * | 2016-03-24 | 2016-08-03 | 中国人民解放军装甲兵工程学院 | Multi-Agent-based dynamic equipment security system efficiency evaluation method and system |
| CN112131755A (en) * | 2020-09-30 | 2020-12-25 | 中国人民解放军国防科技大学 | Method and device for designing architecture capability evaluation test scheme |
| CN112150008A (en) * | 2020-09-25 | 2020-12-29 | 中国人民解放军国防科技大学 | System structure evaluation method and system based on design data and experimental data |
| CN112199842A (en) * | 2020-11-11 | 2021-01-08 | 中国电子科技集团公司第二十八研究所 | Task-oriented-based complex simulation system reliability evaluation method |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2014056558A1 (en) * | 2012-10-12 | 2014-04-17 | Technische Universität Dortmund | Computer implemented method for hybrid simulation of power distribution network and associated communication network for real time applications |
| US10986602B2 (en) * | 2018-02-09 | 2021-04-20 | Intel Corporation | Technologies to authorize user equipment use of local area data network features and control the size of local area data network information in access and mobility management function |
-
2021
- 2021-03-16 CN CN202110279983.7A patent/CN112883527B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101620645A (en) * | 2009-08-17 | 2010-01-06 | 王钰 | Method and system of large-scale simulation electronic information system architecture |
| CN103278828A (en) * | 2013-05-23 | 2013-09-04 | 桂林电子科技大学 | Satellite navigation novel signal system simulation system and method and satellite navigation novel signal system performance evaluation system and method |
| CN104865844A (en) * | 2015-03-26 | 2015-08-26 | 中国电子科技集团公司第五十四研究所 | Distributed platform-based spatial information network comprehensive evaluation demonstration method |
| CN105825022A (en) * | 2016-03-24 | 2016-08-03 | 中国人民解放军装甲兵工程学院 | Multi-Agent-based dynamic equipment security system efficiency evaluation method and system |
| CN112150008A (en) * | 2020-09-25 | 2020-12-29 | 中国人民解放军国防科技大学 | System structure evaluation method and system based on design data and experimental data |
| CN112131755A (en) * | 2020-09-30 | 2020-12-25 | 中国人民解放军国防科技大学 | Method and device for designing architecture capability evaluation test scheme |
| CN112199842A (en) * | 2020-11-11 | 2021-01-08 | 中国电子科技集团公司第二十八研究所 | Task-oriented-based complex simulation system reliability evaluation method |
Non-Patent Citations (2)
| Title |
|---|
| "指挥信息系统体系结构建模与仿真研究平台设计";程安潮;《第19届中国系统仿真技术及其应用学术年会论文集（19th CCSSTA 2018）》;20180810;第167-171页 * |
| "指控网络PHM系统及其关键技术研究";许卫强 等;《电光与控制》;20170511;第24卷(第4期);第46-50,66页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112883527A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112883527B (en) | Simulation evaluation method for network system | |
| Han et al. | A hybrid intelligent model for assessment of critical success factors in high-risk emergency system | |
| Pouryazdan et al. | Quantifying user reputation scores, data trustworthiness, and user incentives in mobile crowd-sensing | |
| Pouryazdan et al. | The smart citizen factor in trustworthy smart city crowdsensing | |
| CN109144724A (en) | A kind of micro services resource scheduling system and method | |
| US7894357B2 (en) | Capability-based testing and evaluation of network performance | |
| Bahutair et al. | Multi-perspective trust management framework for crowdsourced IoT services | |
| CN110298601A (en) | A kind of real time business air control system of rule-based engine | |
| CN109670653A (en) | A kind of method and device predicted based on industrial model predictive engine | |
| CN103577876A (en) | Credible and incredible user recognizing method based on feedforward neural network | |
| Raz et al. | System of systems uncertainty quantification using machine learning techniques with smart grid application | |
| CN104363104A (en) | User-requirement-oriented mass multivariate data situation displaying system and method | |
| CN108415775A (en) | A kind of quality-of-service based resource determining method and system | |
| CN117763555A (en) | Power distribution network data safety protection and evaluation method based on block chain | |
| Shin et al. | Gaussian radial basis functions for simulation metamodeling | |
| Zhang et al. | A comparison of evaluation methods for police patrol district designs | |
| Jung et al. | Domain mapping matrix-based metric for measuring system design complexity | |
| Caldeira et al. | Trust based interdependency weighting for on-line risk monitoring in interdependent critical infrastructures | |
| Khalil et al. | Multi-agent crisis response systems-design requirements and analysis of current systems | |
| Blazevic et al. | University of Things: Opportunities and Challenges for a Smart Campus Environment based on IoT Sensors and Business Processes. | |
| Luo et al. | A Structural Complexity Metric Method for Complex Information Systems. | |
| CN114036570A (en) | Dynamic evaluation method and system for information sensitivity of smart power grid | |
| CN113516229A (en) | Credible user optimization selection method facing crowd sensing system | |
| Yu et al. | Mining email archives and simulating the dynamics of open-source project developer networks | |
| CN108073502B (en) | Test method and system thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |