CN115587693A - Flight training quality rating method based on observable behaviors - Google Patents
Flight training quality rating method based on observable behaviors Download PDFInfo
- Publication number
- CN115587693A CN115587693A CN202211168069.6A CN202211168069A CN115587693A CN 115587693 A CN115587693 A CN 115587693A CN 202211168069 A CN202211168069 A CN 202211168069A CN 115587693 A CN115587693 A CN 115587693A
- Authority
- CN
- China
- Prior art keywords
- observation
- training
- evaluation
- rating
- items
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 78
- 238000000034 method Methods 0.000 title claims abstract description 32
- 230000006399 behavior Effects 0.000 title abstract description 18
- 239000011159 matrix material Substances 0.000 claims abstract description 42
- 238000005259 measurement Methods 0.000 claims abstract description 8
- 238000011156 evaluation Methods 0.000 claims description 44
- 238000013210 evaluation model Methods 0.000 claims description 13
- 238000005457 optimization Methods 0.000 claims description 13
- 238000011835 investigation Methods 0.000 claims description 2
- 238000013507 mapping Methods 0.000 claims description 2
- 238000011160 research Methods 0.000 claims description 2
- 238000013461 design Methods 0.000 abstract description 11
- 238000012880 independent component analysis Methods 0.000 abstract 2
- OMHYGQBGFWWXJK-UHFFFAOYSA-N cyclobutane-1,2,3,4-tetracarboxylic acid;dihydrate Chemical compound O.O.OC(=O)C1C(C(O)=O)C(C(O)=O)C1C(O)=O OMHYGQBGFWWXJK-UHFFFAOYSA-N 0.000 abstract 1
- 238000013441 quality evaluation Methods 0.000 description 4
- 238000007689 inspection Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000001747 exhibiting effect Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000011158 quantitative evaluation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 239000013074 reference sample Substances 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
- G06Q50/2057—Career enhancement or continuing education service
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Educational Administration (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Tourism & Hospitality (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Mathematical Physics (AREA)
- Educational Technology (AREA)
- General Business, Economics & Management (AREA)
- Pure & Applied Mathematics (AREA)
- Operations Research (AREA)
- Data Mining & Analysis (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computing Systems (AREA)
- Primary Health Care (AREA)
- General Health & Medical Sciences (AREA)
- Algebra (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of row training rating, and particularly discloses a flight training quality rating method based on observable behaviors, which is based on a traditional flight training performance assessment operation mode and a CBTA theory constructed by ICAO (independent component analysis) to design a typical subject observation item measurement table and construct a competence assessment matrix, so as to provide a solution for optimizing the competence rating of each stage of initial flight training.
Description
Technical Field
The invention relates to the technical field of flight training rating, in particular to a flight training quality rating method based on observable behaviors.
Background
Flight training is the basis for guaranteeing flight safety and realizing civil aviation high-quality development, and a scientific and standard training quality evaluation system is a key link for controlling training quality and improving training efficiency. In order to alleviate the risks encountered by members of the aircraft fleet in flight, especially in order to deal with increasingly complex civil aviation systems, the International Civil Aviation Organization (ICAO) proposes that flight Training and assessment focus on nine core Competency concepts such as knowledge application (KNO), program application and compliance regulations (APK), automatic flight path management (FPA), manual Flight Path Management (FPM), communication (COM), leadership and team cooperation (LTW), situational awareness and information management (SAW), workload management (WLM), and problem resolution and decision (PSD), and in the Global Aviation Safety Program (GASP), it is emphasized that Competency Based Training and Assessment (CBTA) is used as a strategy for continuously improving global aviation safety.
In addition, the pilot skill life cycle management (PLM) is the development direction of a global civil aviation industry pilot skill management paradigm, and the premise is to establish a set of capability evaluation index system which runs through the pilot skill life cycle, so that the international civil aviation organization and the IATA establish a nine-competency index system and define the investigation dimensionality of each competency. Therefore, in order to advance PLM management paradigm landing in the whole industry, it is necessary to improve the pilot training quality evaluation method according to the core competency assessment index system constructed by ICAO from the beginning of the initial flight training phase.
At present, a temporary flight training assessment mode is implemented by taking subjects as training carriers, and mainly inspectors give conclusions of 'passing' and 'not passing' through observing the flight process of the students, so that scientific assessment on various competency levels and distribution of the students cannot be realized. Therefore, the existing training quality evaluation method needs to be revolutionary and optimized, the existing 'fixed subject type' training quality evaluation is converted into core competence training evaluation (CBTA) based on observable behaviors, and therefore a flight training quality rating method based on observable behaviors needs to be established by taking a core competence evaluation index system constructed by ICAO as a reference.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a flight training quality rating method based on observable behaviors, a typical subject observation item measurement table is designed and an evaluation matrix is constructed based on a traditional flight training performance assessment operation mode and a CBTA theory constructed by ICAO, an OB-based competence rating model is constructed to obtain a rating standard, a solution is provided for optimizing competence rating at each stage of initial flight training, and the problems in the background art are solved.
In order to achieve the purpose, the invention provides the following technical scheme: a method for flight training quality rating based on observable behavior, comprising the steps of:
s1, constructing a training and evaluating work sheet, wherein the work sheet comprises typical subjects to be investigated, observation items of each subject, a scoring standard and a score;
s2, constructing an observation item-OB incidence matrix;
s3, obtaining a measurement vector A by an examiner according to the training evaluation work sheet;
s4, constructing a competence evaluation matrix Y;
s5, evaluating the matrix Y according to competence, and introducing relative norm
Frequency f of representation of observable behavior OB ofn And a presentation number f mny The evaluation model of (2);
s6, obtaining a grading threshold value by solving an optimization problem according to the evaluation model
And gamma 1 、γ 2 、γ 3 The trainees are then rated according to the classified competency rating criteria.
Preferably, in step S2, an association is established between any observation item i and the competency observable behavior index OBj, and is represented by an observation item-OB association matrix B,
wherein, b ij Represents the associated attribute of the ith observation item and the jth OB, i =1, 2.·, m; j =1,2,. N; b ij If =1, it means that the ith observation item has a mapping relation with the jth OB, otherwise, it is 0.
Preferably, in step S3, according to the evaluation criteria of the observation items of the training evaluation worksheet, the corresponding checking and scoring are performed on the observation item performance when the trainee completes each subject, so as to obtain the result of measuring the observation items, and form an observation vector a,
A＝(a i ) m×1 ＝(a 1 ,a 2 ,…,a m ) T ，i＝1,2,...,m，
wherein, a i The score of the ith observation term is the maximum value
For the full score value of the observation item, when all the observation items are full scores, the observation vector can be obtained
Preferably, the scoring criteria include checking items and key review items; item selection is performed: a score of 0 or 1 to evaluate whether completion, 1 completion, not 0 completion; the key research items are as follows: the scores are 3,2,1 and 0, which respectively correspond to the completion standard of the specific observation items.
Preferably, in step S4, elements in the observation vector a are multiplied by corresponding rows in the association matrix respectively by using the observation vector a and the observation item-OB association matrix to obtain an evaluation matrix Y of competence,
wherein, a i b ij Representing the contribution level of the ith observation to OBj.
Preferably, in step S5, the evaluation model is expressed as follows:
when all the observation items are fully divided, namely the observation vector
Then, OB number f is obtained according to the evaluation model mny Frequency f ofn The maximum value of (d) is:
aiming at the condition that training and examination worksheets related to different training institutions and different training courses have differences in the arrangement of observation items and completion standards and are convenient for unifying competency rating standards, relative norms are introduced as follows, wherein
Preferably, the step of solving the optimization problem results in a step thresholdOptimal solution
And gamma 1 、γ 2 、γ 3 The method comprises the following steps:
wherein, P exa Rating of the examiner to the student, P exa = excellent, good, medium, poor } = {4,3,2,1}; p ofn Is composed of
Classification of (2), P mny Is composed of
The degree of the division is such that,
and gamma 1 、γ 2 、γ 3 Is composed of
Classification threshold of, P OB Is according to P ofn And P mny Competency ratings derived based on the VENN criteria.
The invention has the beneficial effects that: the invention designs a typical subject observation item measuring table and a constructed competence evaluation matrix based on a traditional flight training performance evaluation operation mode and a CBTA theory constructed by ICAO, provides a solution for solving the competence rating optimization of each stage of initial flight training, can be used as a quantitative performance evaluation tool in the practical skill evaluation of each stage of the initial flight training, and is in seamless connection with the existing initial flight training evaluation mode based on a work sheet.
Drawings
FIG. 1 is a schematic diagram of a process framework of the present invention;
FIG. 2 is a schematic diagram of the steps of the method of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
Example 1
In order to ensure the maximum consistency and objectivity of the assessment of the competencies of the trainee, the number of OBs (namely HOW MANY), the frequency (namely HOW OFTEN) and the threatens and error management results which are particularly relevant to the competencies need to be evaluated from three dimensions of the OB displayed by the trainee. Where the dimension "How many" (How many) indicates whether the trainee has the ability, "frequency" (How fonn) indicates the robustness used to judge the trainee's ability, "TEM's result" (out com) indicates evidence related to the ability's effectiveness as a countermeasure for threats and errors by individuals and teams, and competency assessment (How well) is a combination of the number of OB shown and their frequency and the corresponding results of threat and error management that are particularly relevant to the competency assessed.
The traditional flight training evaluation mode mainly depends on subjective judgment of an examiner on the quality level of each subject of a flight student, lacks quantitative analysis, particularly cannot carry out comprehensive quantitative evaluation on each competence level and structure of the student, and cannot meet the requirement of full-life-cycle management of pilot skills.
CBTA evaluation systems such as EBT handbooks, PLM systems and the like constructed by ICAO at present are more mature pilots facing to the airline stock company, and are not suitable for training characteristics based on fixed subjects in the initial flight training stage; the proposed VENN assessment criterion does not lack quantitative criteria for the grading of two dimensions, OB exposure "quantity" (How may), "frequency" (How may), involved in the ability rating, but is completely dependent on the experiential judgment of the examiner and thus still subjective. Therefore, for each stage of the initial flight training, how to reasonably define the Observable Behavior (OB) of each competence, and determine the grading standard of the relevant OB exhibiting "quantity" (How many) "and" frequency "(How fonn)" is a problem to be solved, which is related to the operability of the initial flight training CBTA.
For this purpose, it is necessary to establish an operable competency assessment optimization method according to the competency culture requirements of each stage of the initial training, including observation items and scoring criteria of OB, association of observation items and OB, and competency ranking criteria based on OB exhibiting "number (How) data", "frequency (How) data, and the like.
The invention designs a training performance evaluation optimization scheme based on a traditional flight training performance evaluation work sheet mode and a VENN criterion, which is applicable to hierarchical quantitative evaluation of competency performances of each stage of initial flight training, and a specific frame is shown in figure 1, and comprises the following steps:
step 1: constructing a training assessment worksheet
The work order includes typical subjects to be reviewed, observation items for each subject, scoring criteria, and scores.
An initial training evaluation worksheet is designed, a unified evaluation worksheet is designed for each inspection project, and unified quantitative measurement of the mastery degree of the skills of the trainees can be realized by standardizing the observation items and the completion standards of each subject. In the assessment process, the flight examiner scores the observation items of the trainee in each subject according to the completion standard of the training assessment worksheet, and the observation vector scores of the observation items can be obtained.
According to the flight training practice examination standard issued by the bureau and the teaching outline requirements of each training institution, the flight experts design a uniform evaluation work sheet for each inspection project to standardize the typical subjects to be investigated,and the observation items and the completion standards of each subject are checked by the examiner against the evaluation worksheet to form an observation vector A = (a) i ) m×1 ＝(a 1 ,a 2 ,…,a m ) T I =1, 2.., m, the design of the evaluation worksheet is shown in table 1.
TABLE 1 design of initial flight training assessment worksheet
Step 2: construction of Observation term-OB correlation matrix
Each observation item corresponds to an observable behavior OB of a certain competent capacity, the flight expert opinion is solicited by utilizing a Delphi survey method, and the association between any observation item i and an observable behavior index OBj of the competent capacity can be constructed to form an association matrix.
The columns of the correlation matrix are observation terms, the rows are the evaluation dimensions of each term competency, such as OB1, OB2, etc., and the process of obtaining the correlation matrix is shown in table 2 below.
TABLE 2 Observation-OB Table design
From table 2 it can be derived that the correlation matrix is B,
and step 3: obtaining a measurement vector A by contrasting the training evaluation worksheet
And the examiner performs corresponding check scoring on the observation item performance when the student finishes each subject according to the observation item finishing standard of the evaluation work sheet to obtain the result of measuring the observation item and form an observation vector A.
And 4, step 4: constructing competency assessment matrix Y
According to the VENN evaluation criterion, the competence level of the student can be measured by counting the number and the frequency of OB shown in the assessment, and an evaluation matrix Y of the competence can be constructed by utilizing the observation vector A and the observation item-OB incidence matrix; and introduces the concept of norm, so that an OB exhibition frequency (HOW OFTEN) and quantity (HOW MANY) evaluation model based on the competence evaluation matrix can be obtained.
The elements in the observation vector a are multiplied by the corresponding rows in the association matrix B, respectively, to obtain the table shown in table 3.
TABLE 3 Table design of competency assessment matrix
| Observation item | OB1 | OB2 | … | OBn |
| Observation item 1 | a 1 *b 11 | a 1 *b 12 | … | a 1 *b 1n |
| Observation item 2 | a 2 *b 21 | a 2 *b 22 | … | a 2 *b 2n |
| … | … | … | … | … |
| … | … | … | … | … |
| … | … | … | … | … |
| Observation item m | a m *b m1 | a m *b m2 | … | a m *b mn |
From table 3, a competency assessment matrix Y can be obtained,
wherein, a i b ij Represents the ith observation item pairAt the level of contribution of OBj.
And 5: design competency rating criteria
By taking the operation ability rating of the daily instructor to the student as a reference, a competence assessment model is constructed, a rating function is constructed by converting the competence assessment model into an optimization problem, and the performance rating standard divided according to the competence can be obtained.
Example 2
The invention designs a flight training quality rating method based on observable behaviors based on a traditional flight training performance assessment work sheet mode and a VENN criterion, and as shown in figure 2, the flight training quality rating method comprises the following steps:
step (1): training assessment work sheet design based on typical subject observation item checking
A unified assessment worksheet is designed for each inspection project, and unified quantitative measurement of the mastery degree of the student skills can be realized by standardizing the observation items and the completion standards of each subject. And designing an initial training and evaluating work sheet, and scoring the observation items of the trainees in each subject according to the completion standard of the training and evaluating work sheet by flight examiners in the evaluation process.
The scoring criteria comprise two types, (1) items are selected in a check mode, the score is 0 or 1, and the items are used for evaluating whether the items are finished, if so, the items are 1, and the items are not finished being 0; (2) The items are mainly inspected, and are divided into 3,2,1 and 0 which respectively correspond to the specific observation item completion standard.
Step (2): construction of Observation item-OB incidence matrix
Each observation item corresponds to observable behaviors OB of any competence, the observation of flight expert opinion by the Delphi survey method can be used for establishing the association between any observation item i and observable behavior indexes OBj of the competence, and the association is expressed by an association matrix B,
wherein, b ij Represents the associated attribute of the ith observation item and the jth OB, i =1, 2.·, m; j =1,2. b is a mixture of ij If =1, it means that the ith observation item and the jth OB exist in a mappingAnd if not, taking 0.
And (3): forming a measurement vector A based on examiner checking
According to flight training practice examination standards issued by bureau and teaching outline requirements of each training mechanism, observation items and grading standards of each training subject can be analyzed, examiners check completion conditions of the students according to evaluation worksheets to obtain grades, and A = (a =) can be obtained i ) m×1 ＝(a 1 ,a 2 ,…,a m ) T I =1, 2.. Multidot.m, wherein, a i The score of the ith observation item is the maximum value
For the full score value of the observation item, when all the observation items are full scores, the observation vector can be obtained
And (4): constructing a competence evaluation matrix Y, introducing relative norm according to the competence evaluation matrix Y
Representing the frequency f of the observable behavior OB ofn And a presentation number f mny The evaluation model of (3).
According to the VENN criterion, the competency level of the student can be measured by counting the number and the frequency of OB shown in the assessment, and an evaluation matrix of the competency can be constructed by utilizing an observation vector A and an observation item-OB association matrix B as follows:
wherein a is i b ij Representing the contribution level of the ith observation item to OBj, the exhibition frequency f of the Observable Behavior (OB) can be represented by the norm of the Y matrix by using the attribute of the vector/matrix norm which measures the space length or size of the vector (or matrix) ofn And a presentation number f mny 。
When the frequency of OB display is higher than 25% of the maximum value, the OB is considered to be displayed, otherwise, the OB is not displayed. Calculating the norm of the evaluation matrix to obtain OB display number (f) based on the competence evaluation matrix mny ) Frequency (f) ofn ) The evaluation model of (2) is as follows:
when all observation items are full, observation vector is obtained
Then, OB number of exhibits (f) can be obtained from the evaluation matrix mmy ) Frequency (f) ofn ) The maximum value of (c) is:
in view of the situation that training and assessment worksheets related to different training institutions and different training courses have differences in the arrangement of observation items and completion standards and are convenient for unifying competency rating standards, the invention introduces the following relative norms, wherein
And (5): obtaining a threshold optimal solution by solving an optimization problem according to the evaluation model
And gamma 1 、γ 2 、γ 3 And then carrying out rating based on the competence, and obtaining rating standards divided according to the competence for rating optimization.
The method takes the operation ability grade of the examiner to the student as reference sample data, and can be converted into the solution of the optimization problem through sample parameters to obtain the optimal solution of the threshold value
And gamma 1 、γ 2 、γ 3 Performance may be rated based on competency.
P OB ＝min(P ofn ,P mny ) (11)
Wherein, P exa Grade of examiner to student is scored by P exa = excellent, good, medium, poor } = {4,3,2,1}. P ofn Is composed of
Classification of (2), P mny Is composed of
The degree of the division is such that,
and gamma 1 、γ 2 、γ 3 Is OB
Classification threshold of (P) OB Is according to P ofn And P mny And a competency rating based on the VENN criterion.
And gamma 1 、γ 2 、γ 3 Can be determined by solving the optimization problem formed by the equations (8) - (11).
The method is suitable for competence evaluation of each stage, and in view of different stages of initial flight training, different requirements are required on a pilot, different subject observation items and different scoring standards exist, and the capability of the pilot is gradually cultured, so that the method can be applied to other courses in the next step, an evaluation worksheet, an evaluation matrix and the like are respectively set by experts according to requirements and characteristics of the courses, and a performance evaluation model is established according to the process.
Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described in the foregoing embodiments, or equivalents may be substituted for elements thereof.
Claims (7)
1. A flight training quality rating method based on observable behavior is characterized by comprising the following steps:
s1, constructing a training and evaluating work sheet, wherein the work sheet comprises typical subjects to be investigated, observation items of each subject, a scoring standard and a score;
s2, constructing an observation item-OB incidence matrix;
s3, obtaining a measurement vector A by contrasting the training evaluation worksheet;
s4, constructing a competence evaluation matrix Y;
s5, evaluating the matrix Y according to competence, and introducing relative norm
Frequency f of representation of observable behavior OB ofn And the number f of displays mny The evaluation model of (2);
s6, obtaining the optimal solution of the threshold value by solving the optimization problem according to the evaluation model
And gamma 1 、γ 2 、γ 3 And then carrying out rating based on the competence, and obtaining rating standards divided according to the competence for rating optimization.
2. The method for flight training quality rating based on observable behavior of claim 1, wherein: in step S2, an association is established between any observation item i and the observable behavior index of competency OBj, represented by an observation item-OB association matrix B,
wherein, b ij Represents the associated attribute of the ith observation item and the jth OB, i =1, 2.·, m; j =1,2,. Ang, n; b ij If =1, it means that the ith observation item has a mapping relationship with the jth OB, otherwise, it takes 0.
3. The method for observable behavior-based flight training quality rating of claim 1, wherein: in step S3, according to the evaluation standard of the observation item of the training evaluation worksheet, corresponding checking and scoring is carried out on the observation item performance when the trainees finish each subject to obtain the result of measuring the observation item to form an observation vector A,
A＝(a i ) m×1 ＝(a 1 ，a 2 ，，a m ) T ，i＝1，2，...，m，
wherein, a i The score of the ith observation item is the maximum value
For the full score value of the observation item, when all the observation items are full scores, the observation vector can be obtained
4. The method for observable behavior-based flight training quality rating of claim 3, wherein: the scoring standard comprises checking items and key investigation items; item selection is performed: the score is 0 or 1 and is used for evaluating whether the completion is 1 or not and the non-completion is 0; the key research items are as follows: the scores are 3,2,1 and 0, which respectively correspond to the completion standard of the specific observation items.
5. The method for flight training quality rating based on observable behavior of claim 1, wherein: in step S4, elements in the observation vector A are multiplied with corresponding rows in the incidence matrix respectively by using the observation vector A and the observation item-OB incidence matrix to construct an evaluation matrix Y of competence,
wherein, a i b ij Representing the contribution level of the ith observation to OBj.
6. The method for flight training quality rating based on observable behavior of claim 1, wherein: in step S5, the evaluation model is expressed as follows:
when all observation items are full, namely the observation vector
Then, the OB number f is obtained according to the evaluation model mny Frequency f ofn The maximum value of (d) is:
aiming at the condition that training and examination worksheets related to different training institutions and different training courses have differences in the arrangement of observation items and completion standards and are convenient for unifying competency rating standards, relative norms are introduced as follows, wherein
7. The method for observable behavior-based flight training quality rating of claim 1, wherein: solving the optimization problem to obtain a threshold optimal solution
And gamma 1 、γ 2 、γ 3 The method comprises the following steps:
P OB ＝min(P ofn ，P mny )
wherein, P exa Rating of the examiner to the student, P exa = excellent, good, medium, poor } = {4,3,2,1}; p ofn Is composed of
Grade of division, P mny Is composed of
Divided or the likeThe number of stages is such that,
and gamma 1 、γ 2 、γ 3 Is composed of
Classification threshold of (P) OB Is according to P ofn And P mny Competency ratings derived based on the VENN criteria.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211168069.6A CN115587693B (en) | 2022-09-23 | 2022-09-23 | Method for grading flight training quality based on observable behaviors |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211168069.6A CN115587693B (en) | 2022-09-23 | 2022-09-23 | Method for grading flight training quality based on observable behaviors |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115587693A true CN115587693A (en) | 2023-01-10 |
| CN115587693B CN115587693B (en) | 2023-08-11 |
Family
ID=84778990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211168069.6A Active CN115587693B (en) | 2022-09-23 | 2022-09-23 | Method for grading flight training quality based on observable behaviors |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115587693B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291774A (en) * | 2023-11-24 | 2023-12-26 | 山东航空股份有限公司 | Method and system for pilot EBT course development and compliance verification |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007148867A1 (en) * | 2006-06-20 | 2007-12-27 | Industry-Academic Cooperation Foundation, Yonsei University | Method of technology evaluation |
| CN107220761A (en) * | 2017-05-24 | 2017-09-29 | 江西洪都航空工业集团有限责任公司 | A kind of training plane effectiveness measures of effectiveness computational methods |
| CN110008442A (en) * | 2019-03-21 | 2019-07-12 | 上海工程技术大学 | A kind of pilot training's control method based on pilot's comprehensive ability evaluation |
| US20210110731A1 (en) * | 2019-10-14 | 2021-04-15 | Honeywell International Inc. | Systems and methods for evidence-based training of aircraft operators |
| CN112766729A (en) * | 2021-01-20 | 2021-05-07 | 中国人民解放军火箭军工程大学 | Method for evaluating capability of equipment maintenance training personnel |
| CN113298431A (en) * | 2021-06-21 | 2021-08-24 | 北京航空航天大学 | Aviation QAR big data-based pilot competence portrayal method and system |
| CN114241852A (en) * | 2021-12-30 | 2022-03-25 | 中国民航科学技术研究院 | Multi-source data fusion-based flight simulation training evaluation system and method for in-process aircraft |
| CN114299422A (en) * | 2021-12-20 | 2022-04-08 | 中国人民解放军海军航空大学 | Flight quality self-adaptive evaluation method |
| CN114373360A (en) * | 2021-12-17 | 2022-04-19 | 清华大学 | Intelligent training system, method and device for flight simulator |
-
2022
- 2022-09-23 CN CN202211168069.6A patent/CN115587693B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007148867A1 (en) * | 2006-06-20 | 2007-12-27 | Industry-Academic Cooperation Foundation, Yonsei University | Method of technology evaluation |
| CN107220761A (en) * | 2017-05-24 | 2017-09-29 | 江西洪都航空工业集团有限责任公司 | A kind of training plane effectiveness measures of effectiveness computational methods |
| CN110008442A (en) * | 2019-03-21 | 2019-07-12 | 上海工程技术大学 | A kind of pilot training's control method based on pilot's comprehensive ability evaluation |
| US20210110731A1 (en) * | 2019-10-14 | 2021-04-15 | Honeywell International Inc. | Systems and methods for evidence-based training of aircraft operators |
| CN112766729A (en) * | 2021-01-20 | 2021-05-07 | 中国人民解放军火箭军工程大学 | Method for evaluating capability of equipment maintenance training personnel |
| CN113298431A (en) * | 2021-06-21 | 2021-08-24 | 北京航空航天大学 | Aviation QAR big data-based pilot competence portrayal method and system |
| CN114373360A (en) * | 2021-12-17 | 2022-04-19 | 清华大学 | Intelligent training system, method and device for flight simulator |
| CN114299422A (en) * | 2021-12-20 | 2022-04-08 | 中国人民解放军海军航空大学 | Flight quality self-adaptive evaluation method |
| CN114241852A (en) * | 2021-12-30 | 2022-03-25 | 中国民航科学技术研究院 | Multi-source data fusion-based flight simulation training evaluation system and method for in-process aircraft |
Non-Patent Citations (1)
| Title |
|---|
| 刘莉雯等: "多参数融合的飞行品质评估模型的建立", 《计算机工程与科学》, vol. 38, no. 6, pages 1262 - 1268 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291774A (en) * | 2023-11-24 | 2023-12-26 | 山东航空股份有限公司 | Method and system for pilot EBT course development and compliance verification |
| CN117291774B (en) * | 2023-11-24 | 2024-02-20 | 山东航空股份有限公司 | Method and system for pilot EBT course development and compliance verification |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115587693B (en) | 2023-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20040157201A1 (en) | Classroom productivity index | |
| CN109345101A (en) | Evaluation in Education Quality analysis method based on comprehensive evaluation analysis method | |
| CN115587693A (en) | Flight training quality rating method based on observable behaviors | |
| Joseph-Obi | The impact of workers training on the organizational performance of the staff of the Ministry of Labour and productivity, Port Harcourt | |
| Bone et al. | the Teacher Performance Appraisal System in Improving Teachers Performance in Limboto District | |
| Tian et al. | Software Requirements Engineer’s Ability Assessment Method Based on Empirical Software Engineering | |
| Liu et al. | A Study of Factors Affecting Science Achievements of Junior High School Female Students. | |
| Dewi et al. | The Effect of Training Material and Instructor on the Behavior of Graduates of Teacher Academy Program with Learning Outcome as Intervening Variable (Case Study at XYZ Company) | |
| Liu et al. | Performance Evaluation and Identification of Key Influencing Factors for Student Achievement Based on the Entropy‐Weighted TOPSIS Model | |
| Koral-Kordova et al. | Improving capacity for engineering systems thinking (CEST) among industrial engineering students | |
| Levin | Ability testing for job selection: Are the economic claims justified? | |
| CN112184040A (en) | Platform for evaluating software engineering capability based on behavior and learning data | |
| Gupta et al. | Decision analysis approach for quality in technical education | |
| Qi | Construction of Ideological and Political Evaluation System for College Chinese Courses in Vocational Colleges Based on the Theoretical Framework of CIPP Evaluation Model: Taking Guangzhou Nanyang Polytechnic College as an Example | |
| CN117670601A (en) | Optimized initial flight training stage competence rating method | |
| Quan | Post competency evaluation model of English Majors Based on Data Mining | |
| Bjorklund-Young | Principal Evaluation Models | |
| CN117057673A (en) | Competence quantitative evaluation standard formulation method based on VENN model | |
| Zhao et al. | An Analytical Study of Digital Competency Assessment Tools for Higher Education Teachers | |
| Surbakti et al. | Application of fuzzy logic and importance performance analysis in assessing organizational agility of PT. IF Heavy Equipment Distributor | |
| Wang et al. | Research on the Evaluation System of Military Representatives | |
| Lu et al. | Effect of Project-Based Blended Learning (PJBBL) on Algorithm Design and Analysis Ability | |
| Oktaviani et al. | Accounting Implementation, Accountability, Strategy Implementation, and Company Performance: Interconnections and Implications | |
| Sensuse et al. | Analysis of Knowledge Management Readiness with an Embedded Method: A Case Study of an After-Sales Service Organization in PT XYZ | |
| Fang | Research on the Teaching Quality Evaluation System of Ideological and Political Theories Teaching in All Courses Based on Analytic Hierarchy Process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |