CN115587693B - Method for grading flight training quality based on observable behaviors - Google Patents
Method for grading flight training quality based on observable behaviors Download PDFInfo
- Publication number
- CN115587693B CN115587693B CN202211168069.6A CN202211168069A CN115587693B CN 115587693 B CN115587693 B CN 115587693B CN 202211168069 A CN202211168069 A CN 202211168069A CN 115587693 B CN115587693 B CN 115587693B
- Authority
- CN
- China
- Prior art keywords
- observation
- competence
- item
- training
- evaluation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 75
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000006399 behavior Effects 0.000 title abstract description 18
- 238000011156 evaluation Methods 0.000 claims description 40
- 239000011159 matrix material Substances 0.000 claims description 40
- 238000005457 optimization Methods 0.000 claims description 13
- 238000005259 measurement Methods 0.000 claims description 12
- 238000013210 evaluation model Methods 0.000 claims description 8
- 238000011835 investigation Methods 0.000 claims description 6
- 238000012512 characterization method Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 description 11
- 230000000875 corresponding effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 238000001303 quality assessment method Methods 0.000 description 2
- 238000013441 quality evaluation Methods 0.000 description 2
- 238000011158 quantitative evaluation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000000969 carrier Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000012258 culturing Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000004445 quantitative analysis Methods 0.000 description 1
- 239000013074 reference sample Substances 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 239000000523 sample Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06395—Quality analysis or management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/20—Education
- G06Q50/205—Education administration or guidance
- G06Q50/2057—Career enhancement or continuing education service
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Educational Administration (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Tourism & Hospitality (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Mathematical Physics (AREA)
- Educational Technology (AREA)
- General Business, Economics & Management (AREA)
- Pure & Applied Mathematics (AREA)
- Operations Research (AREA)
- Data Mining & Analysis (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Computational Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computing Systems (AREA)
- Primary Health Care (AREA)
- General Health & Medical Sciences (AREA)
- Algebra (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the technical field of training ratings, and particularly discloses a method for rating flight training quality based on observable behaviors.
Description
Technical Field
The invention relates to the technical field of flight training ratings, in particular to a method for rating flight training quality based on observable behaviors.
Background
The flight training is a basis for guaranteeing flight safety and realizing high-quality development of civil aviation, and a scientific and standard training quality assessment system is a key link for controlling training quality and improving training efficiency. In order to alleviate risks encountered by flight crew members in flight, particularly in response to increasingly complex civil aviation systems, international civil aviation organizations (hereinafter ICAO) propose flight training and assessment focusing on knowledge application (KNO), program application and compliance regulations (APK), automatic trail management (FPA), manual trail management (FPM), communication (COM), pilot and team cooperation (LTW), situational awareness and information management (SAW), workload management (WLM), problem resolution and decision (PSD) and the like nine major pilot core competence concepts, and emphasize competence-based training and assessment (Competency Based Training System, CBTA) in Global Aviation Safety Program (GASP) as strategies for continuously improving global aviation safety.
In addition, pilot skill full life cycle management (PLM) is a development direction of a global civil aviation pilot skill management paradigm, and the premise is to establish a set of capability assessment index systems throughout the pilot skill full life cycle, so that the international civil aviation organization and IATA establish a nine-big competence index system and define a investigation dimension of each competence. Therefore, to advance PLM management paradigm landing throughout the industry, there is a need to improve pilot training quality assessment methods in terms of the core competence assessment index system built by ICAO from the beginning of the initial flight training phase.
The current temporary flight training evaluation mode is implemented by taking subjects as training carriers, and a conclusion of passing and not passing is mainly given by an inspector through observation of the flight process of the learner, so that scientific evaluation on each competence level and distribution of the learner cannot be realized. Therefore, the existing training quality evaluation method needs to be changed and optimized, and the old 'fixed subject' training quality evaluation is changed into the core competence training evaluation (CBTA) based on observable behaviors, so that a flight training quality rating method based on observable behaviors needs to be established based on the core competence evaluation index system constructed by ICAO.
Disclosure of Invention
In order to solve the problems in the prior art, the invention provides a method for grading flight training quality based on observable behaviors, which is based on a traditional flight training performance assessment operation mode and an ICAO constructed CBTA theory, designs a typical subject observation item measurement table, constructs an evaluation matrix, constructs an OB-based competence grading model, obtains grading standards, provides a solution for optimizing competence grading in each stage of initial flight training, and solves the problems mentioned in the background art.
In order to achieve the above purpose, the present invention provides the following technical solutions: a method of flight training quality rating based on observable behavior, comprising the steps of:
s1, constructing a training evaluation work sheet, wherein the work sheet comprises typical subjects to be inspected, observation items of each subject, scoring standards and scores;
s2, constructing an observation item-OB association matrix;
s3, obtaining a measurement vector A by an inspector according to a training evaluation work sheet;
s4, constructing a competence assessment matrix Y;
s5, introducing relative norms according to the competence evaluation matrix YCharacterization of the frequency f of the presentation of the observable behavior OB ofn And a presentation number f mny Is a model of the assessment of (a);
s6, obtaining a grading threshold value by solving an optimization problem according to the evaluation modelAnd gamma 1 、γ 2 、γ 3 The learner is then rated according to the divided competency rating criteria.
Preferably, in step S2, any observation i is correlated with the competence observable behavior index OBj, and represented by an observation-OB correlation matrix B,
wherein b ij Representing the association attribute of the ith observation and the jth OB, i=1, 2,. -%, m; j=1, 2,. -%, n; b ij When=1, the mapping relationship exists between the ith observation item and the jth OB, otherwise, 0 is taken.
Preferably, in step S3, according to the observation item scoring criteria of the training evaluation workflow, the observation item performance is scored correspondingly when the learner completes each subject, the result of the observation item measurement is obtained, the observation vector a is formed,
A＝(a i ) m×1 ＝(a 1 ,a 2 ,…,a m ) T ，i＝1,2,...,m，
wherein a is i Scoring the ith observation, its maximumFor the full score of the observation term, an observation vector +.>
Preferably, the scoring criteria include a hook item and a focus review item; selecting items: a score of 0 or 1, for evaluating whether it is completed, completed to 1, and not completed to 0; the key investigation items are as follows: the scores are 3,2,1 and 0, and respectively correspond to specific observation item completion standards.
Preferably, in step S4, the elements in the observation vector a are multiplied by the corresponding rows in the correlation matrix respectively by using the observation vector a and the observation term-OB correlation matrix to obtain a competence evaluation matrix Y,
wherein a is i b ij Representing the level of contribution of the ith observation to OBj.
Preferably, in step S5, the evaluation model expresses the following:
when all observation items are fully divided, namely the observation vectorWhen the OB display quantity f can be obtained according to the evaluation model mny Frequency f ofn The maximum value of (2) is:
aiming at the conditions that training examination worksheets related to different training institutions and different training courses have differences in the setting of observation items and completion standards, and are convenient for unifying competence rating standards, a relative norm is introduced as follows, wherein
Preferably, solving the optimization problem results in an optimal solution for the classification thresholdAnd gamma 1 、γ 2 、γ 3 The method is characterized by comprising the following steps:
wherein P is exa To grade the trainee for the exam staff, P exa = { excellent, good, medium, difference = {4,3,2,1}; p (P) ofn Is thatIs of the order P mny Is->Grade of division->And gamma 1 、γ 2 、γ 3 Is->P is the classification threshold of (2) OB According to P ofn And P mny Competence ratings derived based on the VENN criteria.
The beneficial effects of the invention are as follows: the invention designs a typical subject observation item measuring table and builds a competence assessment matrix based on a traditional flight training performance assessment running mode and an ICAO constructed CBTA theory, provides a solution for solving competence rating optimization of each stage of initial flight training, can be used as a quantized performance assessment tool in practical skill assessment of each stage of initial flight training, is in seamless connection with the existing initial flight training assessment mode based on a working list, introduces a competence-based training and assessment (CBTA) index system in the initial training stage, and provides basic data support for an airline company to implement pilot skill full life cycle management (PLM).
Drawings
FIG. 1 is a schematic diagram of a method framework of the present invention;
FIG. 2 is a schematic diagram of the steps of the method of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
Performance and metrics are emphasized based on competence training and assessment (CBTA), and training is performed for specific performance criteria, in order to ensure maximum consistency and objectivity in the assessment of competence of a learner, it is necessary to assess competence from three dimensions, namely the number of OB's exhibited by the trainee (i.e., HOW MANY), the frequency (i.e., HOW OFTEN), and threat and error management results that are particularly relevant to competence. Where the dimension "How much" indicates whether the trainee is provided with the term's ability, "frequency" (How often) indicates the robustness with which the trainee's ability is judged, "TEM's result" (Outcome) indicates evidence related to the validity of the ability as a countermeasure for threats and errors by individuals and teams, and competence assessment (How well) is a combination of the number of presented OBs and their frequency, and the corresponding results of threat and error management that are particularly relevant to the competence assessed.
The traditional flight training evaluation mode mainly depends on subjective judgment of quality levels of various subjects completed by an inspector on the flight inspector, and lacks quantitative analysis, and particularly cannot make comprehensive quantitative evaluation on various competence levels and structures of the inspector, so that the requirements of full life cycle management of pilot skills cannot be met.
The CBTA evaluation system constructed by ICAO at present, such as EBT handbook, PLM system and the like, is more mature pilots facing the airlines and is not suitable for training characteristics based on fixed subjects in the initial flight training stage; the proposed VENN evaluation criterion does not show quantitative criteria for the classification of the two dimensions of "number (How many)", and "frequency (How of)", which relate to the capability ratings, but is completely dependent on the experience judgment of the inspector, and thus still has subjectivity. Thus, for each stage of the initial flight training, how to reasonably define each competency Observable Behavior (OB) and determine the ranking criteria of the relevant OB's presentation "number (How many)", "frequency (How often)" are challenges to be solved, which relates to the operability of the initial flight training CBTA.
For this reason, it is necessary to establish a competence assessment optimization method with operability according to the capability culturing requirements of the initial training stages, including observation items and scoring criteria of OB, association of observation items with OB, competence rating criteria based on OB-exhibited "number (How management)", "frequency (How of)", and the like.
The invention designs a training performance evaluation optimization scheme based on the VENN criterion based on a traditional work order mode of flight training performance assessment and the VENN criterion, which can be suitable for carrying out hierarchical quantitative evaluation on competence performances of all stages of initial flight training, and a specific framework is shown in figure 1 and comprises the following steps:
step 1: building training evaluation worksheets
The worksheet includes typical subjects to be examined, observations of each subject, scoring criteria, and scores.
The initial training evaluation work list is designed, a unified evaluation work list is designed for each examination item, and unified quantitative measurement of the mastering degree of the skills of the students can be realized by standardizing the observation item and the completion standard of each subject. In the checking process, a flight inspector scores observation items of each subject according to the completion standard of the training evaluation work order, and then the observation vector scores of the observation items can be obtained.
According to the flight training practice examination standard issued by the office and the teaching outline requirement of each training organization, a flight expert designs a unified evaluation work sheet for each examination item, typical subjects to be examined are standardized, and the observation items and the completion standard of each subject are checked by the inspector to check the completion condition of the learner by comparing with the evaluation work sheet to form an observation vector A= (a) i ) m×1 ＝(a 1 ,a 2 ,…,a m ) T I=1, 2,..m, the design of the evaluation worksheet is shown in table 1.
TABLE 1 design of initial flight training assessment worksheets
Step 2: constructing an observation term-OB association matrix
Each observation item corresponds to a competence observable behavior OB, a Derphine investigation method is utilized to solicit flight expert opinion, and an association between any observation item i and a competence observable behavior index OBj can be constructed to form an association matrix.
The columns of the correlation matrix are observation terms, and the rows are evaluation dimensions of each term's competence, such as OB1, OB2, etc., and the process of obtaining the correlation matrix is shown in table 2 below.
TABLE 2 Observation item-OB Table design
The correlation matrix B can be obtained from table 2,
step 3: obtaining a measurement vector A by comparing training and evaluating worksheets
And carrying out corresponding checking scoring on the observation item performance of the trainee when the trainee completes each subject according to the observation item completion standard of the evaluation work order to obtain a measurement result of the observation item, thereby forming an observation vector A.
Step 4: construction of competence assessment matrix Y
According to a VENN evaluation criterion, the competence level of a student can be measured by counting the quantity and frequency of OBs displayed by the student in the examination, and an observation vector A and an observation item-OB correlation matrix can be utilized to construct a competence evaluation matrix Y; and introducing the concept of norm, thereby obtaining an OB display frequency (HOW OFTEN) and number (HOW MANEY) evaluation model based on the competence evaluation matrix.
The elements in the observation vector a are multiplied by the corresponding rows in the correlation matrix B, respectively, to obtain a table as shown in table 3.
TABLE 3 Table design of competence assessment matrix
| Observation item | OB1 | OB2 | … | OBn |
| Observation item 1 | a 1 *b 11 | a 1 *b 12 | … | a 1 *b 1n |
| Observation item 2 | a 2 *b 21 | a 2 *b 22 | … | a 2 *b 2n |
| … | … | … | … | … |
| … | … | … | … | … |
| … | … | … | … | … |
| Observation item m | a m *b m1 | a m *b m2 | … | a m *b mn |
From table 3, a competence assessment matrix Y can be obtained,
wherein a is i b ij Representing the level of contribution of the ith observation to OBj.
Step 5: design competence rating criteria
By referring to the operation ability rating of a daily instructor to a student as a reference, a competence assessment model is constructed, and a rating function is constructed by converting the competence assessment model into an optimization problem, so that performance rating standards divided according to competence can be obtained.
Example 2
The invention designs a method for grading flight training quality based on observable behaviors based on a work order mode and a VENN criterion of traditional flight training performance assessment, which is shown in figure 2 and comprises the following steps:
step (1): training evaluation work sheet design based on typical subject observation item selection
And designing a unified assessment work sheet for each examination item, and realizing unified quantitative measurement of the mastering degree of the skills of the students by standardizing the observation item and the completion standard of each subject. And designing an initial training evaluation work sheet, and scoring observation items of students in each subject by flight examers according to the completion standard of the training evaluation work sheet in the checking process.
The scoring criteria include two types, (1) the item is checked, the score is 0 or 1, and the item is used for evaluating whether the item is finished, if the item is finished, the item is 1, and the item is not finished to be 0; (2) And (3) focusing on the investigation projects, wherein the scores are 3,2,1 and 0, and the scores respectively correspond to specific observation project completion standards.
Step (2): constructing an observation term-OB association matrix
Each observation corresponds to the observable behaviour OB of a competence of a certain term, and the association between any observation i and the competence observable behaviour index OBj can be established by soliciting the flight expert opinion by the delfei survey method and expressed by an association matrix B,
wherein b ij Representing the association attribute of the ith observation and the jth OB, i=1, 2,. -%, m; j=1, 2,..n. b ij When=1, the mapping relationship exists between the ith observation item and the jth OB, otherwise, 0 is taken.
Step (3): measurement vector A is formed based on examination staff selection
According to flight training practice examination standards issued by office parties and teaching outline requirements of each training institution, observation items and scoring standards of each training department can be analyzed, and a learner is checked and selected by a tester to obtain scoring by comparing the completion condition of the learner with an evaluation work order, so that A= (a) i ) m×1 ＝(a 1 ,a 2 ,…,a m ) T I=1, 2,..m, where a i Scoring the ith observation, its maximumFor the full score of the observation term, when all the observation terms are full, an observation vector can be obtained
Step (4): constructing a competence evaluation matrix Y, and introducing relative norms according to the competence evaluation matrix YCharacterization of the frequency f of the presentation of the observable behavior OB ofn And a presentation number f mny Is described.
According to the VENN criterion, the competence level of a student can be measured by counting the quantity and frequency of OBs which the student displays in the examination, and an evaluation matrix of the competence can be constructed by using an observation vector A and an observation item-OB association matrix B as follows:
wherein a is i b ij Representing the contribution level of the ith observation term to OBj, the norms of the Y matrix can be used for representing the display frequency f of the Observable Behavior (OB) by utilizing the attribute of the space length or the size of the metric vector (or matrix) with the norms of the vector/matrix ofn And a presentation number f mny 。
When the OB display frequency is higher than 25% of the maximum value, the OB is considered to be displayed, and otherwise, the OB is not displayed. Calculating the evaluation matrix norms yields the OB presentation number (f mny ) Frequency (f) ofn ) Is set as follows:
when all observation items are fully divided, namely, the observation vectorWhen the OB display quantity (f) can be obtained according to the evaluation matrix mmy ) Frequency (f) ofn ) The maximum value of (2) is:
in view of the fact that training examination worksheets related to different training institutions and different training courses have differences in observation items and completion standard settings and are convenient for unifying competence rating standards, the invention introduces relative norms as follows, wherein
Step (5): obtaining a threshold optimal solution by solving an optimization problem according to the evaluation modelAnd gamma 1 、γ 2 、γ 3 The ratings are then based on competency, resulting in a rating criteria divided by competency for rating optimization.
The control ability of the inspector to the learner is rated as reference sample data, and the sample parameters can be converted into solution optimization problems to obtain a threshold optimal solutionAnd gamma 1 、γ 2 、γ 3 Performance may be rated based on competence.
P OB ＝min(P ofn ,P mny ) (11)
Wherein P is exa For the grade grading of the inspector to the learner, there is P exa = { excellent, good, medium, difference = {4,3,2,1}. P (P) ofn Is thatIs of the order P mny Is->Grade of division->And gamma 1 、γ 2 、γ 3 For OB-> P is the classification threshold of (2) OB According to P ofn And P mny And a competency rating based on the VENN criteria.And gamma 1 、γ 2 、γ 3 Can be determined by solving an optimization problem constituted by the equations (8) - (11).
The method is suitable for competence assessment in each stage, in view of different requirements on pilots in different stages of initial flight training, subjects observation items and scoring standards are different, the capability of the pilots is gradually cultivated, so that the method can be applied to other courses in the next step, an assessment work sheet, an assessment matrix and the like are respectively set by experts according to the requirements and characteristics of the courses, and a performance assessment model is established according to the flow.
Although the present invention has been described with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described, or equivalents may be substituted for elements thereof, and any modifications, equivalents, improvements and changes may be made without departing from the spirit and principles of the present invention.
Claims (2)
1. A method of flying training quality rating based on observable behavior, comprising the steps of:
s1, constructing a training evaluation work sheet, wherein the work sheet comprises typical subjects to be inspected, observation items of each subject, scoring standards and scores;
s2, constructing an observation item-OB incidence matrix: an association is established between any observation item i and a competence observable behavior index OBj, which is represented by an observation item-OB association matrix B,
wherein b ij Representing the association attribute of the ith observation and the jth OB, i=1, 2,. -%, m; j=1, 2,. -%, n; b ij When the index value is=1, the mapping relation exists between the ith observation item and the jth OB, otherwise, 0 is taken;
s3, comparing the training evaluation worksheet to obtain a measurement vector A: according to the observation item scoring standard of the training evaluation work order, the observation item performance is correspondingly checked and scored when the trainee completes each subject, the result of the observation item measurement is obtained, the observation vector A is formed,
A＝(a i ) m×1 ＝(a 1 ,a 2 ,…,a m ) T ，i＝1,2,...,m，
wherein a is i Scoring the ith observation, its maximumTo this viewThe full score of the measurement item, when all the measurement items are full score, the observation vector +.>
S4, constructing a competence assessment matrix Y: multiplying the elements in the observation vector A with the corresponding rows in the correlation matrix by using the observation vector A and the observation item-OB correlation matrix to construct a competence evaluation matrix Y,
wherein a is i b ij Represents the contribution level of the ith observation to OBj;
s5, introducing relative norms according to the competence evaluation matrix YCharacterization of the frequency f of the presentation of the observable behavior OB ofn And a presentation number f mny Is a model of the assessment of (a);
the evaluation model is expressed as follows:
when all observation items are fully divided, namely the observation vectorWhen the OB display quantity f is obtained according to the evaluation model mny Frequency f ofn The maximum value of (2) is:
aiming at the conditions that training examination worksheets related to different training institutions and different training courses have differences in the setting of observation items and completion standards, and are convenient for unifying competence rating standards, a relative norm is introduced as follows, wherein
S6, obtaining a threshold optimal solution by solving an optimization problem according to the evaluation modelAnd gamma 1 、γ 2 、γ 3 Then, grading is carried out based on competence, and grading standards divided according to competence are obtained and used for grading optimization; obtaining a threshold optimal solution by solving the following optimization problem>And gamma 1 、γ 2 、γ 3 The method is characterized by comprising the following steps:
P OB ＝min(P ofn ,P mny )
wherein P is exa To grade the trainee for the exam staff, P exa = { excellent, good, medium, difference = {4,3,2,1}; p (P) ofn Is thatGrade of division, P mny Is->Grade of division->And gamma 1 、γ 2 、γ 3 Is->P is the classification threshold of (2) OB According to P ofn And P mny And a competency rating based on the VENN criteria.
2. The method of observable behavior based flight training quality rating of claim 1, wherein: the scoring standard comprises a hook item and a key investigation item; selecting items: a score of 0 or 1, for evaluating whether it is completed, completed to 1, and not completed to 0; the key investigation items are as follows: the scores are 3,2,1 and 0, and respectively correspond to specific observation item completion standards.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211168069.6A CN115587693B (en) | 2022-09-23 | 2022-09-23 | Method for grading flight training quality based on observable behaviors |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211168069.6A CN115587693B (en) | 2022-09-23 | 2022-09-23 | Method for grading flight training quality based on observable behaviors |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115587693A CN115587693A (en) | 2023-01-10 |
| CN115587693B true CN115587693B (en) | 2023-08-11 |
Family
ID=84778990
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211168069.6A Active CN115587693B (en) | 2022-09-23 | 2022-09-23 | Method for grading flight training quality based on observable behaviors |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115587693B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117291774B (en) * | 2023-11-24 | 2024-02-20 | 山东航空股份有限公司 | Method and system for pilot EBT course development and compliance verification |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007148867A1 (en) * | 2006-06-20 | 2007-12-27 | Industry-Academic Cooperation Foundation, Yonsei University | Method of technology evaluation |
| CN107220761A (en) * | 2017-05-24 | 2017-09-29 | 江西洪都航空工业集团有限责任公司 | A kind of training plane effectiveness measures of effectiveness computational methods |
| CN110008442A (en) * | 2019-03-21 | 2019-07-12 | 上海工程技术大学 | A kind of pilot training's control method based on pilot's comprehensive ability evaluation |
| CN112766729A (en) * | 2021-01-20 | 2021-05-07 | 中国人民解放军火箭军工程大学 | Method for evaluating capability of equipment maintenance training personnel |
| CN113298431A (en) * | 2021-06-21 | 2021-08-24 | 北京航空航天大学 | Aviation QAR big data-based pilot competence portrayal method and system |
| CN114241852A (en) * | 2021-12-30 | 2022-03-25 | 中国民航科学技术研究院 | Multi-source data fusion-based flight simulation training evaluation system and method for in-process aircraft |
| CN114299422A (en) * | 2021-12-20 | 2022-04-08 | 中国人民解放军海军航空大学 | Flight quality self-adaptive evaluation method |
| CN114373360A (en) * | 2021-12-17 | 2022-04-19 | 清华大学 | Intelligent training system, method and device for flight simulator |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11893907B2 (en) * | 2019-10-14 | 2024-02-06 | Honeywell International Inc. | Systems and methods for evidence-based training of aircraft operators |
-
2022
- 2022-09-23 CN CN202211168069.6A patent/CN115587693B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2007148867A1 (en) * | 2006-06-20 | 2007-12-27 | Industry-Academic Cooperation Foundation, Yonsei University | Method of technology evaluation |
| CN107220761A (en) * | 2017-05-24 | 2017-09-29 | 江西洪都航空工业集团有限责任公司 | A kind of training plane effectiveness measures of effectiveness computational methods |
| CN110008442A (en) * | 2019-03-21 | 2019-07-12 | 上海工程技术大学 | A kind of pilot training's control method based on pilot's comprehensive ability evaluation |
| CN112766729A (en) * | 2021-01-20 | 2021-05-07 | 中国人民解放军火箭军工程大学 | Method for evaluating capability of equipment maintenance training personnel |
| CN113298431A (en) * | 2021-06-21 | 2021-08-24 | 北京航空航天大学 | Aviation QAR big data-based pilot competence portrayal method and system |
| CN114373360A (en) * | 2021-12-17 | 2022-04-19 | 清华大学 | Intelligent training system, method and device for flight simulator |
| CN114299422A (en) * | 2021-12-20 | 2022-04-08 | 中国人民解放军海军航空大学 | Flight quality self-adaptive evaluation method |
| CN114241852A (en) * | 2021-12-30 | 2022-03-25 | 中国民航科学技术研究院 | Multi-source data fusion-based flight simulation training evaluation system and method for in-process aircraft |
Non-Patent Citations (1)
| Title |
|---|
| 多参数融合的飞行品质评估模型的建立;刘莉雯等;《计算机工程与科学》;第38卷(第6期);第1262-1268页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115587693A (en) | 2023-01-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Hsu et al. | Structuring critical success factors of airline safety management system using a hybrid model | |
| CN109345101A (en) | Evaluation in Education Quality analysis method based on comprehensive evaluation analysis method | |
| CN115587693B (en) | Method for grading flight training quality based on observable behaviors | |
| Tian et al. | Software requirements engineer’s ability assessment method based on empirical software engineering | |
| CN113793047A (en) | Pilot cooperative communication capacity evaluation method and device | |
| Oh et al. | Rethinking the validity of interviews for employment decision making | |
| CN114218290B (en) | Selection method for equipment man-machine interaction interface usability evaluation | |
| George et al. | Graduate attribute assessment in software engineering program at university of Ottawa–continual improvement process | |
| Norddin et al. | Selecting new lecturers using the Analytical Hierarchy Process (AHP) | |
| Dewi et al. | The Effect of Training Material and Instructor on the Behavior of Graduates of Teacher Academy Program with Learning Outcome as Intervening Variable (Case Study at XYZ Company) | |
| El-Hefnawy et al. | Towards a Composite index for measuring the higher education institutions in Egypt | |
| Koral-Kordova et al. | Improving capacity for engineering systems thinking (CEST) among industrial engineering students | |
| CN112184040A (en) | Platform for evaluating software engineering capability based on behavior and learning data | |
| CN117670601A (en) | Optimized initial flight training stage competence rating method | |
| Zhang et al. | Evaluation of the graduate attributes achievement on specialty of remote sensing science and technology | |
| CN117057673A (en) | Competence quantitative evaluation standard formulation method based on VENN model | |
| Gupta et al. | Decision analysis approach for quality in technical education | |
| Quan | Post competency evaluation model of English Majors Based on Data Mining | |
| Zhi et al. | Development of ERP Financial Statement Evaluation System | |
| Rauschenberger et al. | Measurement approaches for integrated reporting adoption and quality: A systematic literature review, conceptualization and a research agenda | |
| Kotova et al. | Expansion of the Students Educational Indicators Activity Interface in the Moodle Environment by Means of Intelligent Agents | |
| CN115600893A (en) | Intelligent evaluation method and system for navigation safety management of large-scale shipping hub | |
| CN117852952A (en) | Enterprise quality infrastructure comprehensive evaluation method and system | |
| Surbakti et al. | Application of fuzzy logic and importance performance analysis in assessing organizational agility of PT. IF Heavy Equipment Distributor | |
| Zhao et al. | An Analytical Study of Digital Competency Assessment Tools for Higher Education Teachers |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |