AU2013205845B2 - Flight data monitoring method and system - Google Patents
Flight data monitoring method and system Download PDFInfo
- Publication number
- AU2013205845B2 AU2013205845B2 AU2013205845A AU2013205845A AU2013205845B2 AU 2013205845 B2 AU2013205845 B2 AU 2013205845B2 AU 2013205845 A AU2013205845 A AU 2013205845A AU 2013205845 A AU2013205845 A AU 2013205845A AU 2013205845 B2 AU2013205845 B2 AU 2013205845B2
- Authority
- AU
- Australia
- Prior art keywords
- data
- flight data
- parameter
- flight
- test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Traffic Control Systems (AREA)
Abstract
A computer-implemented flight data monitoring method is provided for monitoring flight data, for example flight data obtained by a flight data recorder or quick access recorder 5 on an aircraft. The method comprises receiving input flight data, performing flight data validation on the input flight data to generate validated flight data, and performing flight data analysis on the validated flight data to detect at least one predetermined type of flight event. The events may be useful for generating an aircraft safety report, for example. By validating the data before performing the analysis, the number of events which are falsely detected based 10 on invalid or unreliable data can be reduced. [Figure 2A]
Description
1 BACKGROUND OF THE INVENTION Field of the Invention The present invention relates to flight data monitoring methods and systems. Description of the Prior Art 5 Any discussion of the prior art throughout the specification should in no way be considered as an admission that such prior art is widely known or forms part of common general knowledge in the field. For many years, commercial and military aircraft (including helicopters) have been required by law to carry a Flight Data Recorder (FDR), often referred to as a "black box", to 10 record certain system parameters. Initially this was so that if an aircraft crashed, then investigators could use the recorded data to find out why the crash happened. The recorder is commonly located in an area of the aircraft where it is most likely to survive the crash, for example the tail. However, later the idea of using the recorded data to prevent accidents (as well as 15 investigate them) grew. One drawback was the location of the FDR, which was not easy to access. Therefore, a quick access recorder (QAR) was introduced. The QAR records the same information as the FDR but is located where the maintenance crew can easily access the recorder to download the recorded flight data. Flight data from a flight data recorder or quick access recorder may monitored to 20 identify various events of interest, for example safety-critical events in which a given flight parameter goes beyond known safe limits. SUMMARY OF THE INVENTION According to a first aspect of the present invention, there is provided a computer 25 implemented flight data monitoring method comprising: receiving input flight data; performing flight data validation on the input flight data to generate validated flight data; and performing flight data analysis on the validated flight data to detect at least one 30 predetermined type of flight event. The inventors of the present technique recognised that input flight data obtained from a flight data recorder or quick access recorder is often of poor quality and so can trigger many false positive detections of flight events. For example, as many as two thirds of the detected 2 events may turn out to be caused by glitches and problems with the recorder rather than problems with the aircraft or the way in which the aircraft has been flown. Therefore, during flight data monitoring much time can be wasted in processing invalid flight events. To address this problem, flight data validation of the input flight data is performed 5 before performing flight data analysis on the validated flight data. The flight data validation performs cleaning of the input flight data, and the flight data analysis then uses the cleaned data to detect the flight events. This reduces the number of false positive detections of flight events, allowing further investigation of the detected flight events to focus on real events, reducing or avoiding time spent investigating invalid events. 10 The input flight data may comprise a set of parameters sampled at time intervals providing a series of data values for each parameter. For example, measured parameters may include aircraft heading, pressure or radio altitude, air temperature, acceleration in various directions, whether the landing gear is up or down, the extent to which flaps have been activated, and many other kinds of parameter. The sampling rate does not have to be the same 15 for each type of parameter. For example, it may be desired to sample different parameters with different sampling rates depending on how frequently each parameter is expected to change. The flight data validation may identify invalid data values in the input flight data. For example, within a set of otherwise reliable data values for a particular parameter, there may be 20 some data values which do not make sense in the context of other data values for that parameter or for other related parameters. For example, there may be an erroneous spike in the data values, or an impossible data value may appear that lies outside the possible range of values for that parameter. The flight data validation may identify the invalid values so that they may be excluded from the flight data analysis to reduce the likelihood that erroneous 25 events are detected based on invalid data. The flight data validation may also identify a parameter of the input flight data as an invalid parameter. In this case the entire parameter is deemed to be invalid, rather than just isolated data values for that parameter. For example, during a flight certain parameters would be expected to change (for example, altitude). If no change is detected, then the values for 30 that parameter may be considered invalid. Similarly, other tests may determine that the values measured for a given parameter are unreliable and cannot be trusted. The flight data analysis may exclude any parameter identified as invalid in the flight data validation, so that only parameters which are deemed to be reliable are considered during the analysis.
3 There are different ways in which data values or parameters can be identified as invalid. For instance, data values or parameters may have an associated validity flag indicating whether the data value or parameter is valid or invalid. Also, a separate validity report may be provided which the flight data analysis uses to determine whether certain 5 parameters or data values should be analysed. The report may also highlight which parameters have failed, enabling the analyst(s) to directly investigate faulty parameters on a priority basis. The data values or parameters which have been excluded from the flight data analysis may still be accessible to a flight data analyst who analyses the results of the flight data 10 analysis. For example, the flight data analysis may be performed initially using just the valid data, but the analyst may then view the invalid data separately or select to view events detected based on invalid data as well, so that the analyst may verify that the validation has been performed correctly. The flight data validation may also produce a validation report which indicates the 15 validity of the input flight data. For example, the validation report may indicate which data values or parameters have been identified as invalid and which validation tests have been failed by the invalid data or invalid parameters. This can be useful for identifying systematic problems with the flight data recording system which need to be repaired. The flight data validation may comprise many different validation tests for testing the 20 validity of the input flight data. These tests may be applied in different combinations and different orders. For example, the flight data validation may include a repeating pattern test for detecting occurrences of a repeating signal pattern in data values of a parameter of the input flight data. ARINC -429c is the technical standard used on the data bus for commercial 25 aircraft. An ARINC -429c bus often is used for transferring data from the aircraft sensors and control system to the flight data acquisition unit. Signals transmitted using an ARINC® 429c bus often contain a repeating pattern which is generated when no real flight data is being output via the bus. For example, sensors for sensing the airspeed of the aircraft are often not sensitive enough to record the airspeed when the speed is less than 40 knots. As a 30 result no real flight data is recorded when the speed is low and so during these periods the ARINC -429c patterns are recorded in the flight data. The ARINC -429c bus contains a "non-computation flag" indicating when the data transmitted is not valid flight data. As a result, the data acquisition unit generates the repeating pattern. By performing the repeating 4 pattern test, the patterns can be identified and masked so that they are not analysed by the flight data analysis. Also, the flight data validation may include a spike detection test for detecting spikes in the data values of a parameter of the input flight data. A spike can be detected based on the 5 rate of change of the data values from sample to sample. A spike is usually indicative of an error and so can be identified as invalid. The data values corresponding to the spikes can then be masked in the validated flight data so that they do not trigger events during the flight data analysis. The flight data validation may also include a range comparison test for determining 10 whether data values of a parameter of the input flight data are within a predetermined range. For example, certain parameters may have a range of allowable values, which is not possible for real data values to exceed. If a data value is detected outside the allowable range, it can be marked as invalid. The values outside the range can then be masked during the analysis so that they do not trigger an event. 15 A correlation test may also be included in the flight data validation, for determining a degree of matching between an actual relationship between two or more parameters of the input flight data and an expected relationship between the two or more parameters. According to another aspect of the present invention, there is provided a flight data validation method, comprising: 20 receiving flight data comprising a set of parameters sampled at time intervals providing a series of data values for each parameter; obtaining, from a configuration file, an expected relationship between two or more parameters of the received flight data; determining a degree of matching between the expected relationship and an actual 25 relationship between the two or more parameters of the received flight data; and generating result data based on the determined degree of matching. It will be appreciated that certain relationships between specific parameters of flight data can be expected to be correlated. For example, as the pressure altitude measured by the aircraft instrumentation increases, it can be expected that the radio altitude will increase also. 30 Similarly, as the altitude (pressure or radio) is seen to increase, a decrease in the measured air temperature could be expected due to the lower temperatures experienced at higher altitudes. If such an expected relationship is not present in the received flight data it can be inferred that one or more of the parameters include faulty data which cannot be trusted in assessing the 5 operational performance of the aircraft. In this way, the known kinematics of the aircraft and its operation in flight can be used to identify faulty data in the flight data recorder. The determining of the degree of matching may comprise determining a degree of correlation between the two or more parameters using a correlation equation selected in 5 dependence on the expected relationship. It will be appreciated that different relationships can be expected between certain different parameters, and each of these relationships can be expressed as a correlation equation which can be solved to determine whether the correlation between the parameters is as could be expected. The correlation test or step of generating result data may indicate a fail result for one 10 or more of the two or more parameters if the determined degree of correlation is outside a predetermined range. The determination of the degree of matching may comprise determining a regression line for the actual relationship between the two or more parameters and determining whether one or both of the gradient and offset of the regression line are within a predetermined range. 15 It will be understood that an expected gradient value and offset value along with permitted tolerances for each can be stored, and that the actual gradient and offset can be compared against the stored expected values and tolerances to determine whether the parameters are sufficiently correlated to be assumed valid. During the process of computation it may be assumed that the parameters against which a test parameter is tested, also known as reference 20 parameters, are already valid and have passed previous stages of validation. A fail result may be generated for one or more of the two or more parameters if one or both of the gradient and offset of the regression line are outside of the predetermined range. The correlation test may further comprise determining, from the configuration file, a flight phase of interest in the flight data; and extracting data values of the flight data 25 corresponding to the determined flight phase of interest from the received flight data. In this way, the expected relationship may relate to a particular flight phase of interest and the degree of matching may be determined based on the actual relationship between the two or more parameters in the extracted data values corresponding to the selected flight phase of interest. This is beneficial because certain relationships are only relevant to a particular flight phase. 30 For example, a relationship between the measured roll of the aircraft and its heading is only relevant in the turning flight phase. Preferably the flight phases associated with the data values are determined only using airspeed, pressure altitude and heading parameters. By keeping the number of parameters used to determine the flight phases to a minimum the 6 likelihood of faulty parameters corrupting the flight phase determination is reduced. In this way the flight phases can be determined even if only these three primary parameters are reliable. The determining of the degree of matching may comprise pre-processing one or more 5 of the two or more parameters of the received flight data. The pre-processing may comprise differentiating one or more of the two or more parameters. This pre-processing is required for example where a first parameter can be expected to vary along with a rate of change of a second parameter. In another example, the pre-processing may comprise shifting samples of one or more of the two or more parameters with respect to time to compensate for an expected 10 delay between a change in a first parameter and a corresponding change in a second parameter. This pre-processing is required for example where a second parameter can be expected to change as a result of a change in a first parameter, but not immediately. In another example, the pre-processing comprises combining two or more of the parameters to generate a derived parameter. For example, a lateral acceleration of an aircraft could be 15 expected to vary in a relationship with its heading and ground speed. The heading and ground speed can therefore be combined to form a single derived parameter for correlating with the lateral acceleration parameter. A relationship between two or more parameters may not be tested if one or more of the parameters has failed a validation test. This reduces processing requirements, and also reduces the likelihood of a faulty parameter polluting other parameters 20 with which it might be correlated. The order in which parameters are calibration and correlation tested may be pre-set based on a hierarchy of parameters. In particular, parameters of the flight data may be hierarchically related using a dependency tree in which one or more root reference parameters are defined as a root level in the hierarchy, and in which parameters to be tested are related to 25 one or more of the root reference parameters either directly or via one or more intermediate reference parameters. In this case, relationships between parameters to be tested and one or more direct or intermediate reference parameters are tested in order in a branch of the dependency tree starting with a parameter related only to one or more root reference parameters and progressing to a parameter related to one or more intermediate reference 30 parameters. A relationship between a parameter to be tested and a reference parameter is not tested if the reference parameter has previously failed a validation test. Other types of flight data validation tests include: 7 (i) a sampling rate analysis test for determining whether an actual sampling rate of a parameter of the input flight data substantially matches an expected sampling rate for that parameter; (ii) a change detection test for determining whether a parameter of the input flight data 5 demonstrates an expected change of value; (iii) a noise detection test for determining whether a noise level of data values of a parameter of the input flight data exceeds a predetermined threshold; and (iv) an expected value test for determining whether data values of a parameter of the input flight data have an expected value or a value within an expected range. 10 The flight data analysis may comprise comparing the validated flight data with event thresholds to identify the at least one predetermined type of flight event. The types of event and the thresholds for those events may be defined in an analysis specification. Different analysis specifications may be provided for different aircraft types and different aircraft operators. 15 The flight data monitoring method may be implemented by a computer or other kind of processing device in different ways. For example, a single software program may be executed by the processing device to perform both the validation of the input flight data and the analysis of the validated flight data. In other examples, two separate programs may be provided, one for validation and one for analysis. 20 In some cases existing flight data analysis software may be available which does not have a validation function of its own. However, it is possible to reuse this analysis software when implementing the present technique. This can be done by providing flight data validation software, which has at least one validation function for performing a validation test on the input flight data, and a software hook for modifying the functionality of the flight data 25 analysis software so as to link to the flight data validation software. When input flight data is to be processed by the analysis software, the hook may trigger a function call to at least one validation function in the validation software. The validated data is returned by the validation function, and the hook modifies the behaviour of the analysis software so that it analyses the validated flight data instead of the original input flight data. In this way, the present 30 technique can be implemented without needing to make significant changes to the existing analysis software.
8 According to another aspect of the present invention, there is provided a computer implemented flight data monitoring system for monitoring input flight data, the flight data monitoring system comprising: flight data validation circuitry configured to perform data validation processing on the 5 input flight data to generate validated flight data; and flight data analysis circuitry configured to analyse the validated flight data to detect at least one predetermined type of event. The flight data validation circuitry and flight data analysis circuitry may both be implemented using a single processing device (e.g. a computer) which executes one or more 10 programs for performing the validation and analysis. Alternatively, different devices may be used as the flight data validation circuitry and the flight data analysis circuitry. According to another aspect of the present invention there is provided a flight data validation system, comprising: logic for receiving flight data comprising a set of parameters sampled at time intervals 15 providing a series of data values for each parameter; logic for obtaining, from a configuration file, an expected relationship between two or more parameters of the received flight data; logic for determining a degree of matching between the expected relationship and an actual relationship between the two or more parameters of the received flight data; and 20 logic for generating result data based on the determined degree of matching. According to another aspect of the present invention there is provided a flight data validation method, comprising: receiving flight data comprising a set of parameters sampled at time intervals providing a series of data values for each parameter; 25 processing, at a first validation stage comprising one or more first stage validation tests, the data values associated with each of the parameters, determining a pass or fail condition for each parameter based on the first stage validation tests, and generating first stage result data; processing, at a second validation stage comprising one or more second stage 30 validation tests, the data values associated only with parameters for which a pass condition was determined at the first validation stage, and generating second stage result data; and generating a validation report comprising the first stage result data and the second stage result data.
9 The flight data validation may be performed in two stages: a first validation stage comprising one or more first stage validation tests which determine a pass or fail condition for each parameter, and a second validation stage comprising one or more second stage validation tests which only test the parameters which have passed the first stage of validation test(s). A 5 validation report may be generated comprising the results of the first and second stages. By separating the validation process into two stages, the second stage validation tests can operate only on data satisfying certain basic criteria. This reduces the likelihood of the second stage validation tests producing erroneous results based on poor quality data, and reduces the amount of processing required because certain parameters containing invalid data 10 will already have been filtered out. This is particularly beneficial for the expected case where at least some of the second stage validation tests are substantially more computationally intensive that the first stage processing tests. Disposing of as many faulty parameters based on the simpler and computationally less intensive first stage processing stage tests is therefore highly beneficial in terms of processing efficiency. 15 The first stage result data may comprise an indication of a failure state in relation to those parameters having data values which failed one or more of the first stage validation tests. The second stage result data may comprise an indication of a failure state for those parameters which failed one or more of the second stage validation tests. Part of a data validation process is not simply determining which data or parameters are valid and which are 20 not, but also identifying the extent to which the data or parameters are valid and potentially how they are invalid. This information may be valuable for diagnostic purposes. The first stage validation tests may comprise a sampling rate analysis test for determining whether an actual sampling rate of a parameter of flight data substantially matches an expected sampling rate for that parameter. This is important as the correct 25 sampling rate for a given parameter is a regulatory requirement. If an incorrect sampling rate is detected in such a case then the parameter will be deemed invalid and thus rejected. The first stage result data may indicate whether or not the actual sampling rate of a parameter of flight data substantially matches the expected sampling rate for that parameter. If the sampling rate analysis test determines that the actual sampling rate of a parameter of flight 30 data does not substantially match the expected sampling rate for that parameter, subsequent first stage validation tests are not executed. In this way, the processing burden on subsequent first stage validation tests is relieved. In one example, the sampling rate analysis test is conducted in relation to a single flight and comprises the steps of: 10 identifying a duration corresponding to the flight; calculating an expected number of samples for the flight from the identified flight duration and the expected sampling rate; and comparing the calculated expected number of samples with the actual number of 5 samples corresponding to the flight duration in the flight data; wherein the actual sampling rate of the parameter of flight data is determined to substantially match the expected sampling rate for that parameter if the calculated expected number of samples substantially matches the actual number of samples in the flight data. The first stage validation tests may comprise a change detection test for determining 10 whether a parameter in the flight data demonstrates an expected change in value. This test is based on the fact that certain parameters may be expected to change during the course of a flight. For example it can be expected that the airspeed would change. For such a parameter a lack of such a change would indicate faulty data. The first stage result data may be set to indicate whether or not the parameter of flight data demonstrates an expected change in value. 15 If the change detection test determines that the parameter of flight data does not demonstrate an expected change in value for that parameter, subsequent first stage validation tests may not be executed. In this way, the processing burden on subsequent first stage validation tests is relieved. The change detection test may comprise the step of determining whether a change in 20 the parameter is expected in dependence on a change flag in the flight data associated with the parameter of flight data. In this way, a parameter now showing a change is only rejected if the change flag indicates that a change is expected. The first stage validation tests may comprise a range comparison test for determining whether the data values of a parameter in the flight data are within a predetermined range. 25 This range would be set to exclude values which should be impossible for the aircraft to exhibit, but may extend beyond the normal operational envelope of the aircraft. The first stage result data may be set to indicate whether or not the data values of a parameter in the flight data are within the predetermined range. In particular, if a data value is not within the predetermined range, the data value may be marked as invalid. 30 The first stage validation tests may comprise a spike detection test for determining whether a parameter in the flight data comprises one or more spikes in its data values. The first stage result data may be set to indicate whether or not the parameter in the flight data 11 comprises one or more spikes in its data values. In particular, if a group of one or more data values are determined to be a spike, the group of one or more data values is marked as invalid. It will be appreciated that a small number of data spikes or out of range values in the data may merely reduce the amount of data which is useful, without necessarily invalidating 5 the parameter entirely. Accordingly, in one embodiment there is provided a step of determining whether the proportion of data values for the parameter which are marked as invalid is greater than a predetermined threshold value. If the proportion of data values for the parameter which are marked as invalid is greater than the predetermined threshold value, a fail condition is generated for the parameter. The fail condition may be indicated in the first 10 stage result data. It will be appreciated that the first stage validation tests could be conducted in a different order. Further, in some cases certain of the sampling rate analysis test, the change detection test, the range comparison tests and the spike detection test could be included in the second processing stage rather than the first processing stage. 15 The second stage validation tests may comprise a noise detection test for determining whether a noise level of data values of the parameter exceeds a predetermined threshold. The noise detection test may be conducted separately for each of a plurality of flight phases, which may be subject to differing noise conditions. If the noise level of data values for the parameter exceeds the predetermined threshold, subsequent second stage validation tests may 20 not be executed. In this way, the processing burden on subsequent second stage validation tests is relieved. Furthermore, this prevents the noisy parameter from polluting the results of the subsequent second stage validation tests. The second stage validation tests may comprise an expected value test for determining whether data values of the parameter have an expected value or a value within an expected 25 range. The second stage validation tests may also comprise an expected value test for determining whether data values of the parameter have an average value within an expected range. It will be appreciated that the expected range for the first of these expected value tests is likely to be wider than the expected range for the second of these tests. This is because the first test effectively looks for isolated large deviations from the expected value whereas the 30 second test effectively looks for systematic or more frequent smaller deviations from the expected value. The expected value test is likely to be conducted separately for each of a plurality of flight phases, since the expected values are likely to differ depending on the flight 12 phase. If the expected value has failed, subsequent second stage validation tests may not be executed. The second stage validation tests may comprise a correlation test for determining a degree of matching between the expected relationship and an actual relationship between the 5 two or more parameters of the received flight data. The correlation test may be as described in more detail below. A step of detecting repeating ARINC -429c signal patterns in data values of a parameter, and marking the data values corresponding to any detected repeating ARINC® 429© signal pattern as invalid may also be provided. This is intended to inhibit repeating 10 ARINC®-429© signal patterns from causing erroneous results or fail conditions in subsequent tests. Typically the ARINC test would be conducted between the change detection test and the range comparison test of the first processing stage. The flight phases associated with the data values may be determined only using airspeed, pressure altitude and heading parameters. As explained above, by keeping the 15 number of parameters used to determine the flight phases to a minimum the likelihood of faulty parameters corrupting the flight phase determination is reduced. In this way the flight phases can be determined even if only these three primary parameters are reliable. Further aspects of the invention may include a computer program for causing a computer or a group of computers to execute the above methods, and a recording medium for 20 storing the program. It is an object of the present invention to overcome or ameliorate at least one of the disadvantages of the prior art, or to provide a useful alternative. Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise", "comprising", and the like are to be construed in an inclusive 25 sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to". It will be appreciated that features of the above-described aspects and embodiments of the invention may be combined with features of other aspects and embodiments of the invention as appropriate and in combinations other than those explicitly set out. For example, 30 optional features of the first aspect of the invention may equally optionally be incorporated in embodiments according to other aspects of the invention, for example where the different aspects have corresponding features. BRIEF DESCRIPTION OF THE DRAWINGS 13 Example embodiments of the present invention will now be described with reference to the accompanying drawings in which like parts have the same designated references and in which: Figure 1 is a schematic block diagram of an example flight data recorder or quick 5 access recorder setup as would be provided on a commercial aircraft; Figure 2A schematically illustrates an example of a flight data monitoring system for monitoring flight data from the recorder of Figure 1; Figure 2B is a schematic flow diagram of an example of a flight data monitoring method including validating and analysing flight data ; 10 Figure 3 schematically illustrates a data validation system architecture according to an embodiment; Figure 4 is a schematic flow diagram of the first processing stage of Figure 3; Figure 5 is a schematic flow diagram of the second processing stage of Figure 3; Figure 6 schematically illustrates the various flight phases of a typical aircraft flight; 15 Figure 7 is a schematic flow diagram of the sampling rate test of Figure 4; Figure 8 is a schematic flow diagram of the change detection test of Figure 4; Figure 9 is a graph showing a typical data plot of the FDR recording illustrating invalid ARINC 429 data; Figure 10 is a schematic flow diagram of the repeating pattern ARINC -429c test of 20 Figure 4; Figures 11 A and 11 B show two example ARINC traces; Figure 12 is a schematic flow diagram of the MinMax test of Figure 4; Figure 13 is a schematic flow diagram of the ROC test of Figure 4; Figure 14 is a schematic flow diagram of the noise test of Figure 5; 25 Figure 15 is a schematic flow diagram of the calibration test of Figure 5; Figure 16 schematically illustrates an example calibration configuration file; Figure 17 schematically illustrates a relationship between a radio altitude and pressure altitude of an aircraft during the climb phase of a flight; Figure 18 is a schematic flow diagram of the correlation test of Figure 5; 30 Figure 19 schematically illustrates an example correlation configuration file; Figure 20 schematically illustrates a dependency tree for the correlation test of Figure 18; 14 Figure 21 schematically illustrates a plot showing an example relationship between radio and pressure altitude for an entire flight; Figure 22 schematically illustrates an example software implementation of the flight data monitoring method; 5 Figure 23 is a flow diagram illustrating a saturation test for detecting saturated portions of a parameter of the flight data and transitions between the saturated portions; Figure 24 is a graph illustrating an example of a parameter of the flight data including saturated portions; Figure 25 illustrates the flight data of Figure 24 after the saturated portions have been 10 masked by the saturation test; Figure 26 illustrates the flight data of Figure 25 after the transition portions have also been masked by the saturation test; Figure 27 is a flow diagram illustrating an example of a fluctuation test for detecting fluctuations in a data value with a rate of change less than the rate of change limit for the 15 ROC test; Figure 28 is a graph showing an example of a parameter of flight data which includes fluctuations caused by noise; and Figure 29 shows the flight data of Figure 28 after the fluctuations have been masked by the fluctuation test. 20 DESCRIPTION OF EXAMPLE EMBODIMENTS Referring to Figure 1, an on board flight data recording system 1 is shown. The flight data recording system 1 is fitted to an aircraft to provide "black box" functionality. The flight data recording system 1 comprises a flight data acquisition unit 2 which receives inputs from 25 a sensor 3 (only a single sensor is shown here for the sake of clarity and brevity, but it will be appreciated that multiple sensors can and are likely to be present) and the aircraft systems 4. The flight data acquisition unit 2 is programmed to sample and acquire particular flight data from the sensor 3 and the system 4 based on a logical frame layout 5. The sampled flight data is then stored on a flight data recorder (or quick access recorder) 6. In the remainder of this 30 application, references to "flight data recorder" should be understood to include a quick access recorder. The sensor 3 may measure airspeed, outside temperature, or pressure for example. The inputs from the aircraft systems 4 may be status signals from the navigation system or 15 collision avoidance system for example. The logical frame layout 5 may specify the parameters which are to be sampled, and the sampling interval for those parameters. It will be understood that different flight parameters may be sampled at different intervals/frequencies. The flight data recorder 6 may record onto tape or solid state memory and be designed to meet 5 crash worthiness requirements and can either be removed from the aircraft for data testing or provide a data output which permits data to be transferred to a hand-held download unit when required. As will be appreciated, the flight data is acquired and stored through various flight phases of the aircraft (taxiing, take-off, ascent, cruising, descent, final approach and landing), and may be collected over a number of flights over an extended period of time; the typical 10 minimum recording duration of a flight data recorder being twenty five (25) flying hours. The acquired flight data is to be processed to determine both that the aircraft is functioning correctly and that the flight data recording system 1 is itself collecting and storing flight data correctly. Figure 2A illustrates an example of a flight data monitoring system according to the present technique. The flight data downloaded from the recording system 1 15 is input to validation circuitry 8 as input flight data. The validation circuitry 8 performs validation processing on the input flight data to generate validated flight data. The validation processing cleans up the input flight data, which may be noisy or unreliable. For example, the validation processing may mask invalid data values or parameters. A validity flag may be used to identify the invalid data values or parameters. 20 The validation circuitry 8 also generates a validation report indicating the validity of the input flight data. For example, the validation report may indicate which data values or parameters have been identified as invalid and why. This can be useful to help to identify when repairs to the recording system 1 are needed. Also, from an analysis perspective, it helps analysts to directly investigate the parameters that have failed the tests and make 25 necessary suggestions to customers. The validated flight data is then analysed by analysis circuitry 9, to detect various events of interest. The analysis circuitry 9 may maintain an analysis specification which identifies the events to be detected and the triggering conditions for those events. The analysis circuitry 9 may also generate a safety report listing any safety-critical events which 30 have occurred. At least the safety-critical events, and optionally also other events, may be passed to a human analyst for further investigation. Data values or parameters which are identified as invalid by the validation processing are excluded from the analysis so that they do not trigger invalid events.
16 While Figure 2A shows an example in which distinct validation circuitry 8 and analysis circuitry 9 are provided, in other examples the flight data monitoring system may be implemented on a single processing device programmed to perform the validation and analysis processing. 5 Referring to Figure 2B, an example method of pre-processing and analysing flight data from a flight data recorder for the purpose of safety event monitoring is shown. At a step AO, flight data is received at a data processing apparatus, for example a suitably programmed personal computer or dedicated hardware rig, and processed into a format which permits flight data parameters to be subject to range-based validation and to be understood by a user. 10 For example, the flight data may be processed from a raw binary or hexadecimal form into a format whereby parameters of flight data are presented sequentially with respect to time as suitable engineering data units (e.g. knots or feet). The flight data may be obtained by direct electrical connection to the recorder, by way of removable recording medium from an external storage device (e.g. DVD or hard disk) provided by the aircraft operator, or from a 15 remote device via a communication network (e.g. the Internet). At step Al, the input data is subjected to data validation processing by the validation circuitry 8. The validation processing may include one or more validation tests which will be described in more detail below. At a step A2, for each parameter being analysed the analysis circuitry 9 compares the 20 validated flight data samples for the parameter with one or more ranges set in the event specification for that parameter. Invalid samples or invalid parameters are generally excluded from the analysis at this point. However, in some embodiments the user of the system may optionally select whether or not to include the invalid samples or parameters in the analysis. This can be useful to allow the user to compare the analysis of all samples and parameters 25 with the analysis of only the valid samples and parameters, to check the effect of the validation. At a step A3, if it is determined that the samples are not within any of the set ranges, then no event is triggered and so the procedure will return to the step AO for the next parameter in turn. 30 If some samples of data fall outside any of the safety ranges being tested, then the procedure moves on to a step A4, where an event corresponding to the range is triggered. At a step A5, an event type of a detected event is determined. The event type may either be an event which relates to the safety of the aircraft, for example if the parameter is a critical flight 17 parameter, or an event which does not relate to the safety of the aircraft, for example if the parameter is not a critical flight parameter. At a step A6, if the event is not a safety event then the event is merely logged at a step A7 without any operator input being required. If on the other hand it is determined at the step A6 that the event is a safety event, then the event is 5 flagged to pass to a human analyst at a step A8. While step A2 compares values for a one or more parameters individually, it is also possible to trigger an event when data samples for two or more parameters simultaneously meet respective threshold criteria for the event. Validation System Architecture 10 Figure 3 shows a top level flow diagram of a data validation system 10 which can be used for the validation circuitry 8. After flight data is transferred to the data validation provider from an airline and is converted into a suitable format (in the manner described in step AO of Figure 2), each parameter is processed in two processing stages, a first processing stage 12 and a second processing stage 14. The first processing stage 12 comprises a series of 15 tests which check the following: * That the rate at which the data is sampled matches the expected sampling rate for the parameter (sampling rate); * That parameters which are expected to show a change during the flight actually do so (change detection); 20 e That if the data is of ARINC -429c type, repeating patterns of values which are invalid are identified as such; * That the parameter data seen is within the operational envelope of the aircraft (min max); and * That no data spikes are present in the data (rate of change). 25 Figure 4 shows a flow diagram for the stage one tests and highlights the order in which the tests are to be executed. Stage one starts at a step B 1, where the sampling rate of a parameter of the flight data is compared with an expected sampling rate for that parameter. If the sampling rate is not determined to be correct at a step B2, then the parameter is rejected at a step B3. If the sampling rate is determined to be correct at the step B2, then the parameter is 30 passed to the change detection test at a step B4. The change detection test determines firstly whether there is a change (with respect to time) in the data samples of the parameter at a step B5. If there has been a change then the process continues to a step B7 (ARINC test), whereas if there has not been a change then the process moves to a step B6, where it is determined 18 whether a change in the data samples of the parameter is expected. If it is determined at the step B6 that a change is expected (and by virtue of the logic flow from B5 to B6 has not occurred) then the parameter is rejected at the step B3. If on the other hand it is determined at the step B6 that a change is not expected, then the process continues to the step B7. At the 5 step B7 it is determined whether or not the parameter has an ARINC -429c flag associated with it (an explanation of ARINC -429c is provided below). If the parameter does have an ARINC -429c flag associated with it then an ARINC -429c (repeating values) test is conducted at a step B8, and then a Min-Max test is conducted at a step B9 to determine if the data samples for the parameter fall within the operational envelope of the aircraft. If at the 10 step B7 it is determined that no ARINC -429c flag is associated with the parameter then the process proceeds directly to the step B9 where the Min-Max test is conducted. Following the Min-Max test, a rate of change test is conducted at a step B 10 to identify data spikes within the data set for the parameter. It will be appreciated from Figure 4 that the sampling rate test and the change detection test may lead to the immediate rejection of a parameter, while the 15 ARINC test, Min-Max test and rate of change test do not. The reasons for this will be explained in detail below. Parameters which have been rejected may be marked as invalid so that they are not analysed to detect flight events. Parameters which pass the tests of the first processing stage 12 are then passed to the second processing stage 14. The second processing stage 14 comprises the following tests: 20 e Expected Values Check * Noise analysis; * Calibration Check; and * Correlation check Figure 5 shows a flow diagram depicting the sequence of the stage two tests. The tests 25 of the second processing stage 14 are only run on parameters which successfully pass the first stage of testing at the first processing stage 12. The sequence of the stage two tests starts at a step C1 with noise analysis. If it is determined at a step C2 that there is excessive noise in the data samples for the parameter then a failure report is generated at a step C3. If on the other hand it is determined at the step C2 that there is not excessive noise in the data samples for the 30 parameter then an expected values check is conducted at a step C5, and a pass report in relation to noise conditions is generated at a step C4. If the expected values check at the step C5 indicates that the logic is as expected then a pass report in relation to the expected values check is generated at the step C4. A correlation test is then conducted on the data samples of 19 the parameter with respect to data samples of one or more other parameters at a step C7. If the expected values check at the step C5 indicates that the logic is not as expected then a fail report is generated at the step C3. If the noise and expected values checks are passed, then the correlation check is performed and the pass reports and correlation output are used to generate 5 a report at a step C8. If one or other of the noise check and the expected value check are failed then the fail report is used to generate a report at the step C8, but the correlation check is not performed. This is because the results of the correlation check cannot be relied on if the quality of the data as determined by the noise and logic checks is inadequate. The results for each test are stored at the end of each stage of analysis and then 10 compiled into a report 17 by a report generator 16. The report 16 is then both archived to a storage device 18 and also made available to the customer 19 via a website based graphical user interface for example. It will be appreciated that other combinations of validation tests may be used. It is not necessary to execute all of the tests shown in Figures 4 and 5 - only a subset may be used if 15 desired. Also, the tests may be performed in different orders. For example, the validation may implement the ARINC -429c (repeating values) test, data spike test, Min-Max test, and correlation test, without implementing the other tests. Flight Phases Flight phases are primarily used in the second processing stage 12 of Figure 3, as will 20 be described in detail below. For example, the correlation, calibration, noise and expected value checks of the data validation system 10 may utilise flight phase information heavily. A flight phase refers to a specific period of within a flight. Examples of flight phases are described below and can be seen in Figure 6 where the arrowed line indicates the altitude of the aircraft. 25 Each of the phases of flight has its own definition as follows: * Taxi - This phase includes the movement of the aircraft on the ground under its own power. * Takeoff - The point at which the aircraft transitions from moving on the ground to flying in the air. 30 0 Climb - The period during which the aircraft increases its altitude. * Cruise - The altitude of the aircraft remains constant (approximately).
20 * Turning (not shown in Figure 6) - The turning phase indicates that the aircraft is changing heading (the direction in which the aircraft is moving), which can occur either in air or on the ground. * Descent - The portion of flight where the aircraft decreases its altitude. 5 e Landing - The last segment of flight where the aircraft transitions from flying in the air to moving on the ground. This can also be defined as the moment where the landing gear of the aircraft makes contact with the ground. The flight phases are not included in the flight data stored in the flight data recorder, and instead must be derived from the flight data. All flight phases can be calculated using 10 only airspeed, pressure altitude and heading parameters, as can be understood from the following table. Flight Phase Definition and required parameters Taxi From first change of heading to takeoff or from landing to last change of heading. Uses heading and pressure altitude parameters. Takeoff After the airspeed increases through 80kts, the point where the rate of climb first exceeds 200 fpm. Uses airspeed and pressure altitude parameters. Climb Rate of climb greater than 800 fpm. Uses pressure altitude parameter. Turning Absolute rate of change of heading greater than 1.5 deg/sec. Uses heading parameter. Descent Rate of climb less than -800 fpm. Uses pressure altitude parameter. Landing Before the airspeed decreases through 80kts, the point where the rate of climb last falls below -200 fpm. Uses airspeed and pressure altitude parameters. Cruise Absolute rate of climb less than 300 fpm. Uses pressure altitude parameter. Note: The Rate of climb, measured in feet per minute (fpm) is the rate of change of pressure altitude. 15 Initial Input Information 21 The flight data validation system receives two types of input information. The first of these is a parameter data array. The parameter data array contains all the samples of data recorded by the flight data recorder for the parameter that is to be analysed. The second input is the information that is associated with the parameter (metadata). Examples of this 5 information are: * Parameter name * Sampling rate - The sampling rate (in Hz) at which the parameter is recorded by the flight data recorder. * Upper and lower operating limits - These limits are used to detect samples which are 10 outside the operational limits of the aircraft (see min-max algorithm described below for more information). * Rate of change limit - used to detect spikes in the data (see rate of change algorithm described below for more information). * Change flag - Indicates parameters that may not change through the course of the 15 flight (see change detection algorithm described below for more information). * ARINC -429c flag - used to indicate parameters that need to be checked for ARINC -429c behaviour (for more information on ARINC -429c, please see the ARINC algorithm described below). * Length of the flight - used for the sampling rate check for each parameter (for 20 information see the sampling rate analysis algorithm described below). * Flight phase - the phase of flight (such as cruise, turning, climb etc.) to be used in the second stage of analysis for the purposes of noise analysis and correlation. * Noise Limit - the threshold value against which the results of noise analysis are compared. 25 e Expected value limits - used for checking the parameter behaviour during specific phases of flight matches the expected behaviour. * Correlation limit - the threshold below which the parameter is deemed not to relate sufficiently to the independent parameter used in the comparison. * Regression limits - the minimum and maximum levels of the slope of the least squares 30 fit line. * Offset limits - the minimum and maximum value permissible for the offset of the slope of the least squares fit line from its expected position.
22 Throughout this specification the information listed above shall be referred to generally as parameter information. The data array is processed using the parameter information listed above and the algorithms described below. When the data has been processed, the results are added to a report which is made accessible to the customer via a 5 web based user interface as described above in relation to Figure 3. Flight data validation algorithms The algorithms for the tests mentioned above are described in detail below: Sampling Rate Analysis Parameters may be sampled at different frequencies and it is important that the flight 10 data recorder receives and records information at the correct rate for a particular parameter. The sampling rate test is intended to compare the rate at which the parameter should be recorded with the rate at which it is actually recorded. This is to ensure that the parameters are recording in line with regulatory requirements. Figure 7 shows a flow diagram for the sampling rate analysis algorithm. 15 The parameter data is passed into the sampling rate detection algorithm along with its associated information. From the parameter information, the parameter name, sampling frequency (in Hertz (Hz)) and the length of flight are extracted. The length of the flight is first converted from hours, minutes and seconds into seconds only at a step D1. This duration in seconds is then multiplied by the sampling frequency at a step D2. This provides an 20 estimate of the number of samples that are expected based on the duration of flight and sampling frequency. The number of samples in the parameter data array is obtained at a step D3 and the estimated number of samples is subtracted from it at a step D4. At a step D5, it is determined whether the difference between the actual and estimated numbers of samples is within a given tolerance of zero (for example a difference within ±15). If so, then the data is 25 determined to pass the test at a step D7 and a note of this is made in a report at a step D8. The algorithm is designed to allow for drop outs in the data in line with regulatory requirements. The parameter is then passed on to the next stage of analysis. If however the parameter does not pass the sampling rate test (the result of the subtraction at the step D4 is not close to zero) at the step D5, the parameter is determined to have failed the test at a step D6, and a note is 30 made in the report of the failure and the parameter does not pass on to the next stage of analysis. The parameter can be marked as invalid with an invalid flag to prevent it being analysed in subsequent steps. Change Detection Analysis 23 There are considered here to be two types of parameters with regard to change detection. Parameters such as altitude and airspeed are expected to demonstrate a change during every flight. However, there are parameters which will only show a change during specific conditions and as a result they may not trigger on every flight. Examples of these 5 include ground proximity warnings, fire alerts and month of the year. The change detection algorithm is used to check parameter data for a change that may have occurred during the flight. Parameters which show a change during the course of the flight pass the change detection test. However, the parameters which do not show a change must be separated into two categories: 10 (1) Parameter shows no change due to a fault; or (2) Parameter shows no change because specific conditions were not met during the flight. The algorithm should be able to distinguish between these two categories of parameters. To prevent the rejection of parameters which have not shown a change during the analysis but may not be faulty, a Change Flag is included in the parameter information. This 15 Change Flag is an indicator that the parameter may not change during the course of the flight. If no change is detected for the parameter but a Change Flag is shown in the parameter information, a note is made in the report. This note informs the customer's engineering team that the parameter has not been seen to change because the relevant conditions have not been met. They must then check the parameter manually to ensure that the sensor that provides 20 data to the flight data recorder is working correctly. A flow diagram of the change detection algorithm can be seen in Figure 8. The change detection algorithm receives the parameter information of parameter name, change flag and the parameter data array. At a step El, maximum and minimum values of the sampled data are extracted from the parameter data array. The minimum value is then 25 subtracted from the maximum value at a step E2 and the absolute value is computed. If it is determined at a step E3 that the result of this subtraction is greater than zero, a change is detected at a step E4 and the test is passed at a step E5. A note of this is made in the report at a step E6. If a parameter shows no change at the step E3 the algorithm looks for the presence of a 30 Change Flag at a step E7. If a Change Flag is present in the parameter information, a note is made in the report at a step E9 informing engineers that this parameter must be checked manually. The parameter does not proceed on to the next stage of analysis. If a flag is not 24 present in the parameter information, the parameter fails the test at a step E8 and it is not analysed by any other algorithms. A note is made in the report of the failure. ARINC Detection Algorithm ARINC®-429© is a technical standard in the aviation industry relating to the avionics 5 data buses on commercial and transport aircraft and helicopters. The standard defines the physical and electrical interfaces for a two wire data bus and the associated data protocol for the network. This is of interest in this particular application because on a large number of aircraft, information is received by the flight data recorder from ARINC -429c data sources. The ARINC detection algorithm only applies to information obtained from these sources. An 10 example of parameter data received from an ARINC®-429© source can be seen in Figure 9. The line plot of Figure 9 shows an airspeed plot for fourteen consecutive flights. The solid black areas contain data with a repeating pattern. Whilst these fourteen sections of flight data may be valid, the repeating pattern data must be filtered. This is to prevent the repeating pattern data from causing false warnings in other algorithms such as the rate of change 15 analysis algorithm. The pattern of ARINC signals can often vary from parameter to parameter and the validation process should be able to successfully detect and then mask or identify as faulty any ARINC pattern that is found within the data. The ARINC algorithm consists of two main elements: 20 (1) A function that dynamically detects any pattern(s) and returns the pattern(s) for use during faulty data detection; and (2) Marking values that match the pattern(s) detected in step (1) of the analysis as faulty. A flow diagram of the ARINC detection algorithm can be found in Figure 10. The following subsections describe the main elements of the ARINC algorithm in more detail. 25 Pattern Detection Algorithm This function receives an array of values for the parameter and an ARINC flag to indicate if the parameter possibly contains an ARINC signal. At a step F1 it is determined 30 whether the ARINC flag is set to true or false. If the ARINC flag is not set to true (i.e. is set to false) then the ARINC test is not continued and the parameter is passed on to the next test at a step F2. If the ARINC flag is set to true then the algorithm dynamically detects a pattern using the following steps: 25 At a step F3, traverse through the masked array, reading 12 elements at a time and storing it temporarily in memory. Assuming that a signal can either consist of 4 elements or 8 elements, the following checks are carried out at a step F4 in order to find an ARINC pattern: (1) A set of pattern lengths are determined at the beginning which are either in units of 4, 5 8 or 16. (2) The peak to peak value (difference between the maximum and minimum values in the 4 selected elements) is computed to ensure the pattern(s) do not contain flat valid data (e.g. 45, 45, 45, 45). (3) If elements 1 to 4 are exactly equal to elements 5 to 8 and the set of elements is 10 exactly equal to elements 9 to 12 then the pattern is determined to consist of 4 elements that are repeating. Therefore the first 4 elements are returned as the ARINC pattern. An example of a four element repeating pattern can be seen in Figure 11A. (4) If elements 1 to 4 are not equal to elements 5 to 8 but are equal to elements 9 to 12 then the ARINC pattern is classified as an 8 element repeating pattern. The first 8 15 elements are therefore returned as the ARINC pattern. An example of an eight element repeating pattern can be seen in Figure 11 B. (5) Flight data can contain multiple ARINC patterns in one signal so it is important that the algorithm be capable of detecting each of these patterns. The signal is reversed and the above steps F3 and F4 are applied again. Any patterns that are detected during 20 this second run are then reversed so that they are in the order that was originally passed into the algorithm. (6) The detected patterns are then compared and the unique patterns are stored at a step F5 for use by the ARINC validation algorithm. (7) In order to make sure the data array is divisible by the length of the pattern(s), it is 25 padded with invalid values. These invalid values do not interfere with the correct detection of the pattern and only aligns the length of the data to a factor of the pattern length. (8) The data array is divided into chunks corresponding to each pattern length and all combinations of the pattern relative to the data are stored. Taking into account all the 30 combinations make the system more robust as often patterns are not recorded in the most efficient manner due to time delay and can therefore be out of step with the original pattern. ARINC Validation 26 When validating the ARINC signal, the ARINC pattern(s) detected previously is used to mask or otherwise identify as faulty any values in the parameter data that match the ARINC pattern. This is achieved as follows. At a step F6 a value is read from a parameter array. The read value is then matched with a l" value of a stored pattern at a step F7. At a step F8, data of the 5 parameter array equal to the pattern length is isolated, and is then matched with the stored pattern at a step F9. If a match is detected at a step F10, then the matched parameter data samples are masked (set to zero, null or some other predetermined value) or otherwise identified as faulty at a step F14 to distinguish them from regular data samples. It is then determined at a step F15 whether the end of the data set for the parameter has been reached. 10 If the end of the data set has not been reached then processing returns to the step F6. If the end of the data set has been reached then the process moved to a step Fl1 where it is determined if other ARINC patterns are available (stored) against which the parameter data needs to be checked. If not then the ARINC detection process terminates and the parameter data is passed to the next test at a step F13. If it is determined at the step Fl that other 15 ARINC patterns are available, then another pattern is picked at a step F12 and then processing reverts to the step F6 where the data set is evaluated against the newly selected pattern. Referring back to the step F10, if a match is not detected, then processing moves on to the step Fl1. By way of the above technique, ARINC signals can be detected and masked off or otherwise identified as faulty so that they do not disturb subsequent validation testing. 20 MinMax Algorithm The purpose of the MinMax algorithm is to ensure that the parameter data contains only values that are within or only a little beyond the operational range of the aircraft. It should be appreciated that FDRs are used to examine potentially unsafe events, such as, straying beyond aircraft limits and hence the MinMax test preserves data in such occurrences. 25 A flow diagram representing the MinMax algorithm can be seen in Figure 12. Upper and lower operational limits are provided in the parameter information that is fed into the algorithm (in addition to the parameter name and the parameter data array). The algorithm first checks at a step G1 if the upper limit is set to 'None' and at a step G3 is the lower limit is set to 'None'. If either limit is set to 'None' a limit is unavailable for the 30 parameter, and a respective fail report is generated at respective steps G2 and G4. A note of this is made in the report at a step G16 and the parameter moves on to the next test. If both upper and lower limits are available, the algorithm parses through all the values in the parameter data array and compares the value with the upper and lower limits. If the value 27 exceeds either of these limits, the sample is masked or identified as faulty. This process is shown in steps G5 to G9. In particular, at the step G5 a data sample value is compared with the upper and lower limits indicated in the parameter information. At a step G6 it is determined if the value is less than or equal to the upper limit and greater than or equal to the 5 lower limit. If the value does not fall within this range then a mask is applied to that data sample at a step G7 and it is then determined at a step G8 whether the value is the last one in the parameter data array. If the value is determined at the step G6 to fall within the range, then the data sample is not masked and the process moves directly to the step G8. If it is determined at the step G8 that the value under consideration is not the last value in the data 10 array, then at a step G9 the process moves on to the next value in the parameter data array. Once the last value in the parameter data array has been handled (as determined by the step G8), then the number of masked points is counted and a percentage mask is calculated at a step G10. The percentage mask is the proportion of the data samples within the parameter data array which have been masked. At a step Gl, it is determined whether the percentage 15 mask exceeds a threshold maximum value (80% in the present example), and if so a fail report is generated at a step G12 indicating that the parameter has failed the MinMax test and indicating the percentage mask. If the percentage mask does not exceed the threshold maximum value then it is next determined at a step G13 whether the percentage mask is greater than 0% (although it will be appreciated that other low thresholds could be used), and 20 if so generates a pass with mask report at a step G14. If it is determined that the percentage mask is not greater than 0% it is determined that no limit exceedances are detected, and a pass result with no detected exceedances is generated at a step G15 and is noted in the report at the step G16. It will be appreciated that by the above technique, if any limit exceedances are detected but the total number of exceedances is less than the preset threshold, a pass result is 25 registered. If the number of masked data points exceeds the preset threshold a fail result is registered in the report along with the percentage of masked data points. The data is not analysed by any further algorithms in the case of a fail report with percentage mask but is analysed further in the case of a pass report with percentage mask (subject to the exception mentioned below). 30 Rate of Change Detection Algorithm The rate of change (ROC) test is an algorithm which checks for spikes in the data. A flow diagram for the rate of change detection algorithm can be seen in Figure 13. The parameter data array, parameter name, rate of change limit and the sampling frequency for the 28 parameter are passed into the algorithm as parameter information. The rate of change limit is obtained from the parameter information that is input into the algorithm with the parameter data array. The first step of the algorithm is to check the value of the ROC limit at a step Hi. If the ROC limit reads 'None', no limit is available for the parameter so the test fails at a step 5 H2 and a note is made in the report at a step H17. If the ROC limit contains any value except 'None' the limit is divided by the sampling frequency at a step H3 to calculate the maximum permissible change between each sample. When the maximum change has been computed, the differences between every consecutive pair of samples is calculated at a step H4. These differences are then converted into absolute values at a step H5 to account for any negative 10 numbers. The differences are then compared with the ROC limit at a step H6 and it is determined at a step H7 whether the ROC limit is exceeded. If the ROC limit is determined to be exceeded then the data samples are masked at a step H8. If however the ROC limit is determined not to be exceeded then it is determined at a step H9 whether the two consecutive values are the last two in the parameter data array. If there are further samples then at a step 15 H10 the next two consecutive values are selected and the processing returns to the step H4. Once the last two consecutive values in the parameter data array have been handled (as determined by the step H9), then the number of masked points is counted and a percentage mask is calculated at a step H1l. The percentage mask is the proportion of the data samples within the parameter data array which have been masked. At a step H12, it is determined 20 whether the percentage mask exceeds a threshold maximum value (80% in the present example), and if so a fail report is generated at a step H13 indicating that the parameter has failed the ROC test and indicating the percentage mask. If the percentage mask does not exceed the threshold maximum value then it is next determined at a step H14 whether the percentage mask is greater than 0% (although it will be appreciated that other low thresholds 25 could be used), and if so generates a pass with mask report at a step H15. If it is determined that the percentage mask is not greater than 0% it is determined that no exceedances are detected, and a pass result with no detected exceedances is generated at a step H16 and is noted in the report at the step H17. It will be appreciated that by the above technique, if any exceedances are detected but the total number of exceedances is less than the preset threshold, 30 a pass result is registered. If the number of masked data points exceeds the preset threshold a fail result is registered in the report along with the percentage of masked data points. The data is not analysed by any further algorithms in the case of a fail report with percentage mask 29 but is analysed further in the case of a pass report with percentage mask (subject to the exception mention below). It will be appreciated that both the MinMax test and the ROC test may result in certain portions of the parameter data array being masked so that some invalid data is ignored by the 5 analysis stage when detecting events, but the rest of the samples for that parameter are still considered. A determination of whether the parameter passes to the second processing stage may be based on the overall percentage mask. Noise Detection Algorithm (NDA) The level of noise shown in a signal can often be used as an indicator to the health of 10 the flight data recorder system and its associated sensors. One formula for computing the noise level in the signal is the root mean squared of each sample value minus the mean of the two adjacent samples, but it will be appreciated that alternative functions can be used to compute noise levels in the signal. The equation for the calculation of the RMS noise can be seen below: 15 Xn NI Xn-1 +Xn+1 2 R ~ ~ ~ J_1xn Cni R 1VN & "2i- (N- 2) A flow diagram for the noise detection algorithm can be seen in Figure 14. To initiate the noise detection algorithm, the following information is fed into the algorithm: * Parameter data. 20 e Flight phase of interest. * RMS noise threshold. * Flight phase information. The parameter data is passed into the algorithm with the noise threshold for the particular parameter and the flight phase information for the flight. At a step J1 a difference is 25 computed between each sample and its predecessor. At a step J2 a difference is computed between each sample and its successor. At a step J3 a difference is computed between the values calculated in the steps J1 and J2 and halved. Each of the values computed at the step J3 is then squared at a step J4. At a step J5, an average of the squared values is calculated, and at a step J6 the square root of the average (RMS value) is calculated. The RMS value is 30 then compared with the noise limit provided in the parameter information. If at a step J8 the limit is determined to be exceeded, then a flag is raised and put on the report at a step J10. If 30 however at the step J8 the limit is determined not to be exceeded, a report is made indicating the calculated noise level at a step J9. In either case, a report is generated at a step J11. At a step J12, the process continues onto the next stage. The data shown below shall be used as sample data to demonstrate the RMS method. 8 7 6 9 4 8 2 7 5 1 1 0 5 The parameter data is first divided into sections according to the flight phase information. FLIGHT PHASE INFORMATION: Samples 1 to 9 = Descent 10 Samples 10 to 12= Landing For the purposes of this explanation, a calculation of the noise during the descent phase shall be presented. The data for the descent phase is isolated for analysis: 8 17 |6 |9 |4 18 |2 |7 |5 15 The algorithm then iterates through the signal data calculating the difference between each sample and its predecessor (calculation set 1). The algorithm then iterates through the signal data once again calculating the difference between each sample and its successor (calculation set 2). The data from these calculations can be seen in tabular format below: Calculation Set 1 Calculation Set 2 -1 0 -1 -1 3 -1 -5 3 4 -5 -6 4 5 -6 -2 5 0 -2 20 31 The difference between each of the related values (shown above in adjacent columns) is then calculated. Calculation Set 1 Calculation Set 2 Difference -1 0 Not used -1 -1 -1 - (-1)= 0 3 -1 -1-(3) -4 -5 3 3 - (-5) 8 4 -5 -5 -(4)=-9 -6 4 4 -(-6) 10 5 -6 -6-(5)=-11 -2 5 5 - (-2) = 7 0 -2 Not used Each of the calculated values is then squared as can be seen in the table below. Calculation Calculation Difference / 2 (Difference)A2 Set 1 Set 2 -1 0 Not used Not used -1 -1 0 0 3 -1 -2 4 -5 3 4 16 4 -5 -4.5 20.25 -6 4 5 25 5 -6 -5.5 30.25 -2 5 3.5 12.25 0 -2 Not used Not used 5 'btal = 107.75 The total of the squared values is then divided by the total number of values to calculate the average: Number of samples in the data set excluding endpoints = 7 10 Total= 107.75 Average = 107.75 / 7 = 15.393 32 Finally, the square root of the average is calculated to provide an estimation of the noise in the signal: RMS = \Average RMS = I15.393 5 RMS = 3.9234 This calculated RMS value is compared with the noise threshold to determine if excessive noise was detected for the phase of flight. RMS Noise result = 3.9234 RMS Noise Threshold = 3.8 10 Result <= Threshold = FALSE RMS Noise result = FAIL If the calculated RMS value exceeds the threshold, the parameter data for that phase is not made available for future analysis. Consequently, a fail result for the phase is noted on the report. If the limit is not exceeded, a pass result for the phase is logged in the report. The 15 procedure outlined above is repeated for all flight phases for each parameter. Expected Values The Expected Values Check is designed to detect system errors that produce incorrect values at specific points in a flight and report them to an airlines' engineering team. An example of an Expected Values Check is ensuring that airspeed during take off is between 240 20 and 120 knots. A flow diagram of the Expected Values Check can be seen in Figure 15. To initiate the Expected Values Check algorithm, the following information is fed into the algorithm: * Configuration file containing: o List of parameters to be checked. 25 o Flight phase of interest. o Type of test to be applied. o Upper and lower thresholds for the parameter. * Flight phase information. The configuration file is a text based file containing the names of the parameters to be 30 checked, the specific phase of interest, the upper and lower thresholds for the parameter and the type of test to be applied. An example Expected Values configuration file is shown in Figure 16. The Expected Values configuration file in this case shows only two entries (rows), but it will be appreciated 33 that the Expected Values configuration file would be likely to include many entries potentially one per flight data parameter per flight phase. At a step K1 data is read from the configuration file. At a step K2 a parameter to be tested is selected from the configuration file. Parameter data is then obtained at a step K3, 5 and data is isolated based on the flight phase of interest specified in the configuration file at a step K4. At a step K5 the selected Expected Values test is performed. There are two different types of Expected Values test that can be applied to the parameter data. The first of these is the Phase Range test. This is similar to the min-max test described above in the sense that the data is checked to ensure all samples lie within a specific 10 range for the phase. The first entry in the configuration file of Figure 16 uses the phase range test and is extracted below: "Airspeed, Takeoff, Phase Range, 240, 120" 15 The above indicates that for the Airspeed parameter, the flight phase to be analysed is the takeoff phase. At the point of takeoff, the airspeed should lie between 240 and 120 knots. The algorithm iterates through the configuration file selecting one parameter for testing before moving on to the next. The name of the parameter is read from the list of parameters to be tested. The complete data array for the parameter is then obtained. Based on 20 the flight phase indicated in the configuration file and the flight phase information provided, the data for the phase is isolated for testing. The Phase Range test is then called where each sample of airspeed in the phase is compared with the upper and lower limits. If the value lies within the expected range, the relationship is classified as a pass. If the value is not within the expected range, the sample is masked. 25 The other available test is Phase Average test. This is used as an estimate for the behaviour of a parameter during a specific period of time. An upper and lower limit is provided to allow a tolerance between the expected average value and the calculated average value. The second entry in the configuration file of Figure 16 uses the Phase Average test and 30 is extracted below: "Acceleration Normal, Taxi, Phase Average, 1.1, 0.9" 34 The above indicates that for the Acceleration Normal parameter, the flight phase to be analysed is the taxi phase. While the aircraft is taxiing from the terminal to the runway, the expected average value should be approximately 1g. To allow for subtle fluctuations in the data, an upper limit of 1.1 and a lower limit of 0.9 are applied. 5 At a step K6 it is determined whether an Expected Values test result of the Expected Values test performed at the step K5 falls between Expected Value limits. If so, then a report that the Expected Values test has been passed is generated at a step K7. If the Expected Values test result does not fall between the Expected Value limits then the parameter is identified as faulty at a step K8 and a report that the Expected values test has failed for that 10 parameter is generated at a step K9. At a step K10, following on from one of the steps K7 or K9, it is determined whether the Expected Values test just completed was the last one in the configuration file. If there are further Expected Values tests to be completed then the process passes back to the step K2. If no further Expected Values tests are to be completed then a final report is generated at a step 15 K11. As will be understood from Figure 15, the algorithm iterates through the configuration file selecting one parameter for testing before moving on to the next. The name of the parameter is read from the list of parameters to be tested. The complete data array for the parameter is then obtained. Based on the flight phase indicated in the configuration file and 20 the flight phase information provided, the data for the phase is isolated for testing. The Phase Average test is then called where the average value for the phase is calculated using the isolated data set. The calculated average value is then checked to see if it lies between the upper and lower limits. If the calculated value lies within the expected range, the relationship is classified as a pass. If the value is not within the expected range, a fail result is recorded in 25 the report and the parameter is identified as faulty. For example, the calibration check process for a normal acceleration parameter is as follows. An entry from the configuration file is selected: "Acceleration Normal, Taxi, Phase Average, 1.1, 0.9" 30 Based on this entry, the algorithm obtains data for the Acceleration Normal parameter. Acceleration Normal 35 0.95 0.95 0.95 0.96 0.96 0.95 0.96 0.94 0.95 0.97 1.00 0.99 1.02 0.99 0.98 1.03 1.01 1.00 0.99 1.01 0.98 0.99 1.00 1.01 Based on the flight phase listed in the configuration file and the flight phase information the data for the Taxi phase is isolated. FLIGHT PHASE INFORMATION 5 Samples 1 to10 - Taxi Samples 11 to 17 - Takeoff Samples 18 to 24 - Climb Acceleration Normal 36 0.95 0.95 0.95 0.96 0.96 0.95 0.96 0.94 0.95 0.97 TOTAL = 9.54 The PhaseAverage test is applied and an average value for the Taxi phase is calculated by adding all the samples together and dividing by the total number of samples. Calculated average value = 9.54 / 10 5 Calculated average value = 0.954 The expected range for the parameter is then read from the configuration file entry. Upper threshold = 1.1 Lower threshold = 0.9 The calculated average value is then compared with the upper and lower limits to 10 ensure that it is within the expected range. If all calculated values for a given parameter lie within the expected range, the test passes. If any calculated value is outside the expected range the test fails. Correlation and Calibration Checks The final stages of the flight data validation system are the correlation and calibration 15 tests. These tests are considered together as a single calculation provides the numerical results for both tests. The aim of the correlation test is to check that relationships that exist between parameters are valid. An example of a correlation check is ensuring that as the altitude of the aircraft is increasing, the air temperature is decreasing. A diagram of parameters with a strong 20 correlation, namely pressure altitude and radio altitude, can be seen in Figure 17. The two traces represent the radio altitude of the aircraft and the pressure altitude of the aircraft. It can be clearly seen that for low altitudes, as the pressure altitude of the aircraft is increasing, the 37 radio altitude is also increasing. This indicates that there is a strong correlation between these two parameters. The purpose of the Correlation Check is to assess the strength of the relationship and the Calibration Check tests for the correct linear characteristics between pairs of parameters. 5 Once the strength of the relationship has been established as satisfactory, the mathematical relationship between the two parameters is tested to ensure that it lies within the expected range. A flow diagram of how the Correlation and CalibrationChecks assess the relationship between parameters can be seen in Figure 18. To initiate the correlation analysis the 10 following information is fed in to the algorithm at a step Li: * Correlation and Calibration file containing: o Pair of correlated parameters. o Optional pre-processing required for each parameter o Nature of the relationship 15 o Correlation coefficient threshold o Calibration coefficient limits o Flight phase of interest * Flight phase information. The Correlation and Calibration file is a text based file containing the names of the 20 parameters to be correlated, the specific phase of interest, the expected nature of the relationship and the correlation coefficient range. An example Correlation and Calibration configuration file is shown in Figure 19. The Correlation and Calibration configuration file in this case shows only four entries (rows), but it will be appreciated that the Correlation and Calibration configuration file would be likely to 25 include many more entries than this. The columns in the Correlation and Calibration configuration file comprise three parameter values Parl, Par2 and Par 3. The parameter Par 1 is the dependent parameter, and the parameters Par2 and Par 3 (one or both of which could be used in a single test) are reference parameters. The fourth column defines an expression representing pre-processing 30 that is to be performed on the reference parameters before the processing takes place. Typically this would be a differentiation of a reference parameter, but other operations may take place such as integration. The fifth column defines the nature of the expected relationship between the parameters. This may be a linear relationship which may or may not 38 pass through the origin of the axes (i.e. zero), a discrete relationship or a variable-discrete relationship for example. It will be appreciated that other relationships might be expected, such as a logarithmic relationship. The sixth column indicates the flight phase of interest. The seventh column indicates 5 an expected Correlation coefficient threshold, while the eighth and ninth columns respectively indicate an upper and lower value for the gradient of the Calibration. The tenth and eleventh columns respectively indicate an upper and lower offset for the Calibration. The twelfth and thirteenth columns are provided to define the expected conditional values for a discrete or a variable discrete relationship. In particular the twelfth column indicates an expected value for 10 Par 1 given the value or range specified in the thirteenth column. As will be understood from the above, in some cases one or more of the parameters will require pre-processing to establish a relationship, for example the roll attitude can be tested against the rate of change of the aircraft heading (rather than against the aircraft heading itself), and so the heading is differentiated prior to testing. 15 In Figure 19 this is indicated in the expression column in relation to the first entry (row) of the table as dp2. The full expression for dp2 may be as follows: dp2 = differentiate (parameter 2) = differentiate (heading) In the example of the second entry (row) of the table of Figure 19, the expression is dp2 * p3, indicating the following full expression: 20 dp2 * p3 = differentiate (parameter 2) x parameter 3 = differentiate (heading x groundspeed) Other expressions for pre-processing are of course possible. For example, two parameters may be related, but there may be expected a delay between a change in one of the parameters and the corresponding change in the other. In order to pre-process for this type of 25 relationship, the data samples of one or other of the parameters may be time shifted by an amount equivalent to the expected delay. The expected nature of the relationship listed previously refers to whether the relationship between the parameters is linear, quadratic, between two discrete parameters (discrete) or between one variable parameter and one discrete parameter (variable-discrete). 30 An example of a linear correlation test is the relationship between pressure altitude and radio altitude, whereas the relationship between radio altitude and landing gear is a variable discrete relationship. This nature is listed so that the correct correlation equation is applied to the data. For correlation testing of variable parameters, the modulus of Pearson's correlation 39 coefficient may be used, so that it is only necessary to test for positive values when checking the strength of the correlation. Thereafter the slope is used to identify the sign of the correlation. For linear relationships, the Pearson correlation equations listed below are applied to 5 determine the correlation coefficient, the gradient and the offset of the line of regression. 11r pranrters us e : x = feamn: pnrer y = deped pranrter N = nunbr cf sane pinis Wn anaysng tr dt e onPte Y, x Y, y Sx 2 , y2 so fit attir en cf 1 data anlysis sea: \e cn cacate b = slcp a =fs c = omelatio n gntitr (umigne finn b -NjAy -jxjy N~x 2 _ a=1Y -bix] N c= N~xy-Ex~y N~x2-(x N~y2_(y For correlation testing of relationships involving both variable and discrete parameters, the percentage of points that adhere to a specific relationship is calculated. The configuration file entries for correlation tests involving discrete parameters differ slightly 10 from the entries for linear correlation tests. While entries for linear correlation test contain values relating to regression (which are used to check the calibration of a parameter), it is not possible to perform calibration testing on relationships relating to discrete parameter data. In order to test relationships using discrete data, it is necessary to know how the dependent parameter would act based on the given behaviour of the reference parameter. An entry for a 15 relationship between a variable and a discrete parameter can be seen below: Gear Down (L), Altitude Radio, Ground, 0.90, 1, <=O 40 The above entry, corresponding to the fourth entry (row) in Figure 19, indicates that whilst the aircraft is on the ground, it is expected that when the radio altitude of the aircraft is less than or equal to zero then the landing gear should have a value equal to 1. For this particular method, when the data is obtained for both parameters and the relevant section of 5 data is isolated, the following matrix is applied to determine the strength of the relationship: Landing Gear = 1 Landing Gear = 0 Radio Altitude < 0 A B Radio Altitude > 0 B A 10 The area in the matrix marked 'A' indicate results which match the expected relationship listed in the configuration file. The areas marked 'B' indicate results which do not match. The correlation algorithm first iterates through the portions of data. If the radio altitude is less than or equal to zero and the landing gear is equal to 1, the counter increases by 1 for the top left box shown above. If the landing gear is equal to 0, the counter increases in 15 the top right box. Similarly, if the radio altitude is greater than 0 and the landing is equal to 0, the counter increases for the bottom right portion of the matrix. If the radio altitude is greater than 0 and the landing gear is equal to 1, the counter increases in the bottom left portion. When all samples of data have been checked, the 'A' counts are added together and divided by the total number of samples checked. For the relationship shown in the configuration file 20 example to pass the correlation test, over 90% of the results must lie within the 'A' section of data. This equates to a correlation coefficient of 0.9. If more than 20% of the results fall within the 'B' areas, a fail result is recorded in the report. The above method is applied to both variable-discrete and discrete-discrete correlations. The correlation coefficient threshold indicates the anticipated strength of the 25 correlation being tested. A correlation coefficient result that is close to 1 indicates that there is a strong correlation between the parameters being tested. Whereas a correlation coefficient result with values less than 0.5 indicates weak or random correlation between the parameters. The correlation coefficient test forms only one part of the Correlation and Calibration analysis. To ensure that the numerical relationship between parameters is correct, regression 30 testing can also be performed on the parameters. Regression analysis provides an estimation of how a dependent variable changes when an independent variable varies. In this system, Calibration testing is based on ensuring that the slope and offset of the regression line falls within the expected range. The primary reason for performing a Calibration Check is that a 41 relationship could potentially pass the correlation coefficient check while the scaling of the parameter is incorrect. The first stage of the Calibration Check is the calculation of the slope of the regression line based on the dependent and independent variables' data. This limit is then compared with 5 the Calibration Check gradient limits from the configuration file. If the gradient of the line lies within this range, the test is classified as a pass. However, if the calculated gradient exceeds either the upper or lower limit, the scaling is deemed to be incorrect. A fail result is recorded in the report and a note is made that the scaling may be incorrect for the parameter. As mentioned previously, incorrect calibration or scaling can also affect the offset of 10 the regression line. The offset is calculated as part of the correlation algorithm and the calculated value is compared with the Calibration Check offset limits. If the calculated offset value lies within the limit range, the relationship passes the test and a pass result is recorded in the report. If the calculated offset value exceeds either of the offset limits, then a fail result is recorded in the report and a comment is made in the report indicating that the Calibration 15 Check has failed. Finally, one or more flight phases of interest are listed for each relationship. Applying the correlation equations to all flight phases can result in inaccurate data for some relationships. An example of this is in altitude and temperature. As aircraft take off in one location and land in another, it is not uncommon for the temperature at takeoff and the 20 temperature at landing to differ. As a result, the overall results for the climb and descent sectors of flight may also differ. There may be a stronger correlation between the parameters during the climb phase than in the descent phase which has the potential to influence the overall correlation result. Therefore correlations are performed based on flight phase to provide more accurate results. To allow this, flight phase information is included in the 25 correlation algorithm so that the relevant phases of flight noted in the configuration file can be isolated and tested individually. An example of relationship information that is stored in the correlation configuration file (first entry in Figure 19) can be seen below: Independent variable: Heading (Rate of Change) 30 Dependent variable: Roll Phase(s): Turning In Air Relationship: Linear through zero Correlation threshold: 0.8 42 Calibration slope limits: 6.0, 8.0 To efficiently execute the correlation algorithms, the configuration file is used to generate a dependency tree at a step L2. A dependency tree is a directed graph which 5 represents the dependencies of parameters. It is used to derive the order in which parameters should be evaluated. An example of a dependency tree can be seen in Figure 20. From a correlation testing perspective, A, B, C and D represent parameters such as Altitude STD (A), Acceleration Normal (B), Altitude Radio (C) and Pitch Attitude (D). Figure 20 indicates that the relationship between parameters A and B (Altitude STD and Acceleration Normal) is to be 10 checked before the relationship between parameters B and D (Acceleration Normal and Pitch Attitude) can be analysed. During correlation analysis, the algorithm iterates through a branch of the dependency tree selected at a step L3, selecting one relationship (an edge) for testing at a step L4 before moving on to the next. If any of these relationships fail the correlation test, subsequent 15 correlations involving the test parameter are not tested. In the case shown previously, A is determined to be the reference parameter and parameter B is the test parameter. If the correlation between parameters A and B fails, no further correlations are completed using parameter B. Therefore the correlation between parameters B and D will not be tested. If the previous relationship passes the correlation test, the complete parameter data arrays for the 20 next relationship in the dependency tree are obtained. The parameter data relating to the relationship is obtained at a step L5, and then based on the flight phase indicated in the configuration file and the flight phase information, the data for the phase of interest is then isolated for testing at a step L6. The configuration file contains information regarding the nature of the relationship (e.g. linear or quadratic). This 25 determines which correlation equation is applied to the data to determine the correlation between parameters. By applying these equations to the data, the correlation coefficient can be calculated at a step L7. The value estimated for the correlation coefficient is then compared with the upper and lower limit for the relationship at a step L8. If the value lies within the expected range listed in the configuration file, the relationship is classified as a 30 pass at a step L9. If the value is not within the expected range, a fail result is recorded in the report at a step L10. If a fail result is recorded, at a step L11 other correlations using the test parameter are removed from the configuration file so that they are not analysed. Then, at a step L12 it is determined whether the correlation test was the last one in the current 43 dependency tree branch. If it is determined that the correlation test was not the last one in the current dependency tree branch, then the process returns to the step L4 where a further relationship is selected and analysed. If it is determined that the correlation test was the last one in the current dependency tree branch, then it is determined at a step L13 whether all 5 available branches have been tested. If not then a further branch is selected at the step L3. If all available branches have been tested then the correlation analysis ends and a final report is prepared. For example, one of the relationships in the configuration file shown previously was between Altitude STD and Altitude Radio: 10 "Altitude STD, Altitude Radio, Linear, 1, 0.8, Initial Climb" Based on this entry in the configuration file, the algorithm first checks the results from the first stage of analysis for the two parameters. 15 Altitude STD - Result = PASS Altitude Radio - Result = PASS As both parameters passed the first stage of analysis, the algorithm then obtains data 20 for each of the parameters as seen below: Altitude STD Altitude Radio 2 -2 2 -1 8 18 36 40 88 72 136 111 188 149 244 189 300 235 344 285 44 400 409 402 409 401 409 399 409 400 409 368 278 329 195 300 102 257 64 189 32 188 14 187 -1 187 -1 187 -2 A graphical representation of this data can be seen in Figure 21. If all the data for these parameters was analysed, the result may be inaccurate. Information about the phases of flight for this particular recording can be accessed to provide 5 a more accurate result. FLIGHT PHASE INFORMATION: Samples 1 to 2= Taxi Samples 3 to 10 = Climb Samples 11 to 15 = Cruise 10 Samples 16 to 20 = Descent Samples 21 to 24 = Landing The flight phase of interest for this particular relationship is the climb phase. Samples 3 to 10 are isolated for both sets of data as seen below: Altitude STD - Climb Altitude Radio - Climb 8 18 36 40 88 72 136 111 45 188 149 244 189 300 235 344 285 This data can then be analysed to generate a result for the correlation between radio altitude and pressure altitude during the climb phase. The configuration file states that the expected nature of the relationship is linear 5 therefore the data is processed against the linear correlation equation. The result of this equation is in the form of an absolute correlation coefficient which is a value between 0 and +1. Correlation Result = 0.97 To identify if the correlation passes or fails the correlation test, the calculated 10 correlation coefficient is compared with the threshold listed in the configuration file. If the result is greater than or equal to the threshold, a pass result is given. If the calculated correlation coefficient is below this threshold, a fail result is given. Correlation Lower Limit = 0.8 Result >= Lower Limit = TRUE 15 Correlation coefficient result = PASS The next stage of the correlation analysis is to test the scaling and offset of the relationship by performing a regression analysis. The gradient of the regression line for the data is calculated. Gradient Result = 0.7058 20 To identify if the relationship passes or fails the gradient test, the calculated gradient value is compared with the upper and lower limits listed in the configuration file. If the result is between the thresholds, a pass result is recorded. If the calculated gradient is above the upper threshold or below the lower threshold, a fail result is recorded. Gradient Upper Limit = 1.1 25 Gradient Lower Limit = 0.9 Result <= Upper Limit = TRUE Result >= Lower Limit = FALSE Correlation coefficient result = FAIL 46 The final stage of the correlation analysis is to determine if the offset of the regression line falls within the expected range for the relationship. The offset of the line of regression is calculated. Offset Result = 0.88 5 To identify if the relationship passes or fails the offset test, the calculated offset value is compared with the upper and lower offset limits listed in the configuration file. If the result is between the two thresholds, a pass result is recorded. If the calculated offset is above the upper threshold or below the lower threshold, a fail result is recorded. Offset Upper Limit = 0.9 10 Offset Lower Limit = 0.8 Result <= Upper Limit = TRUE Result >= Lower Limit = TRUE Correlation coefficient result = PASS When all the steps mentioned previously have been executed, the algorithm then 15 selects the next relationship listed in the configuration file for testing. The results of each of the correlation tests and the calculated correlation coefficients are noted in the report which is given to the customer upon completion of the flight data validation process. The above techniques permit the efficient and reliable validation of flight data with minimal or no user input. By performing validation before analysing the validated flight data, 20 false positive detections of events caused by unreliable flight data can be reduced, reducing the amount of time spent by flight data analysts in analysing invalid events. The results of all tests except for the ARINC c test are likely to be included in the final report. The ARINC c test is provided only to prevent the occurrence of false positives in subsequent tests, and its reporting is therefore not required. 25 It will be appreciated that the original flight data set may not be altered by the above processes. The report indicating the validity of the data in the flight data set may be provided as a separate document. Figure 22 shows a software implementation of the flight data monitoring method. The software includes flight data analysis software 20, a software hook 24, and flight data 30 validation software 28 including a number of validation functions fnl, fn2 corresponding to different validation tests such as those described above. The analysis software 20 receives the input flight data obtained by the flight data recording system 1, and in the absence of the software hook 24 would analyse the input flight data to detect the events of interest.
47 However, the software hook 24 intercepts the processing of the input flight data by the flight data analysis software 20 and calls one or more validation functions of the flight data validation software 28. The validation functions validate the input flight data to generate validated flight data. The software hook 24 modifies the behaviour of the analysis software to 5 analyse the validated flight data instead of the input flight data. This implementation can be useful to allow legacy analysis software 20 which does not have a validation function to be reused for implementing the present technique. The software hook 24 allows the analysis software 20 to be modified to implement the validation functions of the validation software 28, with little modification to the program code of the analysis software 20. 10 Saturation Test It has been seen that the radio altitude saturates to a specific value after a certain period of time. This is due to the fact that the recording range of the sensor is exceeded after a certain altitude is reached by the aircraft. Hence a point is attained where the recorded value 15 of radio altitude remains constant until the aircraft starts descending to a height at which the sensor can start recording again. As a result of the saturation, the recorded value of radio altitude above a certain height does not match with the correct pressure altitude and hence need to be masked out in order to avoid bad data polluting the test results. 20 A pre-processing stage may be carried out before the other validation tests in order to remove (mask) the saturated values while leaving the actual and correct recorded values intact. It is achieved by running the saturation algorithm just before the other testing algorithms. The saturation algorithm may be performed for any parameter which is likely to encounter saturation. 25 Figure 23 shows an example of the saturation algorithm. At step Ml, the flight phases for a specific flight are computed based on various parameters of the flight data. At step M2, data for the 'Fast' flight phase is extracted based on the flight phases derived in step M1 (the 'Fast' flight phase occurs when the airspeed is higher than a threshold level). This is 30 done since investigations have shown that the radio altitude sensors reach saturation during the 'Fast' phase. If saturation occurs for other flight phases, then data for these flight phases can also be extracted. By focusing on the specific flight phases and parameters for which 48 saturation is likely to occur, processing can be made more efficient since it is not necessary to process all the flight data. At step M3, the saturation points are computed throughout the 'Fast' phase. For 5 example, the saturation points can be detected if the difference between successive data values is zero for more than a predetermined number of consecutive data values. The start and end points of a saturation point can be identified based on when the data values stop varying and then start varying again. 10 At step M4, a filtering step is performed for filtering the saturation points for which the data values are above a threshold value (in this example, if the radio altitude is above 10 feet). When the radio altitude is less than 10 feet and the radio altitude remains constant for more than the predetermined number of consecutive data values, then it is likely that the aircraft is stationary on the ground, in which case the radio altitude values are correct and do 15 not need to be masked. For other parameters for which saturation is a problem, other thresholds can be set to filter out expected non-varying portions which do not need to be masked. At step M5, the duration of saturation is determined for a saturation point, based on 20 the number of consecutive samples for which the difference with respect to the previous sample is zero. At step M6, the duration of the saturation point is compared with a threshold duration (in this example, 5 seconds). If at step M6 the duration of the saturation is greater or equal to the threshold, then at step M7 the saturation point is masked from the data stream (for example, by marking the saturated data values as invalid or setting markers indicating the start 25 and end of the portion to be ignored for subsequent processing). If the duration of saturation is less than the threshold, then step M7 is omitted and the data values are considered valid. At step M8, it is determined whether there are any more saturation points which have not yet been processed. Steps M5, M6 and M7 are repeated for each saturation point until they have all been processed. 30 Figure 24 shows an example of a set of radio altitude values measured over time for a flight. As shown in Figure 24, there are several saturated portions 50 where the data values remain constant for some time. The saturated portions 50 may occur at different values of 49 radio altitude at different times in the flight, because when flying over objects at varying heights, such as mountains or other aircraft, the radio signal used to detect radio altitude may be reflected earlier or later than at other times, resulting in different values for the radio altitude. 5 As shown in Figure 25, by detecting periods of saturation and masking the saturated values, the saturated portions can be eliminated from the parameter stream. However, this leaves some transition points 60 where data changed from one saturated value to another. The transition points are of very short duration, often lasting for less than 30 samples. The 10 transition points are not detected as saturation points in step M3 because at the transition points the difference between successive data values is not zero. To make sure that the remaining transition points do not cause spikes which might pollute the test results, the transition points should be masked too. 15 For example, steps M9 to M11 of Figure 23 can be used to mask the transition points 60. At step M9, the transition points are computed. This can be done by breaking down the unmasked pieces of data into slices with indices indicating the start and stop of each slice. The end index of the slice can be subtracted from the beginning index (or vice versa) to compute the duration of transition points. At step M10, the duration of each transition point is 20 compared with a threshold duration (30 samples in this example). If the duration is less than or equal to the threshold, then the slice is regarded as a transition point and at step M11 eac transition point is masked out. Once the saturation and transition points have been masked, the data including the 25 mask is saved so that the other tests are supplied with an updated state of unpolluted parameter values. For example, Figure 26 shows an example of the radio altitude values with the saturation points 50 and transition points 60 masked, leaving the valid portions 70 where the radio altitude rises and falls as the aircraft ascends or descends. For the valid portions 70, the radio altitude can be combined with other measures of altitude such as pressure altitude, to 30 determine the actual altitude of the aircraft. The more sources of altitude information are used, the more reliable the altitude determination. On the other hand, the invalid portions of the radio altitude values can be masked, and at these times only the pressure altitude can be used to determine the aircraft altitude.
50 Process Jumps within Rate of Change Limit It has been noticed that some parameters display a noisy behaviour during the cruise phase of flight. A noisy parameter is defined as a continuous fluctuation of recorded values 5 which do not exceed the rate of change limit for it to be considered a data spike thus rendering the data spike algorithm ineffective in such a scenario. For example, see Figure 28, in which block 80 represents fluctuations of the parameter with a rate of change a little below the rate of change limit. The zoomed-in portion 90 shows an example of the fluctuations caused by noise which are too quick to see in block 80. The operator of the aircraft may demand that a 10 particular rate of change limit is used in the rate of change test of Figure 13, and so the rate of change limit cannot be lowered to filter out the continuous fluctuations. To address this problem, a fluctuation test may be performed to mask out values which are noisy but yet not noisy enough to be considered as data spikes. For example, the 15 fluctuation test (process jump test) may be as shown in Figure 27. At step N1 of Figure 27, the flight phases are computed. At step N2, the data values for the "cruise" flight phase are extracted. Investigations have found that the noisy fluctuations described above generally occur during the "cruise" flight phase, and so hence the fluctuation detection algorithm may focus on the cruise flight phase. If fluctuations also occur in other flight phases, then data for 20 these phases may also be extracted. At step N3, the sampling frequency of the data used to determine the "cruise" flight phase and the data values of the parameter being tested are aligned with each other. For example, the flight phase data may be sampled at 1 Hz, while the parameter being tested may 25 be sampled at 4 Hz. In this case, the parameter values may be sub-sampled so that 3 out of every 4 parameter data values are ignored to match the parameter sampling frequency with the flight phase sampling frequency. Alternatively, if the flight phase data is sampled at a greater frequency than the parameter values, then the flight phase data may be downsampled to match the sampling rate of the parameter values. The alignment of the sampling 30 frequencies allows the algorithm to determine the flight phase at each sample of the parameter being tested and also to extract the phases at correct indices with respect to the parameter to be tested. For example: if 'cruise' is derived from pressure altitude which is sampled at 4Hz, the computed phase would also be sampled at 4Hz as a result However, using a 4Hz phase to 51 test a parameter sampled at 1Hz (radio altitude) would mean the indices of data sections to be tested would be incorrect and often point at a location which does not exist. For instance, assuming a flight duration of 1 hour, length of recorded data for pressure altitude would be 14400 samples (1x60x60x4) whereas that of radio altitude would be 3600 (lx60x6Oxl). 5 Hence it is advantageous for the phase for which the parameter being tested to be aligned at the sampling rate of the test parameter. At step N4, the sections of the parameter values which correspond to the "Cruise" flight phase are determined, and for each "cruise" section, the root mean square value is 10 obtained. The root mean square value is determined by squaring the data values for each sample within the "cruise" section, determining the average of the squared data values, and then taking the square root of the average. The root mean square value is an indication of how far the data values stray from the mean value. The greater the root mean square value, the greater the fluctuation in data values within the "cruise" section, and so the more likely it 15 is that the fluctuations represent noise. Hence, at step N5, the root mean square value (RMS) is compared with a predetermined threshold. If the root mean square value exceeds the threshold, then at step N6 the data values in the cruise section are masked. If the root mean square value does not exceed the threshold, then step N6 is omitted. At step N7, it is determined whether there are any more "cruise" sections which are still to be processed, and 20 if so, then steps N4, N5 and N6 are repeated for each "cruise" section. Once all the "cruise" sections have been processed, then the test ends. As shown in Figure 29, by masking the portions of the data stream for which the root mean square is greater than a given threshold, the noisy fluctuations can be eliminated, leaving the valid portions of the data stream for subsequent analysis. 25 The fluctuation test can be performed either at a pre-processing stage or in the first or second stage of validation. Preferably, the fluctuation test is performed after performing the data spike test so that data spikes which can already be masked by the data spike test do not skew the root mean square value towards higher values. 30 Although illustrative embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various changes and modifications can 52 be effected therein by one skilled in the art without departing from the scope and spirit of the invention as defined by the appended claims. 5 CLAUSES: The following clauses define further features of embodiments of the invention which may be combined with features of the claims. 1. A computer-implemented flight data monitoring method comprising: 10 receiving input flight data; performing flight data validation on the input flight data to generate validated flight data; and performing flight data analysis on the validated flight data to detect at least one predetermined type of flight event. 15 2. The flight data monitoring method according to clause 1, wherein the input flight data comprises a set of parameters sampled at time intervals providing a series of data values for each parameter. 3. The flight data monitoring method according to clause 2, wherein a data value which 20 fails the flight data validation is identified as an invalid data value. 4. The flight data monitoring method according to clause 3, wherein the flight data analysis excludes data values identified as invalid by the flight data validation. 25 5. The flight data monitoring method according to any of clauses 2 to 4, wherein a parameter which fails the flight data validation is identified as an invalid parameter. 6. The flight data monitoring method according to clause 5, wherein the flight data analysis excludes a parameter identified as invalid by the flight data validation. 30 7. The flight data monitoring method according to any of clauses 1 to 6, wherein the flight data validation generates a validation report indicating the validity of the input flight data.
53 8. The flight data monitoring method according to any of clauses 1 to 7, wherein the flight data validation comprises a repeating pattern test for detecting occurrences of a repeating signal pattern in data values of a parameter of the input flight data. 5 9. The flight data monitoring method according to clause 8, wherein the flight data validation includes masking data values corresponding to the occurrences of the repeating signal pattern detected in the repeating pattern test, wherein the masked data values are excluded from the flight data analysis. 10 10. The flight data monitoring method according to any of clauses 2 to 9, wherein the flight data validation comprises a spike detection test for detecting spikes in the data values of a parameter of the input flight data. 15 11. The flight data monitoring method according to clause 10, wherein the flight data validation includes masking data values corresponding to the spikes detected by the spike detection test, wherein the masked data values are excluded from the flight data analysis. 1 lb. The flight data monitoring method according to any of clauses 2 to 11, wherein the 20 flight data validation comprises a fluctuation test for detecting fluctuation of the data values of a parameter of the input flight data. 11c. The flight data monitoring method according to clause 1 1b, wherein the fluctuation test includes masking a sequence of data values having the detected fluctuation, wherein the 25 masked sequence of data values are excluded from the flight data analysis. lid. The flight data monitoring method according to any of clauses 1lb and 11c, wherein the fluctuation test includes determining a fluctuation value for a sequence of data values and detecting the fluctuation in said sequence of data values if the fluctuation value is greater than 30 a predetermined threshold. 1le. The flight data monitoring method according to clause 1ld, wherein the fluctuation value comprises a root mean square value for the sequence of data values.
54 12. The flight data monitoring method according to any of clauses 2 to lie, wherein the flight data validation comprises a range comparison test for determining whether data values of a parameter of the input flight data are within a predetermined range. 5 13. The flight data monitoring method according to clause 12, wherein the flight data validation includes masking data values that are determined to be outside the predetermined range by the range comparison test, wherein the masked data values are excluded from the flight data analysis. 10 14. The flight data monitoring method according to any of clauses 2 to 13, wherein the flight data validation comprises a correlation test for determining a degree of matching between an actual relationship between two or more parameters of the input flight data and an expected relationship between the two or more parameters. 15 15. The flight data monitoring method according to clause 14, wherein the flight data validation comprises marking a parameter that fails the correlation test as invalid, wherein the flight data analysis excludes the parameter marked as invalid. 20 15a. The flight data monitoring method according to any of clauses 2 to 15, wherein the flight data validation comprises a saturation test for detecting saturated portions of the data values of a parameter of the input flight data, the saturated portions comprising portions of the data values for which the difference between successive data values is substantially zero. 25 15b. The flight data monitoring method according to clause 15a, wherein the saturation test includes masking said saturated portions of the data values, wherein the masked saturated portions are excluded from the flight data analysis. 15c. The flight data monitoring method according to any of clauses 15a and 15b, wherein 30 the saturated portions comprise portions of the data value for which the difference between successive data values is substantially zero for at least a predetermined number of consecutive data values.
55 15d. The flight data monitoring method according to any of clauses 15a to 15c, wherein the saturation test includes detecting transition portions of the data values for the parameter of the input flight data, the transition portions comprising data values corresponding to a transition between saturated portions of data values. 5 15e. The flight data monitoring method according to clause 15d, wherein the transition portions comprise portions of the data values which are not detected as saturated portions and which have a duration which is not greater than a predetermined threshold duration. 10 15f. The flight data monitoring method according to any of clauses 15d and 15e, wherein the saturation test includes masking the detected transition portions, wherein the masked transition portions are excluded from the flight data analysis. 16. The flight data monitoring method according to any of clauses 2 to 15f, wherein the 15 flight data validation comprises at least one of: (i) a sampling rate analysis test for determining whether an actual sampling rate of a parameter of the input flight data substantially matches an expected sampling rate for that parameter; (ii) a change detection test for determining whether a parameter of the input flight data 20 demonstrates an expected change of value; (iii) a noise detection test for determining whether a noise level of data values of a parameter of the input flight data exceeds a predetermined threshold; and (iv) an expected value test for determining whether data values of a parameter of the input flight data have an expected value or a value within an expected range. 25 17. The flight data monitoring method according to any of clauses 1 to 16, wherein the flight data analysis compares the validated flight data with event thresholds to identify the at least one predetermined type of flight event. 30 18. The flight data monitoring method according to any of clauses 1 to 17, wherein the flight data analysis is performed by flight data analysis software for analysing the input flight data; 56 the flight data validation is performed by flight data validation software comprising at least one validation function for performing a validation test on the input flight data; and the method includes a step of using at least one software hook to modify the functionality of the flight data analysis software to call the at least one validation function of 5 the flight data validation software and to analyse the validated flight data instead of the input flight data. 19. A computer-implemented flight data monitoring system for monitoring input flight data, the flight data monitoring system comprising: 10 flight data validation circuitry configured to perform data validation processing on the input flight data to generate validated flight data; and flight data analysis circuitry configured to analyse the validated flight data to detect at least one predetermined type of event. 15 20. At least one computer program which, when executed on a computer, controls the computer to perform the method of any of clauses 1 to 18. 21. A computer-readable recording medium storing the at least one computer program of clause 20. 20 22. A flight data validation method, comprising: receiving flight data comprising a set of parameters sampled at time intervals providing a series of data values for each parameter; obtaining, from a configuration file, an expected relationship between two or more 25 parameters of the received flight data; determining a degree of matching between the expected relationship and an actual relationship between the two or more parameters of the received flight data; and generating result data based on the determined degree of matching. 30 23. A flight data validation method according to clause 22, wherein the step of determining the degree of matching comprises determining a degree of correlation between the two or more parameters using a correlation equation selected in dependence on the expected relationship.
57 24. A flight data validation method according to clause 23, wherein the step of generating result data indicates a fail result for one or more of the two or more parameters if the determined degree of correlation is outside a predetermined range. 5 25. A flight data validation method according to any of clauses 22 to 24, wherein the step of determining the degree of matching comprises determining a regression line for the actual relationship between the two or more parameters and determining whether one or both of the gradient and offset of the regression line are within a predetermined range. 10 26. A flight data validation method according to clause 25, wherein the step of generating result data indicates a fail result for one or more of the two or more parameters if one or both of the gradient and offset of the regression line are outside of the predetermined range. 15 27. A flight data validation method according to any of clauses 22 to 26, comprising: determining, from the configuration file, a flight phase of interest in the flight data; and extracting data values of the flight data corresponding to the determined flight phase of interest from the received flight data; wherein the expected relationship relates to the flight 20 phase of interest and the degree of matching is determined based on the actual relationship between the two or more parameters in the extracted data values. 28. A flight data validation method according to clause 27, wherein the flight phases associated with the data values are determined only using airspeed, pressure altitude and 25 heading parameters. 29. A flight data validation method according to any of clauses 22 to 28, comprising pre processing one or more of the two or more parameters of the received flight data prior to determining the degree of matching. 30 30. A flight data validation method according to clause 29, wherein the pre-processing comprises differentiating the one or more of the two or more parameters.
58 31. A flight data validation method according to clause 29, wherein the pre-processing comprises shifting samples of one or more of the two or more parameters with respect to time to compensate for an expected delay between a change in a first parameter and a corresponding change in a second parameter. 5 32. A flight data validation method according to any one of clauses 29 to 31, wherein the pre-processing comprises combining two or more of the parameters to generate a derived parameter. 10 33. A flight data validation method according to any of clauses 22 to 32, wherein a relationship between two or more parameters is not tested if one or more of the related parameters has failed a validation test. 34. A flight data validation method according to any of clauses 22 to 33, wherein 15 parameters of the flight data are hierarchically related using a dependency tree in which one or more root reference parameters are defined as a root level in the hierarchy, and in which parameters to be tested are related to one or more of the root reference parameters either directly or via one or more intermediate reference parameters; relationships between parameters to be tested and one or more direct or intermediate reference parameters are tested 20 in order in a branch of the dependency tree starting with a parameter related only to one or more root reference parameters and progressing to a parameter related to one or more intermediate reference parameters; and a relationship between a parameter to be tested and a reference parameter is not tested if the reference parameter has failed a validation test. 25 35. A flight data validation system, comprising: logic for receiving flight data comprising a set of parameters sampled at time intervals providing a series of data values for each parameter; logic for obtaining, from a configuration file, an expected relationship between two or more parameters of the received flight data; 30 logic for determining a degree of matching between the expected relationship and an actual relationship between the two or more parameters of the received flight data; and logic for generating result data based on the determined degree of matching.
59 36. A flight data validation method, comprising: receiving flight data comprising a set of parameters sampled at time intervals providing a series of data values for each parameter; processing, at a first validation stage comprising one or more first stage validation tests, the 5 data values associated with each of the parameters, determining a pass or fail condition for each parameter based on the first stage validation tests, and generating first stage result data; processing, at a second validation stage comprising one or more second stage validation tests, the data values associated only with parameters for which a pass condition was determined at the first validation stage, and generating second stage result data; and 10 generating a validation report comprising the first stage result data and the second stage result data. 37. A flight data validation method according to clause 36, wherein the first stage result data comprises an indication of a failure state in relation to those parameters having data 15 values which failed one or more of the first stage validation tests; and the second stage result data comprises an indication of a failure state for those parameters which failed one or more of the second stage validation tests. 38. A flight data validation method according to clause 36 or claim 37, wherein the first 20 stage validation tests comprise a sampling rate analysis test for determining whether an actual sampling rate of a parameter of flight data substantially matches an expected sampling rate for that parameter. 39. A flight data validation method according to clause 38, wherein the first stage result 25 data is set to indicate whether or not the actual sampling rate of a parameter of flight data substantially matches the expected sampling rate for that parameter. 40. A flight data validation method according to clause 38 or clause 39, wherein if the sampling rate analysis test determines that the actual sampling rate of a parameter of flight 30 data does not substantially match the expected sampling rate for that parameter, subsequent first stage validation tests are not executed.
60 41. A flight data validation method according to any one of clauses 38 to 40, wherein the sampling rate analysis test is conducted in relation to a single flight and comprises the steps of: identifying a duration corresponding to the flight; 5 calculating an expected number of samples for the flight from the identified flight duration and the expected sampling rate; and comparing the calculated expected number of samples with the actual number of samples corresponding to the flight duration in the flight data; wherein the actual sampling rate of the parameter of flight data is determined to 10 substantially match the expected sampling rate for that parameter if the calculated expected number of samples substantially matches the actual number of samples in the flight data. 42. A flight data validation method according to one of clauses 36 to 41, wherein the first stage validation tests comprise a change detection test for determining whether a parameter in 15 the flight data demonstrates an expected change in value. 43. A flight data validation method according to clause 42, wherein the first stage result data is set to indicate whether or not the parameter of flight data demonstrates an expected change in value. 20 44. A flight data validation method according to clause 42 or clause 43, wherein if the change detection test determines that the parameter of flight data does not demonstrate an expected change in value for that parameter, subsequent first stage validation tests are not executed. 25 45. A flight data validation method according to any one of clauses 42 to 44, wherein the change detection test comprises the step of determining whether a change in the parameter is expected in dependence on a change flag in the flight data associated with the parameter of flight data. 30 46. A flight data validation method according to any one of clauses 36 to 45, wherein the first stage validation tests comprise a range comparison test for determining whether the data values of a parameter in the flight data are within a predetermined range.
61 47. A flight data validation method according to clause 46, wherein the first stage result data is set to indicate whether or not the data values of a parameter in the flight data are within the predetermined range. 5 48. A flight data validation method according to clause 46 or clause 47, wherein if a data value is not within the predetermined range, the data value is marked as invalid. 49. A flight data validation method according to any one of clauses 36 to 48, wherein the 10 first stage validation tests comprise a spike detection test for determining whether a parameter in the flight data comprise one or more spikes in its data values. 50. A flight data validation method according to clause 49, wherein the first stage result data is set to indicate whether or not the parameter in the flight data comprises one or more 15 spikes in its data values. 51. A flight data validation method according to clause 49 or clause 50, wherein if a group of one or more data values are determined to be a spike, the group of one or more data values is marked as invalid. 20 52. A flight data validation method according to clause 49 or clause 50, comprising: determining whether the proportion of data values for the parameter which are marked as invalid is greater than a predetermi2ned threshold value; and if the proportion of data values for the parameter which are marked as invalid is greater than the predetermined threshold 25 value, generating a fail condition for the parameter. 53. A flight data validation method according to clause 52, wherein the fail condition is indicated in the first stage result data. 30 54. A flight data validation method according to any one of clauses 36 to 53, wherein the second stage validation tests comprise a noise detection test for determining whether a noise level of data values of the parameter exceeds a predetermined threshold.
62 55. A flight data validation method according to clause 54, wherein the noise detection test is conducted separately for each of a plurality of flight phases. 56. A flight data validation method according to clause 54 or clause 55, wherein if the 5 noise level of data values for the parameter exceeds the predetermined threshold, subsequent second stage validation tests are not executed. 57. A flight data validation method according to any one of clauses 36 to 56, wherein the second stage validation tests comprise an expected value test for determining whether data 10 values of the parameter have an expected value or a value within an expected range. 58. A flight data validation method according to any one of clauses 36 to 57, wherein the second stage validation tests comprise an expected value test for determining whether data values of the parameter have an average value within an expected range. 15 59. A flight data validation method according to clause 57 or 58, wherein the expected value test is conducted separately for each of a plurality of flight phases. 60. A flight data validation method according to any one of clauses 57 to 59, wherein if 20 the expected value test is failed, subsequent second stage validation tests are not executed. 61. A flight data validation method according to any one of clauses 36 to 60, wherein the second stage validation tests comprise a correlation test for determining a degree of matching between the expected relationship and an actual relationship between the two or more 25 parameters of the received flight data. 62. A flight data validation method according to any one of clauses 36 to 61, comprising a step of detecting ARINC signal patterns in data values of a parameter, and marking the data values corresponding to any detected ARINC signal pattern as invalid. 30 63. A flight data validation method according to clause 55 or clause 59, wherein the flight phases associated with the data values are determined only using airspeed, pressure altitude and heading parameters.
63 64. A flight data validation system, comprising: logic for receiving flight data comprising a set of parameters sampled at time intervals providing a series of data values for each parameter; 5 logic for processing, at a first validation stage comprising one or more first stage validation tests, the data values associated with each of the parameters, determining a pass or fail condition for each parameter based on the first stage validation tests, and generating first stage result data; logic for processing, at a second validation stage comprising one or more second stage 10 validation tests, the data values associated only with parameters for which a pass condition was determined at the first validation stage, and generating second stage result data; and logic for generating a validation report comprising the first stage result data and the second stage result data. 15 65. A computer program, which when executed on a computer will cause the computer to perform a method according to any one of clauses 22 to 34 or 36 to 63. 66. A recording medium storing the computer program of clause 65.
Claims (8)
1. A computer-implemented flight data monitoring method comprising: receiving input flight data; performing flight data validation on the input flight data to generate validated flight data; and performing flight data analysis on the validated flight data to detect at least one predetermined type of flight event.
2. The flight data monitoring method according to claim 1, wherein the input flight data comprises a set of parameters sampled at time intervals providing a series of data values for each parameter.
3. The flight data monitoring method according to claim 2, wherein a data value or parameter which fails the flight data validation is identified as invalid.
4. The flight data monitoring method according to claim 3, wherein the flight data analysis excludes data values or a parameter identified as invalid by the flight data validation.
5. The flight data monitoring method according to claim 2 or claim 3, wherein the flight data validation comprises at least one of: (a) a repeating pattern test for detecting occurrences of a repeating signal pattern in data values of a parameter of the input flight data; (b) a spike detection test for detecting spikes in the data values of a parameter of the input flight data; (c) a range comparison test for determining whether data values of a parameter of the input flight data are within a predetermined range; (d) a correlation test for determining a degree of matching between an actual relationship between two or more parameters of the input flight data and an expected relationship between the two or more parameters; (e) a sampling rate analysis test for determining whether an actual sampling rate of a parameter of the input flight data substantially matches an expected sampling rate for that parameter; 65 (f) a change detection test for determining whether a parameter of the input flight data demonstrates an expected change of value; (g) a noise detection test for determining whether a noise level of data values of a parameter of the input flight data exceeds a predetermined threshold; (h) an expected value test for determining whether data values of a parameter of the input flight data have an expected value or a value within an expected range; (i) a saturation test for detecting saturated portions of the data values of a parameter of the input flight data, the saturated portions comprising portions of the data values for which the difference between successive data values is substantially zero; and (j) a fluctuation test for detecting fluctuation of the data values of a parameter of the input flight data.
6. The flight data monitoring method according to any one of the preceding claims, wherein the flight data analysis compares the validated flight data with event thresholds to identify the at least one predetermined type of flight event.
7. A computer-implemented flight data monitoring system for monitoring input flight data, the flight data monitoring system comprising: flight data validation circuitry configured to perform data validation processing on the input flight data to generate validated flight data; and flight data analysis circuitry configured to analyse the validated flight data to detect at least one predetermined type of event.
8. A non-transitory computer-readable recording medium storing at least one computer program which, when executed on a computer, controls the computer to perform the method of any one of claims 1 to 6.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
AU2015201516A AU2015201516B2 (en) | 2012-10-19 | 2015-03-24 | Flight data monitoring method and system |
AU2015201517A AU2015201517B2 (en) | 2012-10-19 | 2015-03-24 | Flight data monitoring method and system |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
GB1218856.1 | 2012-10-19 | ||
GB1218856.1A GB2494553B (en) | 2012-04-16 | 2012-10-19 | Flight data monitoring method and system |
US13/721,505 US9346557B2 (en) | 2012-04-16 | 2012-12-20 | Flight data monitoring and validation |
US13/721,505 | 2012-12-20 |
Related Child Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
AU2015201517A Division AU2015201517B2 (en) | 2012-10-19 | 2015-03-24 | Flight data monitoring method and system |
AU2015201516A Division AU2015201516B2 (en) | 2012-10-19 | 2015-03-24 | Flight data monitoring method and system |
Publications (2)
Publication Number | Publication Date |
---|---|
AU2013205845A1 AU2013205845A1 (en) | 2014-05-08 |
AU2013205845B2 true AU2013205845B2 (en) | 2015-04-30 |
Family
ID=50629083
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
AU2013205845A Active AU2013205845B2 (en) | 2012-10-19 | 2013-05-14 | Flight data monitoring method and system |
Country Status (1)
Country | Link |
---|---|
AU (1) | AU2013205845B2 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107909106A (en) * | 2017-11-14 | 2018-04-13 | 北京航空航天大学 | A kind of detection method of aircraft flight environment |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112084143B (en) * | 2020-08-14 | 2023-05-30 | 陕西千山航空电子有限责任公司 | Synchronous playback platform architecture method for multi-type flight parameter data |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6480770B1 (en) * | 1999-04-01 | 2002-11-12 | Honeywell International Inc. | Par system for analyzing aircraft flight data |
US20050192717A1 (en) * | 2004-02-26 | 2005-09-01 | Tafs William D. | Methods and systems for automatically tracking information during flight |
WO2012013548A1 (en) * | 2010-07-29 | 2012-02-02 | Sagem Defense Securite | Method and system for analyzing flight data recorded during the flight of an airplane |
-
2013
- 2013-05-14 AU AU2013205845A patent/AU2013205845B2/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6480770B1 (en) * | 1999-04-01 | 2002-11-12 | Honeywell International Inc. | Par system for analyzing aircraft flight data |
US20050192717A1 (en) * | 2004-02-26 | 2005-09-01 | Tafs William D. | Methods and systems for automatically tracking information during flight |
WO2012013548A1 (en) * | 2010-07-29 | 2012-02-02 | Sagem Defense Securite | Method and system for analyzing flight data recorded during the flight of an airplane |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN107909106A (en) * | 2017-11-14 | 2018-04-13 | 北京航空航天大学 | A kind of detection method of aircraft flight environment |
Also Published As
Publication number | Publication date |
---|---|
AU2013205845A1 (en) | 2014-05-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9346557B2 (en) | Flight data monitoring and validation | |
US10380277B2 (en) | Application of virtual monitoring of loads | |
CN102859133B (en) | Method and system for monitoring the level of oil contained in a tank of an aircraft engine | |
US7826943B2 (en) | Method and system for diagnosing faults in a particular device within a fleet of devices | |
EP3105644B1 (en) | Method of identifying anomalies | |
EP2637010B1 (en) | Method and system for monitoring a structure | |
CN107463161A (en) | Predict the method and system and monitoring system of the failure in aircraft | |
EP2384971B1 (en) | Method of determining a maneuver performed by an aircraft | |
US20150331975A1 (en) | A method for analyzing flight data recorded by an aircraft in order to cut them up into flight phases | |
CN104303122B (en) | By automatically determining the method that decision threshold monitors aircraft airborne ageing equipment | |
EP3173762B1 (en) | Systems and methods for fatigue monitoring | |
EP2597602A1 (en) | Aircraft trending systems and methods | |
KR101896457B1 (en) | The method of generating operational loads spectrum based on actual operational flight data | |
EP3173890A1 (en) | Fault detection methods and systems | |
AU2013205845B2 (en) | Flight data monitoring method and system | |
US20150120214A1 (en) | Non-regression method of a tool for designing a monitoring system of an aircraft engine | |
KR20190094884A (en) | Method of individual aircraft tracking based on actual operational flight data | |
EP1927915B1 (en) | State initialization for gas turbine engine performance diagnogtics | |
AU2015201516A1 (en) | Flight data monitoring method and system | |
CN114692760B (en) | Descent rate estimation model construction method, descent rate estimation device and electronic equipment | |
GB2494569A (en) | Flight data validation apparatus and method | |
Daouayry et al. | Data-centric helicopter failure anticipation: The mgb oil pressure virtual sensor case | |
Rokhsaz et al. | Effect of Data Filtering on Flight Loads Spectra | |
US20230108129A1 (en) | Systems and methods for assessing aircraft performance, aircraft fuel efficiencies, and aircraft fuel reduction technologies | |
KR101930022B1 (en) | The method of generating stress transfer function based on actual operational flight data |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FGA | Letters patent sealed or granted (standard patent) | ||
HB | Alteration of name in register |
Owner name: L3HARRIS FLIGHT DATA SERVICES LIMITED Free format text: FORMER NAME(S): FLIGHT DATA SERVICES LIMITED |