CN114506460A

CN114506460A - System and method for monitoring flap faults

Info

Publication number: CN114506460A
Application number: CN202210253022.3A
Authority: CN
Inventors: 曹俊章; 王伟达; 张瑜琦; 徐清; 孙全艳; 韩昕悦
Original assignee: Commercial Aircraft Corp of China Ltd
Current assignee: Commercial Aircraft Corp of China Ltd
Priority date: 2022-03-15
Filing date: 2022-03-15
Publication date: 2022-05-17

Abstract

A system and method for monitoring flap failure is disclosed. The system may include a camera configured to capture images of a plurality of airfoils of the flap, each airfoil including a plurality of extension segments corresponding to a plurality of flap detents, the plurality of extension segments having different visual indicia thereon; an image processing component configured to extract image features in the image corresponding to the visual indicia; a position determining component configured to determine a current position of each airfoil based on the image features extracted by the image processing component; and a fault condition determination assembly configured to determine whether the flap is malfunctioning based on the determined current position of each airfoil.

Description

System and method for monitoring flap faults

Technical Field

The invention relates to the field of aircrafts, in particular to a system and a method for monitoring flap faults.

Background

The high lift systems of modern large aircraft include slats located at the leading edge of the wing and flaps located at the trailing edge of the wing. The wing area is increased, the configuration is changed and the aircraft lift force is provided by the outward extension and downward bending of the leading edge slat and the trailing edge flap in the low-speed stages of the takeoff and landing of the aircraft, so that the reasonable running distance and the safe takeoff speed of the aircraft are ensured, and the climbing rate, the approach speed and the approach attitude of the aircraft are improved.

There are three serious failure modes for high lift systems:

a) actuator release/airfoil tip: one actuator of the single airfoil or a hinge block connected with the machine body inclines under the influence of external force, or one actuator generates blocking (Jamming) or free wheel rotation (Freewheeling) in the actuator, and meanwhile, the other actuator of the airfoil drives the airfoil to move continuously.

b) Airfoil asymmetry: the single airfoil does not move synchronously with the other airfoils, which may be a secondary failure of airfoil lean.

c) And the wing surface is not commanded, namely the actual position reached by the wing surface is inconsistent with the command of the slat handle.

If one or more types of failures of the airplane occur in the takeoff or landing stage, the serious failures can cause serious damage and even crash of the airplane body structure. The monitoring of such failures has therefore become an integral part of the design of modern aircraft high lift systems. Conventional tilt sensing means mount a tilt sensor to the structure adjacent the airfoil for sensing actuator throw/airfoil tilt. However, installing sensors requires more sensors and associated cabling, increases space usage and aircraft weight, and may be affected by installation errors.

Accordingly, there is a need in the art for an improved system and method for monitoring flap failure.

Disclosure of Invention

The present invention provides an improved system and method for monitoring flap failure. More specifically, the present invention provides a system and method for monitoring airfoil faults using image recognition techniques. One or more position markers may be provided on the airfoil extension corresponding to each flap detent, e.g., each airfoil may be provided with a unique visual marker corresponding to each extension detent. Visual markers on each airfoil surface may be captured by a camera as the flap moves. The system may identify the current position of the airfoil based on visual indicia captured by the camera. Further, when only a partial visual marking is displayed in the maximum extension, the system can calculate a proportional relationship of the captured visual marking to the complete marking of the corresponding detent, which proportional relationship can indicate the (partial) current detent of the airfoil. And comparing a plurality of current screens of one or more identified airfoils or comparing the current screens with expected flap screens, and judging whether the airfoils have faults of inclination, asymmetry, non-instruction and the like.

In one embodiment of the present invention, there is provided a system for monitoring flap failure, comprising: a camera configured to capture images of a plurality of airfoils of the flap, each airfoil including a plurality of stretches corresponding to a plurality of flap screens, the plurality of stretches having different visual indicia thereon; an image processing component configured to extract image features in the image corresponding to the visual indicia; a position determining component configured to determine a current position of each airfoil based on the image features extracted by the image processing component; and a fault condition determination assembly configured to determine whether the flap is malfunctioning based on the determined current position of each airfoil.

In one aspect, the current position of the airfoil is determined based on a visual marker of a maximum-extension section on the image of the airfoil, wherein when the maximum-extension section on the image of the airfoil has a complete visual marker, the current position of the airfoil is a complete position of the maximum-extension section; or wherein when the maximum extension has an incomplete visual marker on the image of the airfoil, the current detent of the airfoil is the complete detent of the next maximum extension plus the partial detent of the maximum extension.

In one aspect, the partial occlusion is determined based on a ratio of the incomplete visual indicia of the maximum extension compared to the complete visual indicia of the maximum extension.

In one aspect, a single airfoil includes a first set of visual indicia on the plurality of stretches and a second set of visual indicia on the plurality of stretches, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance, wherein the fault condition determination component determines that a tilt fault has occurred for the airfoil if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tilt threshold.

In one aspect, the fault status determination component determines that an asymmetric fault has occurred with a first airfoil and a second airfoil of the flap if a first current detent identified based on the visual indicia of the first airfoil differs from a second current detent identified based on the visual indicia of the second airfoil by more than an asymmetry threshold.

In one aspect, the fault status determination component determines an expected flap position corresponding to an operating command generated by a control computer for controlling flap movement, wherein the fault status determination component determines that a non-commanded malfunction of the flap occurs if a current position of one or more airfoils differs from the expected flap position by more than a threshold value.

In one aspect, the system for monitoring for flap failure further comprises an activation assembly that activates the camera in response to one or more of: the aircraft enters a takeoff or approach phase; flap operating handle movement; or to command flap fault monitoring.

In one embodiment of the present invention, there is provided an aircraft comprising: a flap coupled to a wing of the aircraft; and a system for monitoring flap failure as described in any of the above.

In one embodiment of the invention, a method for monitoring flap failure is provided, comprising: receiving images captured by a camera of a plurality of airfoils of a flap, each airfoil including a plurality of extension sections corresponding to a plurality of flap detents, the plurality of extension sections having different visual indicia thereon; extracting image features in the image corresponding to the visual markers; determining a current position of each airfoil based on the image features extracted by the image processing component; and determining whether the flap is malfunctioning according to the determined current position of each airfoil.

In one aspect, a single airfoil includes a first set of visual indicia on the plurality of stretches and a second set of visual indicia on the plurality of stretches, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance, wherein a tip failure of the airfoil is determined if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip threshold.

In an aspect, an asymmetry fault is determined for a first airfoil and a second airfoil of the flap if a first current detent identified based on the visual indicia of the first airfoil differs from a second current detent identified based on the visual indicia of the second airfoil by more than an asymmetry threshold.

In one aspect, the system for monitoring flap failure further comprises: determining an expected flap position corresponding to an operating command generated by a control computer for controlling the flap movement; and determining that a non-commanded failure of the flap occurs if the current position of one or more airfoils differs from the expected flap position by more than a threshold.

In one aspect, the system for monitoring for flap failure further comprises activating the camera in response to one or more of: the aircraft enters a takeoff or approach phase; flap operating handle movement; or to command flap fault monitoring.

Drawings

FIG. 1 is a schematic illustration of flap fault monitoring according to one embodiment of the present invention.

FIG. 2 is a schematic view of a visual indicia on a flap according to one embodiment of the present invention.

FIG. 3 is a schematic view of a flap portion extended according to one embodiment of the invention.

FIG. 4 is a schematic view of a tilt occurring during flap extension according to one embodiment of the present invention.

FIG. 5 is a schematic illustration of asymmetry occurring during flap extension according to one embodiment of the present invention.

FIG. 6 is a block diagram of a flap fault monitoring system according to one embodiment of the invention.

FIG. 7 is a flow chart of a method of flap fault monitoring according to one embodiment of the present invention.

Detailed Description

The present invention will be further described with reference to the following specific examples and drawings, but the scope of the present invention should not be limited thereto.

FIG. 1 is a schematic illustration of flap fault monitoring according to one embodiment of the present invention. The invention can capture the flap image through the camera, the flap can comprise one or more wing surfaces, each wing surface can comprise different visual marks corresponding to a plurality of extension screens, and the current screens of the wing surfaces can be determined through image recognition technology based on the visual marks shot by the camera.

Image recognition technology is a computer vision mechanism that utilizes a computer to process, analyze, and understand images to identify various patterns of objects and objects. With the great increase of the computing power of a Graphic Processing Unit (GPU) and a digital signal processing unit (DSP), the continuous proposition of new computing methods, the large-scale growth of available data resources and the continuous emergence of novel application modes, the image recognition and the application technology thereof show new trends in the aspects of research breadth and depth, the performance of recognition effect and the expansion of technology and application, thereby enabling the image recognition and the application technology to be used as a fault detection means in a high-lift system. The present invention may utilize existing image recognition algorithms or suitable algorithms developed in the future to recognize visual indicia on the airfoil.

A typical sequence of operation of a high lift system is as follows: the pilot moves a Flap/Slat Control Lever (FSCL) to reach a command position and then stops. After detecting an effective handle command signal, a Slat/Flap Control Computer (SFCC) internally processes and analyzes the signal, and then sends an operation command signal to a Power Driver Unit (PDU). The PDU outputs a rotational torque that is transmitted through drive train components such as torque tubes, bearing mounts, etc., to a rotary gear actuator that in turn drives movement of a control surface (e.g., a flap airfoil, a slat airfoil). A Position Sensor Unit (PSU) at the wing tip feeds back Position signals of the control surfaces to the SFCC. And when the SFCC detects that the control surface reaches a sensor signal of a command position, sending a command signal to enable the PDU to stop outputting torque, sending a command signal to a brake device on the PDU, and locking the transmission line system to further keep the control surface at the current position. When the SFCC detects faults of asymmetric airfoil, underspeed airfoil (jamming of an actuator) and the like, a command signal can be sent to a wing tip brake device, and a transmission line system is locked so as to keep the control surface at the current position.

According to one embodiment of the invention, a camera can be mounted on the vertical tail body structure to capture flap images, and the visual angle range of the camera can reach the distance between the outer edges of the outer flaps on the left side and the right side. Based on the visual indicia on the airfoil, the extended position of the flap (e.g., the current detent) can be determined and a determination made as to whether the airfoil has failed uncommanded, tilted, or asymmetrical, as described in more detail below.

FIG. 2 is a schematic view of a visual marker on one airfoil of a flap according to an embodiment of the invention. Each airfoil can include a plurality of

extension segments

231 and 234 corresponding to a plurality of flap detents. When the flap is not extended, a portion 230 of the flap is presented, but the

respective extension sections

231 and 234 are located below the wing and are not presented. When the pilot moves the flap/slat steering handle, a control computer (e.g., SFCC) determines operating commands and controls flap motion based on the handle steering commands. Different operating commands can cause the flaps to extend by different lengths, i.e. the detents corresponding to the operating commands.

For example, if the operating command is to extend the 1 detent, the flap extends a corresponding distance back from under the wing, causing the stretch 231 to be presented; if the operating command is to extend the 2 detents, the flap extends back a corresponding distance from under the wing so that the extension 232 is presented; and so on. That is, the

extension segment

231 and 234 are divided along the direction of movement of the flap such that the extension segment 231 corresponds to 1 detent, the extension segment 232 corresponds to 2 detents, the extension segment 233 corresponds to 3 detents, the extension segment 234 corresponds to 4 detents, and so on.

Thus, visual indicia may be added on each airfoil (e.g., upper surface) to distinguish between multiple stretches of the flap. The visual markers corresponding to the individual extension sections can be different from one another, so that the current position of the flap can be determined on the basis of the detected visual markers. Although FIG. 2 shows the division between the

various expansion segments

231 and 234 in dashed lines, it should be understood that the dashed lines need not be used as visual indicia on the airfoil.

The division between the

extension segments

231 and 234 is related to the flap configuration (e.g., flap size, distance of movement corresponding to each detent, etc.) and can be determined based on the flap configuration.

By way of example and not limitation, in one high lift system, the slat configuration is shown in the following table.

Handle position clip	Slat position (degree)	Flap position (degree)	Configuration(s)
				0	0	0	Cruise control system
1	20	8	High-weight takeoff
				2	20	18	Takeoff at normal weight
3	25	30	Fly-back
				FULL	25	40	Landing

The flap motion mechanism may be of the articulated type, i.e. the flap makes an arc motion about a hinge point on the pivot shaft via its associated rocker arm. The sequence of movement of the slat is: the slat extends out first, and the flap extends out later; the flap is retracted first and the slat is retracted later. The movement time of the slat was as follows:

because the flap motion mechanism is hinged, the width of each stretch (e.g., the dimension in the direction of flap motion) can be calculated more easily.

Let PI (PI) ═ 3.14159, and L be the distance of the flap airfoil to the hinge point. The theoretical width D1 of the extension section 231 corresponding to the flap detent 1 is 2 × PI × L × (8 ÷ 360), the theoretical width D2 of the extension section 232 corresponding to the flap detent 2 is 2 × PI × L × (10 ÷ 360), the theoretical width D3 of the extension section 233 corresponding to the flap detent 3 is 2 × PI × L × (12 ÷ 360), and the theoretical width D4 of the extension section 234 corresponding to the flap detent 4(FULL) is 2 × PI × L × (10 ÷ 360).

While examples of determining the width of each extension segment are given above, it should be understood that the

extension segments

231 and 234 corresponding to each detent may be determined in any suitable manner.

A unique visual indicia may be added to each stretch to distinguish between the

different stretches

231 and 234. By way of example and not limitation, the visual indicia may be selected from graphics, letters, numbers, or a combination thereof. The pattern may be regular in shape, such as a triangle, a quadrangle, a circle or other polygons, or irregular in shape, such as a cloud or a complex composite pattern. In one example embodiment, a combination of shapes and numbers are selected as visual indicia to describe the position of the flap detent. In a further embodiment, a circle is selected and a number is embedded as the visual indicia.

Preferably, the size of the visual marking is adapted to the respective stretch (i.e. the area of the flap airfoil exposed in each flap configuration). For example, the visual indicia may occupy the entire width of each stretch, or more than a specified percentage thereof (e.g., more than 90%, more than 95%, etc. of the width of the stretch, etc.). By way of example and not limitation, the diameter of the circle selected may be adapted to the extension length of each flap configuration. In some implementations, the size of the visual indicia may be slightly less than the entire width of the respective stretch to allow no connection or overlap between visual indicia of adjacent stretches.

In one embodiment according to the invention, the visual indicia may have a different color than the airfoil to enable clear identification of the visual indicia on the airfoil. For example, the visual indicia may have a different color than the airfoil, such as red, purple, black, green, blue, fluorescent, and the like. The visual indicia may be formed from paint, tape, or other materials (e.g., metal, plastic, etc.). The visual indicia may be formed on the airfoil by coating, gluing, plating, and the like. Preferably, the visual indicia may be waterproof, abrasion resistant, or the like.

Figure 2 shows a schematic view of a circle and embedded numbers as visual indicia. The numbers within the visual indicia may have any orientation, such as perpendicular to the axis of rotation of the airfoil. For example, in the flap position 1 (circle + 1); the flap detent 2 is indicated by (circle + 2); the flap detent 3 is indicated by (circle + 3); the flap detent 4 is indicated by (circle + 4).

If the flap extends to the position of the position 1, the visual mark of the position 1 is displayed on the wing surface of the flap, and the visual marks of the positions 2-4 are shielded. If the flap extends to the 2 screens, the visual marks of the

screens

2 and 1 are displayed on the wing surface of the flap, and the visual marks of the screens 3-4 are blocked. If the flap extends to the 3 screens, the visual marks of the

screens

3, 2 and 1 are displayed on the wing surface of the flap, and the visual mark of the screen 4 is shielded. If the flap extends to the 4 detents, visual indicia of detent 4, detent 3, detent 2, and detent 1 are displayed on the flap airfoil. By analogy, if the flap is extended to N screens (e.g., N ≧ 4), then screens N, N-1.

In one embodiment, a single airfoil may include a set of visual indicia on multiple stretches. In another embodiment, a single airfoil may include a first set of visual indicia on a plurality of stretches (e.g., the visual indicia on the left side of FIG. 2) and a second set of visual indicia on the plurality of stretches (e.g., the visual indicia on the right side of FIG. 2), the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance. In addition, each set of visual indicia may be aligned or misaligned in the flap extension direction.

FIG. 3 is a schematic view of a visual indicia display when the flap is extended according to one embodiment of the present invention.

When the flap operating command indicates a detent, the flap may extend outward, exposing the visual indicia on the

extension segment

231 and 234. By recognizing the visual marking on the flap, the current position of the flap can be determined. Since the

extension sections

231, 232, 233, 234 are gradually exposed as the position of engagement increases, the extension section 234 can be considered to be larger than the extension section 233, the extension section 233 is larger than the extension section 232, and the extension section 232 is larger than the extension section 231.

The current position of the airfoil is determined based on the visual indicia of the maximum extension (i.e., the extension nearest the airfoil) on the airfoil image, and the current position may be determined as a full position or a partial position. When the maximum extension section on the airfoil image has a complete visual mark, the current position of the airfoil is the complete position of the maximum extension section. When the maximum extension has an incomplete visual marker on the image of the airfoil, the current detent of the airfoil is the complete detent of the next largest extension (i.e., the extension next to the wing) plus the partial detent of the largest extension.

For example, when the flap operating command indicates detent 1, the flap may extend outward and expose the extension segment 231. If a complete marking of the maximum extension 231 is detected, the current position of the airfoil can be considered to be a complete position of the maximum extension 231.

In another example, when the flap operating command indicates a detent 1, the flap may be extended outward to the position shown in fig. 3, which displays a partial visual indicia of the extension segment 232 (maximum extension) and a full visual indicia of the extension segment 231 (next maximum extension). Thus, the current position of the airfoil can be determined as 1.X, wherein 1 indicates the position of the second maximum extension 231 and X indicates the partial position of the extension 232. In one embodiment, the value of X may be determined based on a ratio of the incomplete visual indicia of the stretch section 232 compared to the corresponding complete visual indicia of the stretch section 232. For example, the ratio may be the height of the displayed visual indicia (e.g., the height in the direction of flap movement) divided by the height of the complete visual indicia. From this, it can be determined that the value of X is, for example, 0.3, 0.5, 0.8, etc., and accordingly it can be determined that the current position of the airfoil is 1.3, 1.5, 1.8, etc. According to one embodiment of the invention, it can be determined whether the current position of the airfoil has reached the desired flap position corresponding to the operating command. If the current position of one or more airfoils differs from the expected flap position by more than a threshold, a non-commanded failure of the flap can be determined.

In another embodiment, it may not be necessary to determine a ratio of the partial visual indicia to the full visual indicia for the stretch section 232. For example, in the case where there is a gap between the visual indicia of

adjacent extension segments

231, 232 to account for flap tolerance errors, once an incomplete visual indicia of the extension segment 232 is detected, the airfoil can be considered to have failed to reach an expected flap jam or to have an uncommanded fault.

FIG. 4 is a schematic view of a tilt occurring during flap extension according to one embodiment of the present invention. A single airfoil may include a first set of visual indicia (as shown on the left) and a second set of visual indicia (as shown on the left) on a plurality of stretches, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance. The exposed portions of the two sets of visual indicia are not identical when the airfoil is tilted.

Thus, a first current position of the airfoil (e.g., 1.8) may be identified based on a first set of visual indicia of the airfoil, and a second current position of the airfoil (e.g., 1.3) may be identified based on a second set of visual indicia of the airfoil. If the first current detent differs from the second current detent by more than a tilt threshold, a tilt fault of the airfoil can be determined.

FIG. 5 is a schematic illustration of asymmetry occurring during flap extension according to one embodiment of the present invention. Asymmetric failure can occur when the two airfoils extend to different degrees.

For example, a first current detent (e.g., 1) for the left inner flap may be identified based on the visual indicia for the left inner flap, and a second current detent (e.g., 1.7) for the right inner flap may be identified based on the visual indicia for the right inner flap. If the first current position differs from the second current position by more than an asymmetry threshold, an asymmetry fault can be determined for the two airfoils.

FIG. 6 is a block diagram of a flap fault monitoring system according to one embodiment of the invention. The flap Fault Monitoring system may include one or more cameras and a Fault Monitoring Unit (FMU) 610. The fault monitoring unit 610 may communicate with one or

more control computers

601, 602.

The

control computers

601, 602 may be slat control computers (SFCCs) or may implement flap control functions or the functions of the slat control computers described above. Furthermore, the

control computers

601, 602 also monitor flap faults via a fault monitoring unit 610. By way of example and not limitation, the

computers

601, 602 and the fault monitoring unit 610 may be connected using a data bus or hard wire, or may be connected using both a data bus and a hard wire. The data bus may be ARINC429, CAN, RS232/485, etc. The SFCC communicates with the FMU via a data bus as necessary, for example, data such as an "awake" command from the SFCC to the FMU, and a "flap failure" report from the FMU to the SFCC.

The FMU may also be connected to two SFCCs via a plurality of hard wires, such as two hard wires for "flap Tilt Fault" signaling, two hard wires for "flap asymmetry Fault" signaling, two hard wires for "Airfoil Uncommand Fault" signaling, etc. Additionally or alternatively, these fault signals may be transmitted over a data bus.

One or more cameras may be used to capture images of the flaps. The camera may be mounted on the fuselage in a position to capture an image of the flap. In one embodiment, two digital cameras may be employed, both mounted on a three-axis pan head mounted on the vertical tail structure of the aircraft. The purpose of using the cloud platform is to eliminate the influence of the vibration of the airplane body on the image pickup quality.

The two cameras may have the following basic features:

a) frame rate, preferably supporting 50/60fps full speed transmission;

b) image size, guaranteed to be 800 × 600 and above;

c) the protection security level should be guaranteed at IP60 and above.

The main features of the two cameras may differ as follows:

a) the imaging capabilities differ:

one camera can be a fog-transparent level camera, has basic optical fog-transparent characteristics and backlight shooting capability, and is mainly used in the daytime with better illumination conditions.

An aircraft typically performs an approach procedure at 3000 feet (1000 meters), i.e., begins to deploy flaps and landing gear. There is a cloud layer under the height, and suspended particles in the cloud layer can cause strong absorption and scattering effects on light. In this case, the image acquired by the imaging device is overall blurred, whitish, low in contrast, and severely loses image details. It is therefore necessary to select a fog-transparent level camera.

The other camera can be a starlight level camera and is mainly used for night conditions with poor lighting conditions. The two cameras can be switched according to the lowest illumination.

In one embodiment, the basic criteria for a fog-penetrating level camera are as follows: 1/1.8' progressive scanning CMOS devices are adopted, 3D digital noise reduction is realized, a 120dB wide dynamic range is realized, and optics and an algorithm are adopted to transmit fog. The basic indexes of the starlight level camera are as follows: by adopting 4/3' line-by-line scanning CMOS device and 20 times optical zoom lens, when the illumination condition is 0.012LUX, the color picture is converted into black and white picture, and 50 m infrared ray is compensated.

b) The photosensitive devices are different: one may use a CCD (charge coupled device) photosensitive device and the other may use a cmos (complementary metal oxide semiconductor) photosensitive device; or the two cameras adopt CCD photosensitive devices with different sizes; or the two cameras adopt CMOS light sensitive devices with different sizes;

c) the imaging colors are different: one may be a black and white camera and one may be a color camera;

d) the pixels are different: the pixels of one camera can be 200 ten thousand or less, and the pixels of one camera can be more than 200 ten thousand.

Although cameras with different imaging capabilities are listed above, in particular practice one camera or a plurality of identical cameras may be used.

The fault monitoring unit 610 can include an interface component 612 (e.g., an input/output I/O module), an image processing component 614, a jam determination component 616, a fault status determination component 618, a startup component 611, and the like. The fault monitoring unit 610 may also include a power module, memory, etc. For example, the power module can be connected to a 28V dc bus bar on an aircraft to convert the 28V 600Hz dc to dc of different magnitudes to power the modules and chips within the FMU, as well as the camera. The fault monitoring unit 610 may be implemented using a computer, processor, server, controller, or the like. In other embodiments, the fault monitoring unit 610 may be integrated into one or more control computers.

The interface component 612 can be responsible for receiving and transmitting data, such as implemented by an FPGA. In one embodiment, the interface component 612 can be configured to:

a) the system is connected with the camera 1 and the camera 2, and receives image frames sent by the camera in an acquisition period of 20 milliseconds or less;

b) connected to the

control computers

601, 602, the processing results of the fault monitoring unit 610 (for example in the form of bus data and discrete signals, respectively) are sent to each control computer;

c) the integrity of the data is guaranteed.

1) The interface FPGA can store 5V, 3.3V, 7.5V, 2.5V and ground reference voltage, and the data are periodically read and verified by the control chip.

2) The interface FPGA periodically generates a pseudo-random number sequence and sends the pseudo-random number sequence to the control chip, and the control chip returns the pseudo-random number sequence to the interface FPGA. If the returned sequence is the same as the original sequence, the interface FPGA resets the watchdog FPGA of the chip. Otherwise, a watchdog is triggered to cause the failure monitoring unit 610 to enter a fail-safe state.

The image processing component 614 may process the flap image captured by the camera to extract the visual indicia on the flap airfoil. Because of the influence of various factors such as body shake, natural light, weather conditions, etc., a certain degree of interference and noise are inevitably introduced into the acquired image, the image processing component 614 optionally eliminates these adverse factors in the preprocessing, and balances the light of the image through links such as image balancing and image enhancement. In the image enhancement step, algorithms such as image defogging and low-light enhancement can be included.

After the image from the above links is subjected to feature color extraction and image denoising, the image processing component 614 may use a preconfigured algorithm to extract visual markers on the flap airfoil surface. For example, depending on the shape and/or character of the visual indicia employed, the image processing component 614 may be preconfigured with suitable algorithms to determine whether the corresponding visual indicia or indicia (complete or partial visual indicia) are present in the captured flap image. The image processing component 614 may provide the identified one or more visual indicia for each airfoil of the flap. The image processing component 614 may also provide the relative positional relationship of these visual markers to facilitate determination of the visual marker for the most extended segment.

By way of example and not limitation, when visual features of circular embedded numbers are employed, the image processing component 614 can employ a Canny edge detection operator to extract edge information and then use a Hough transform to find circular regions, completing the identification of image features. In one embodiment, the feature color may use RGB, HSV, YUV, or the like color space. The HSV color model is insensitive to illumination factors, the illumination influence can be reduced to the maximum extent, and the physical significance of the model is relatively in line with the visual characteristics of people, so that the HSV color model can be optimized.

If there are two cameras capturing redundant flap images, the image processing component 614 should be able to obtain nearly identical image features (visual indicia) for the images captured by the first and second cameras.

The jam determination component 616 can determine the current jam of each airfoil based on the visual indicia of the airfoil identified by the image processing component 614.

In one embodiment, a circle may be used to represent a first detent, a rectangle a second detent, a hexagon a third detent, a pentagon a fourth detent, and so on. If the image processing component 614 identifies that a complete circle and a complete rectangle are present on an airfoil, the position determination component 616 can determine that the airfoil is in the second position. If the image processing component 614 identifies that a circle, a rectangle, and a portion of a hexagon are present on an airfoil, the position location component 616 can determine that the airfoil is in a portion of the third position location. Further, the position determining component 616 can determine the specific ratio of the portion of the third position based on the display ratio of the portion of the hexagon (e.g., the ratio of the height of the portion of the hexagon to the height of the complete hexagon). For example, if the visual indicia of the third detent presents 30% or 60%, the detent determining component 616 can determine that the airfoil is in the 2.3 or 2.3 detent position.

In one embodiment where circles plus numbers are used to indicate corresponding stretches, the position location component 616 can determine the position location of the airfoil based on the numbers in the identified circle.

The fault status determination component 618 can be configured to determine whether the flap is malfunctioning based on the determined current position of each airfoil.

In one embodiment, a single airfoil includes a first set of visual indicia on a plurality of stretches and a second set of visual indicia on the plurality of stretches, wherein the first set of visual indicia is spaced apart from the second set of visual indicia by at least a specified distance. The fault status determination component 618 can determine that the airfoil has a tilt fault if the first current position of the airfoil identified based on the first set of visual indicia differs from the second current position of the airfoil identified based on the second set of visual indicia by more than a tilt threshold.

In one embodiment, the fault status determination component 618 can determine that the first airfoil and the second airfoil have an asymmetric fault if the first current detent identified based on the visual indicia of the first airfoil differs from the second current detent identified based on the visual indicia of the second airfoil by more than an asymmetry threshold.

In one embodiment, the fault status determination component 618 can determine an expected flap position corresponding to an operating command generated by the control computer for controlling flap movement based on the operating command. The fault status determination component 618 can determine that a non-commanded failure of a flap occurs if the current position of one or more airfoils differs from the expected flap position by more than a threshold value. For example, the control computers 601 and/or 602 can transmit the operating instructions or their corresponding expected flap position to the fault monitoring unit 610 (e.g., the interface component 612), whereby the fault status determination component 618 can be aware of the expected flap position of the flap. The fault status determination component 618 can determine whether the flap is malfunctioning by comparing the identified flap current detent with the expected flap detent.

If the fault status determination component 618 determines that the flap is faulty, flap fault information can be reported to the control computer 601 and/or the control computer 602. In another embodiment, the fault status determination component 618 can report the fault monitoring results (e.g., normal, fault, status information, etc.) of the flaps to the control computer.

In one embodiment of the invention, the camera and/or the fault monitoring unit 610 (e.g., assembly 614 and 618) may be in a dormant or standby state when flap fault monitoring is not required, such as when the wing remains retracted during cruise procedures. In a sleep or standby state, the startup component 611 may detect whether a trigger event occurs in conjunction with the interface component 612 and wake up one or more cameras and/or other components of the fault monitoring unit 610 when the trigger event occurs. For example, the activation component 611 may activate the camera (and optionally the component 614 and 618) in response to one or more of: the aircraft enters a takeoff or approach phase; flap operating handle movement; or a command to perform flap fault monitoring.

In one embodiment, the activation component 611 may activate the appropriate camera based on ambient light conditions. For example, a clear fog level camera is activated when lighting conditions are good, and a starlight level camera is activated when lighting conditions are poor. In other embodiments, the activation component 611 may also activate or switch the camera based on ambient light changes, time changes, whether the pictures taken are on demand, etc.

The camera and fault monitoring unit 610 may continuously monitor for wing faults after startup. For example, the camera may take pictures (frames) of the wing at a specified sampling rate, and the fault monitoring unit 610 may process each frame to determine whether a flap fault has occurred. After the fault monitoring unit 610 determines that the flap has stopped moving for a certain time threshold, or after the flap operation command has occurred for a certain time threshold, the activation component 611 may cause the camera and the

component

614 and 618 to again enter the sleep or standby state.

In one embodiment of the present invention, the card position determining component 616 and the fault status determining component 618 can be implemented in combination, for example, using a CPU. The CPU may be divided into two channels, a control channel and a monitor channel. Each channel may be implemented by a Core in a dual or Multi-Core Processor (Multi Core Processor), or by a separate chip.

The control channel is used as a calculation link and processes a positive code form comprising image characteristic data; the monitoring channel is used as a checking link to process a complement form of image characteristic data. And the two channels respectively determine the current position clamping of the flap according to the received image characteristic data and judge whether the flap is in fault.

Both channels send the determination to the interface component 612. The interface component 612 performs a comparison after performing a unification process on the coding formats of the two sets of output results. If the results of the comparison are consistent, the interface component 612 sends the results to the two control computers as system output results.

If a consecutive number of data frame (e.g., 3 data frames) alignment between two channels is unsuccessful, the supervisory channel may trigger fail-safe logic to first notify both control computers of the status of the channel (e.g., via the ARINC429 bus) and then lock the fault monitoring unit 610 down. For example, the positions of the ARINC429 bus data SSM transmitted outwards are all set to "FAILURE WARNING". The control computer, upon receiving the signal, can immediately stop the flap from moving and lock the flap in the current position.

In one embodiment of the invention, after the airplane is powered on, the camera and the FMU enter a standby state after completing the power-on self-test, and wait for a wakeup command of the SFCC. Once one of the SFCCs detects that the flap lever is out of the last detent, the FMU may be awakened. The FMU may further activate different cameras based on minimum illumination or time. If the fog level camera fails, the FMU starts the starlight level camera.

The image processing component should be switched from the "standby" state to the "active" state within a certain time threshold Ttrans. This time threshold Ttrans should be less than the time of SFCC from the acquisition of the slat handle signal to the determination of the handle operating command. The SFCC can send the information of handle clamping position to the FMU, and the FMU can adjust the threshold value for judging the fault correspondingly.

After receiving the images transmitted from the camera, the FMU analyzes the image feature changes of all frames within a certain time threshold Tth one by one, and informs the SFCC of the feature change result (and/or failure). This time threshold Tth may be determined taking into account the following factors:

a) processing the image, determining the position clamping and judging the time required by the fault, and recording as Ta;

b) the time from the reception of the fault information to the generation of a locking instruction for a Power Drive Unit (PDU) brake device by the SFCC is recorded as Tb;

c) the time from the response of the PDU brake device to the completion of the locking action is marked as Tc;

d) a certain safety margin;

e) a safety factor, typically 1 to 3.

The time threshold Tth is therefore (Ta + Tb + Tc) -safety limit safety factor.

If the FMU determines that a flap malfunction (e.g., tilt, asymmetry, no command, etc.) has occurred, the SFCC will immediately lock the brakes on the power drive and the airfoil will be locked in the current position and will not move. If the high lift system is equipped with wingtip brakes, the SFCC will lock both the power drive unit brakes and the wingtip brakes.

Determination of the airfoil tip-off threshold Pth1 or airfoil asymmetry threshold Pth2 may take into account the following factors:

a) the angle of flap movement over time (Ta + Tb + Tc);

b) the flap allows for maximum tilt angles or asymmetric angles;

c) tolerance of mechanical parts, including clearance of a torque tube, a bearing support, a ball screw actuator and the like and a flap deviation angle caused by rigidity deformation;

d) a certain safety margin;

e) a safety factor, typically 1 to 3.

Thus, the airfoil tip threshold may be:

pth 1-maximum tilt angle allowed for the flap-angle of flap movement within (Ta + Tb + Tc) -safety margin.

The airfoil asymmetry threshold may be:

pth 2-maximum angle of asymmetry allowed for the flap-angle of flap movement within (Ta + Tb + Tc) -safety margin.

If the fault condition determination component 618 determines that the first current position and the second current position of the individual airfoil differ by more than the tilt threshold Pth1, then the airfoil tilt can be determined to have occurred. If the fault condition determining component 618 determines that the current position of the detent of one airfoil differs from the current position of the detent of the other airfoil by more than the asymmetry threshold value Pth2, then an asymmetric fault can be determined for the first airfoil and the second airfoil.

The FMU may process the image frames for each airfoil sequentially or concurrently, determine whether a pitch or non-commanded failure has occurred for each airfoil, and determine whether an asymmetric failure has occurred between the airfoils.

In one embodiment, for two sets of visual markers for a monolithic airfoil, a ratio R of the height of the two visual markers for the maximum span to the height of the expected full visual marker can be determined_i1And R_i2It is determined whether the two visual markers of the flap of the block differ by more than the tilt threshold Th 1. The tilt threshold Th1 may be a threshold obtained by applying appropriate conversion to Pth1, or may be an appropriate threshold determined in other ways as needed.

In another embodiment, the differential angle of the two side visual indicia may also be determined:

p ═ flap configuration_iExtension angle flap configuration_(i-1)Extension angle) × | R_i1–R_i2|

Actual angle of inclination P of the airfoil_actul＝P×K。

Note: k is a scaling factor, the difference angle P of the two visual markers may be different from the actual angle of inclination Pactal of the airfoil. The two angles are linear. The size of K is related to the distance between the center of the visual mark and the left edge and the right edge of the airfoil, and can be finely adjusted according to actual conditions. If P is_actul≥P_th1The current flap airfoil tip can be determined to be tilted.

In one embodiment, for any airfoil, such as the left outer airfoil, the left inner airfoil, the right inner airfoil, and the right outer airfoil, a ratio value R of the visual marker height to the expected full visual marker height may be determined and the difference R of any two airfoils calculated as ABS (R-Difference R)₁-R₂) It is determined whether the difference is larger than the asymmetry threshold Th 2. The tilt threshold Th2 may be a threshold obtained by applying appropriate conversion to Pth2, or may be an appropriate threshold determined in other ways as needed.

In another embodiment, the differential angle of the visual indicia of the two airfoils can also be determined:

p difference (flap configuration)_iExtension angle flap configuration_(i-1)Extension angle) x R difference;

if the P difference is greater than or equal to Pth2, it can be determined that the corresponding two airfoils are asymmetric.

In addition, the FMU may transition from the "on" state to the "standby" state after the visual indicia has not changed for a certain time threshold, and may also transition the camera to the sleep or standby state. This time threshold should be slightly larger than the tolerance of the flap movement time, e.g. 1 second, 2 seconds.

It should be noted that the manner of calculation and the values of the various thresholds are given above as examples only and are not limiting. In particular practice, the respective threshold values may be set as desired without departing from the scope of the present application.

FIG. 7 is a flow chart of a method 700 of flap fault monitoring according to one embodiment of the present invention. Method 700 may be performed by a fault monitoring unit as described above, or may be performed by a control computer, processor, computer, or the like.

After the airplane is powered on, the camera and the flap fault monitoring unit are firstly powered on for self-checking, and then are in a standby state after the self-checking is finished, and the command of the SFCC is waited.

At step 702, a wake-up signal may be detected. For example, the interface components in the fault monitoring unit may remain operational and receive the wake-up signal. The wake-up signal may be, for example, a command for the aircraft to enter a takeoff or approach phase, for flap lever movement, or for flap fault monitoring, etc. For example, after a double-start aircraft, the pilot may set the flaps as required by standard operating procedures, such as moving the flap handle from 0 position to 2 positions. And when the SFCC monitors that the flap control handle leaves the position of 0 clamp, the SFCC wakes up the flap fault monitoring unit.

At step 704, after detecting the wake-up signal, the fault monitoring unit and the camera may be activated. For example, the fault monitoring unit may start different cameras according to a preset time, start the fog-penetrating level camera from 7:00 a.m. to 17 pm, and start the starlight level camera at other time periods.

At step 706, images of a plurality of airfoils of the flap captured by the camera may be received. For example, the camera may monitor eight visual markers on four flaps of the left and right wings. The camera outputs frames of data at 60Hz, 30fps to the image processing component.

At step 708, image features in the image corresponding to the visual indicia may be extracted. For example, the image processing assembly may monitor the movement of the flap within, for example, 14 ± 2 seconds. The image processing component takes 150 milliseconds as a time region and extracts the image characteristics of the data frame.

At step 710, a current position fix for each airfoil may be determined based on the extracted image features. For example, the current position of the airfoil can be determined based on a visual indication of the maximum extension on the image of the airfoil. When the maximum extension section on the image of the airfoil surface has a complete visual mark, the current clamping position of the airfoil surface is the complete clamping position of the maximum extension section; or when the maximum extension section on the image of the airfoil has an incomplete visual mark, the current position of the airfoil is the complete position of the next maximum extension section plus the partial position of the maximum extension section. The partial entrapment can be determined based on a ratio of the incomplete visual indicia of the maximum extension compared to the complete visual indicia of the maximum extension.

At step 712, a determination can be made as to whether the flap is malfunctioning based on the determined current position of each airfoil. In one example, a single airfoil includes a first set of visual indicia on a plurality of stretches and a second set of visual indicia on a plurality of stretches, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance, wherein a tip failure of the airfoil is determined if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip threshold. In one example, an asymmetric fault is determined to occur with a first airfoil and a second airfoil if a first current detent identified based on a visual indicia of the first airfoil of the flap differs from a second current detent identified based on a visual indicia of the second airfoil of the flap by more than an asymmetry threshold.

In one example, an expected flap position corresponding to the operating instructions may also be determined based on the operating instructions generated by the control computer for controlling the flap motion, and a non-commanded malfunction of the flap may be determined if the current position of the one or more airfoils differs from the expected flap position by more than a threshold value.

If a flap fault occurs, the flap fault may be reported to the control computer at step 714. In other embodiments, the flap status may be reported to the control computer whether or not a flap failure has occurred.

If no fault is found, then steps 706-712 continue until an end condition. The end condition may be, for example, that the flap stops moving for a specified time (e.g., 2 seconds after the flap reaches the 18 degree position), that an instruction to stop monitoring is received, or the like. The camera and fault monitoring unit can be switched from "active" to "standby" state.

During the takeoff phase of the aircraft, 400 feet off the ground, the pilot retracts the slat, first moves the flap handle from 2 to 1 position, which can wake up the fault monitoring unit and camera for the first time in the air and monitor the flap retraction process from 18 degrees to 8 degrees, and these components can enter a "standby" state 2 seconds after the flap reaches the 8 degree position.

After the slat is fully retracted to the 0 degree position, the pilot can again move the handle from 1 detent to 0 detent, which can wake up the fault monitoring unit and camera a second time in the air and monitor the flap retraction from 8 degrees to 0 degrees, and these components can again enter the "standby" state 2 seconds after the flap reaches the 0 degree position.

In the approach stage of the airplane, 3000 feet away from the ground, the pilot releases the slat to the landing configuration, and the position of the handle is from 0- >1- >2- >3- > 4. The fault monitoring unit and the camera are thus awakened 4 times to monitor the flap movement process.

After the airplane finishes the taxiing stage after landing, the pilot can retract the slat to the cruising configuration, and the position of the handle can be from 4- >3- >2- >1- > 0. The fault monitoring unit and the camera are thus awakened 4 times to monitor the flap movement process.

The various steps and modules of the methods and apparatus described above may be implemented in hardware, software, or a combination thereof. If implemented in hardware, the various illustrative steps, modules, and circuits described in connection with the disclosure may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic component, hardware component, or any combination thereof. A general purpose processor may be a processor, microprocessor, controller, microcontroller, or state machine, among others. If implemented in software, the various illustrative steps, modules, etc. described in connection with the disclosure may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. A software module implementing various operations of the present disclosure may reside in a storage medium such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, a removable disk, a CD-ROM, cloud storage, and the like. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium, and execute the corresponding program modules to perform the various steps of the present disclosure. Furthermore, software-based embodiments may be uploaded, downloaded, or accessed remotely through suitable communication means. Such suitable communication means include, for example, the internet, the world wide web, an intranet, software applications, cable (including fiber optic cable), magnetic communication, electromagnetic communication (including RF, microwave, and infrared communication), electronic communication, or other such communication means.

The numerical values given in the embodiments are only examples and do not limit the scope of the present invention. In addition, other components or steps not recited in the claims or specification of the invention may be present as a whole. Moreover, the singular reference of a component does not exclude the plural reference of such components.

It is also noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged.

The disclosed methods, apparatus, and systems should not be limited in any way. Rather, the present disclosure encompasses all novel and non-obvious features and aspects of the various disclosed embodiments, both individually and in various combinations and sub-combinations with each other. The disclosed methods, apparatus, and systems are not limited to any specific aspect or feature or combination thereof, nor do any of the disclosed embodiments require that any one or more specific advantages be present or that a particular or all technical problem be solved.

The present invention is not limited to the above-mentioned embodiments, which are only illustrative and not restrictive, and those skilled in the art can make many modifications without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A system for monitoring flap failure, comprising:

a camera configured to capture images of a plurality of airfoils of the flap, each airfoil including a plurality of stretches corresponding to a plurality of flap screens, the plurality of stretches having different visual indicia thereon;

an image processing component configured to extract image features in the image corresponding to the visual indicia;

a position determining component configured to determine a current position of each airfoil based on the image features extracted by the image processing component; and

a fault status determination assembly configured to determine whether the flap is malfunctioning based on the determined current position of each airfoil.

2. The system for monitoring flap faults as claimed in claim 1, characterized in that:

the current position of the airfoil is determined based on the visual marking of the maximum extension on the image of the airfoil,

when the maximum extension section on the image of the airfoil surface has a complete visual mark, the current clamping position of the airfoil surface is the complete clamping position of the maximum extension section; or

When the maximum extension section on the image of the airfoil has an incomplete visual mark, the current position of the airfoil is the complete position of the next maximum extension section plus the partial position of the maximum extension section.

3. The system for monitoring flap faults as claimed in claim 2, characterized in that:

the partial capture is determined based on a ratio of the incomplete visual indicia of the maximum extension compared to the complete visual indicia of the maximum extension.

4. The system for monitoring flap faults as claimed in claim 2, characterized in that a single airfoil includes a first set of visual markers on the plurality of stretches and a second set of visual markers on the plurality of stretches, the first set of visual markers being spaced apart from the second set of visual markers by at least a specified distance,

wherein the fault condition determination component determines that a tip fault has occurred for the airfoil if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip threshold.

5. The system for monitoring flap faults as claimed in claim 2, characterized in that:

the fault status determination component determines that an asymmetric fault has occurred with a first airfoil and a second airfoil of the flap if a first current detent identified based on the visual indicia of the first airfoil differs from a second current detent identified based on the visual indicia of the second airfoil of the flap by more than an asymmetry threshold.

6. The system for monitoring flap faults as claimed in claim 2, characterized in that:

the fault status determination component determines an expected flap position corresponding to an operating command generated by a control computer for controlling the flap movement,

wherein the fault condition determination component determines that a non-commanded failure of the flap occurs if the current position of one or more airfoils differs from the expected flap position by more than a threshold.

7. The system for monitoring flap faults as claimed in claim 1, further comprising an activation assembly that activates the camera in response to one or more of:

the aircraft enters a takeoff or approach phase;

flap operating handle movement; or

Commands for flap fault monitoring are made.

8. An aircraft, characterized in that it comprises:

a flap coupled to a wing of the aircraft; and

the system for monitoring flap faults as claimed in one of claims 1 to 7.

9. A method for monitoring flap failure, comprising:

receiving images captured by a camera of a plurality of airfoils of a flap, each airfoil including a plurality of extension sections corresponding to a plurality of flap detents, the plurality of extension sections having different visual indicia thereon;

extracting image features in the image corresponding to the visual markers;

determining a current position of each airfoil based on the image features extracted by the image processing component; and

and judging whether the flap is in fault according to the determined current clamping position of each wing surface.

10. The system for monitoring flap faults as claimed in claim 9, characterized in that the current position of the airfoil is determined on the basis of a visual marking of the maximum extension on an image of the airfoil,

11. The system for monitoring flap faults as claimed in claim 10, characterized in that:

12. The system for monitoring flap faults as claimed in claim 10, wherein a single airfoil includes a first set of visual markers on the plurality of stretches and a second set of visual markers on the plurality of stretches, the first set of visual markers being spaced apart from the second set of visual markers by at least a specified distance,

wherein a tip-out failure of the airfoil is determined if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip-out threshold.

13. The system for monitoring flap faults as claimed in claim 10, characterized in that:

determining that an asymmetric fault has occurred with a first airfoil of the flap and a second airfoil if a first current detent identified based on the visual indicia of the first airfoil differs from a second current detent identified based on the visual indicia of the second airfoil of the flap by more than an asymmetry threshold.

14. The system for monitoring flap faults as claimed in claim 10, further comprising:

determining an expected flap position corresponding to an operating command generated by a control computer for controlling the flap movement; and

determining that a non-commanded failure of the flap occurs if the current position of one or more airfoils differs from the expected flap position by more than a threshold.

15. The system for monitoring flap faults as claimed in claim 9, further comprising activating the camera in response to one or more of:

the aircraft enters a takeoff or approach phase;

flap operating handle movement; or

Commands for flap fault monitoring are made.