CN114506460B

CN114506460B - System and method for monitoring flap faults

Info

Publication number: CN114506460B
Application number: CN202210253022.3A
Authority: CN
Inventors: 曹俊章; 王伟达; 张瑜琦; 徐清; 孙全艳; 韩昕悦
Original assignee: Comac Shanghai Aircraft Design & Research Institute; Commercial Aircraft Corp of China Ltd
Current assignee: Comac Shanghai Aircraft Design & Research Institute; Commercial Aircraft Corp of China Ltd
Priority date: 2022-03-15
Filing date: 2022-03-15
Publication date: 2024-05-10
Anticipated expiration: 2042-03-15
Also published as: CN114506460A

Abstract

A system and method for monitoring flap faults is disclosed. The system may include a camera configured to capture images of a plurality of airfoils of a flap, each airfoil including a plurality of extension segments corresponding to a plurality of flap detents, the plurality of extension segments having different visual indicia disposed thereon; an image processing component configured to extract image features in the image corresponding to the visual indicia; a clamping position determining component configured to determine a current clamping position of each airfoil based on the image features extracted by the image processing component; and a fault condition determination assembly configured to determine whether the flap is faulty based on the determined current position of each airfoil.

Description

System and method for monitoring flap faults

Technical Field

The present invention relates to the field of aircraft, and more particularly to a system and method for monitoring flap faults.

Background

The high lift systems of modern large aircraft include slats located at the leading edge of the wing and flaps located at the trailing edge of the wing. During low-speed stages of taking off, landing and the like of the aircraft, the wing area is increased, the configuration is changed and the lift force of the aircraft is provided by outwards extending and downwards bending the front edge slat and the rear edge flap, so that the reasonable running distance and the safe taking-off speed of the aircraft are ensured, and meanwhile, the climbing rate, the approach speed and the approach posture of the aircraft are improved.

There are three serious failure modes of high lift systems:

a) Actuator disengagement/airfoil tip: a single actuator of the airfoil or a hinge connected to the body is blocked and tilted by an external force, or a block (Jamming) or Freewheel Rotation (FREEWHEELING) occurs in the actuator itself, while another actuator of the airfoil is still driving the airfoil to continue moving.

B) Airfoil asymmetry: the monolithic airfoil moves out of synchronization with other airfoils, which may be a secondary failure of airfoil lean.

C) The airfoil is not commanded, i.e., the actual arrival position of the airfoil is inconsistent with the flapping handle command.

If one or more types of failure of the aircraft occur in the take-off or landing stage, serious damage and even crash of the aircraft body structure can be caused seriously. The monitoring of such failures has therefore become an integral part of the design of modern aircraft high lift systems. Conventional tilt sensing approaches mount tilt sensors on the structure adjacent the airfoil for sensing actuator disengagement/airfoil tilt. However, installing the sensor requires more sensors and associated cabling, increases space usage and aircraft weight, and may be subject to installation errors.

Accordingly, there is a need in the art for an improved system and method for monitoring flap faults.

Disclosure of Invention

The present invention provides an improved system and method for monitoring flap faults. More specifically, the present invention provides a system and method for monitoring airfoil faults using image recognition techniques. One or more position markers may be provided on the airfoil extension corresponding to each flap detent, for example, each airfoil may be provided with a unique visual marker corresponding to each extension detent. Visual indicia on each airfoil surface may be captured by a camera as the flap is moved. The system may identify the current position of the airfoil based on visual indicia captured by the camera. Further, when the maximum extension displays only a portion of the visual indicia, the system may calculate a proportional relationship of the captured visual indicia to the full indicia of the corresponding detents, which may be indicative of the (portion of) the current detent of the airfoil. Comparing the plurality of current positions of the identified one or more airfoils or the expected flap positions can determine whether the airfoils have faults such as inclination, asymmetry, non-instruction and the like.

In one embodiment of the invention, there is provided a system for monitoring flap faults, comprising: a camera configured to capture images of a plurality of airfoils of a flap, each airfoil including a plurality of stretches corresponding to a plurality of flap detents, the plurality of stretches having different visual indicia thereon; an image processing component configured to extract image features in the image corresponding to the visual indicia; a clamping position determining component configured to determine a current clamping position of each airfoil based on the image features extracted by the image processing component; and a fault condition determination assembly configured to determine whether the flap is faulty based on the determined current position of each airfoil.

In one aspect, the current position of the airfoil is determined based on a visual marker of a maximum stretch on the image of the airfoil, wherein the current position of the airfoil is a full position of the maximum stretch when the maximum stretch on the image of the airfoil has a full visual marker; or wherein when the maximum extension has an incomplete visual marker on the image of the airfoil, the current position of the airfoil is the complete position of the next maximum extension plus a partial position of the maximum extension.

In one aspect, the partial detent is determined based on a ratio of the incomplete visual indicia of the maximum stretch segment to the complete visual indicia of the maximum stretch segment.

In one aspect, a single airfoil includes a first set of visual indicia on the plurality of extension segments and a second set of visual indicia on the plurality of extension segments, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance, wherein the fault status determination component determines that the airfoil is subject to a tip fault if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip threshold.

In one aspect, the fault condition determination component determines that an asymmetric fault has occurred with a first airfoil of the flap if a first current position difference identified based on a visual marker of the first airfoil of the flap and a second current position difference identified based on a visual marker of the second airfoil of the flap exceeds an asymmetric threshold.

In one aspect, the fault condition determination component determines an expected flap position corresponding to an operating instruction generated by a control computer for controlling flap movement based on the operating instruction, wherein the fault condition determination component determines that a non-commanded fault has occurred with the flap if a current position of one or more airfoils differs from the expected flap position by more than a threshold value.

In one aspect, the system for monitoring flap faults further comprises an actuation component that actuates the camera in response to one or more of: the aircraft enters a take-off or approach stage; flap lever movement; or a command to perform flap fault monitoring.

In one embodiment of the invention, there is provided an aircraft comprising: a flap coupled to a wing of the aircraft; and a system for monitoring flap faults as claimed in any preceding claim.

In one embodiment of the invention, a method for monitoring flap faults is provided, comprising: receiving images of a plurality of airfoils of a flap captured by a camera, each airfoil including a plurality of extension segments corresponding to a plurality of flap detents, the plurality of extension segments having different visual indicia thereon; extracting image features corresponding to the visual marks in the image; determining a current clip position for each airfoil based on the image features extracted by the image processing component; and judging whether the flap fails according to the determined current clamping position of each airfoil surface.

In one aspect, a single airfoil includes a first set of visual indicia on the plurality of stretch sections and a second set of visual indicia on the plurality of stretch sections, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance, wherein a tip failure of the airfoil is determined if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip threshold.

In one aspect, an asymmetric fault is determined for a first airfoil of the flap if a first current position difference identified based on a visual marker of the first airfoil and a second current position difference identified based on a visual marker of the second airfoil of the flap exceeds an asymmetric threshold.

In one aspect, the system for monitoring flap faults further comprises: determining an expected flap position corresponding to an operation instruction generated by a control computer for controlling flap movement; and if the current clamping position of one or more airfoils and the expected flap clamping position difference exceed a threshold value, determining that the flap fails in a non-instruction mode.

In one aspect, the system for monitoring flap faults further comprises activating the camera in response to one or more of: the aircraft enters a take-off or approach stage; flap lever movement; or a command to perform flap fault monitoring.

Drawings

FIG. 1 is a schematic illustration of flap fault monitoring according to one embodiment of the invention.

FIG. 2 is a schematic illustration of visual indicia on a flap according to one embodiment of the invention.

FIG. 3 is a schematic illustration of a flap portion extended according to one embodiment of the invention.

FIG. 4 is a schematic view of the tilting during extension of a flap according to one embodiment of the invention.

FIG. 5 is a schematic illustration of an asymmetry occurring during extension of a flap according to an embodiment of the invention.

FIG. 6 is a block diagram of a flap fault monitoring system according to one embodiment of the invention.

FIG. 7 is a flow chart of a method of flap fault monitoring according to one embodiment of the invention.

Detailed Description

The invention will be further described with reference to specific examples and figures, which should not be construed as limiting the scope of the invention.

FIG. 1 is a schematic illustration of flap fault monitoring according to one embodiment of the invention. According to the invention, the image of the flap can be captured through the camera, the flap can comprise one or more wing surfaces, each wing surface can comprise different visual marks corresponding to a plurality of stretching clamping positions, and the current clamping position of the wing surface can be determined through an image recognition technology based on the visual marks shot by the camera.

Image recognition technology is a computer vision mechanism that utilizes a computer to process, analyze, and understand images to identify targets and objects in various different modes. With the great improvement of the computing power of a Graphics Processor (GPU) and a Digital Signal Processor (DSP), the continuous proposal of new computing methods, the large-scale growth of available data resources, the continuous emergence of new application modes, and the image recognition and application technology thereof show new trends in the aspects of research breadth and depth, the performance of recognition effect and the expansion of technology and application, thereby making the image recognition and application technology possible as a fault detection means in a high lift system. The present invention may utilize existing image recognition algorithms or suitable algorithms developed in the future to identify visual indicia on the airfoil.

A typical sequence of maneuvers for a high lift system is as follows: the pilot moves the Flap/slat handles (Flap/Slat Control Lever, FSCL) to the commanded position and stops. The flap computer (Slat/Flap Control Computer, SFCC) detects the effective handle command signal, and then sends the operation command signal to the power drive unit (Power Driver Unit, PDU) after internal processing analysis. The PDU outputs rotational torque that is transmitted to a rotary gear actuator via a torque tube, bearing mount, or other drive line component to drive movement of a control surface (e.g., flap airfoil, slat airfoil). Position sensors (Position Sensor Unit, PSU) located at the wingtips feed back the control surface position signals to the SFCC. When the SFCC detects that the control surface reaches the sensor signal of the command position, a command signal is sent to enable the PDU to stop outputting torque, and the command signal is sent to a brake device on the PDU to lock the transmission line system so as to enable the control surface to be kept at the current position. When SFCC detects the faults of asymmetrical wing surface, underspeed wing surface, etc., it can send out command signal to wing tip brake device to lock the drive line system to maintain the control surface in the current position.

According to one embodiment of the invention, a camera may be mounted on the vertical tail body structure to capture flap images, the camera having a viewing angle that is such that the distance between the outer edges of the outer flaps on the left and right sides is reached. Based on the visual indicia on the airfoil, the extended position (e.g., current position) of the flap may be determined and a determination may be made as to whether the airfoil is experiencing an uncommanded, skewed, asymmetric, or the like failure, as described in more detail below.

FIG. 2 is a schematic illustration of visual indicia on an airfoil of a flap according to an embodiment of the invention. Each airfoil may include a plurality of extension segments 231-234 corresponding to a plurality of flap detents. When the flap is not extended, a portion 230 of the flap is presented, but each extension 231-234 is located below the wing and is not presented. As the pilot moves the flap/slat handles, a control computer (e.g., SFCC) determines operating commands and controls flap motion based on the handle manipulation instructions. Different operating commands may cause the flap to extend different lengths, i.e. detents corresponding to the operating commands.

For example, if the operating command is a reach 1 detent, the flap is extended a corresponding distance from under the wing back so that the extension 231 is presented; if the operational command is a 2-out detent, the flap is extended a corresponding distance from under the wing rearward such that the extended section 232 is presented; and so on. That is, the extension segments 231-234 are divided along the direction of movement of the flap such that extension segment 231 corresponds to a 1-stop, extension segment 232 corresponds to a 2-stop, extension segment 233 corresponds to a 3-stop, extension segment 234 corresponds to a 4-stop, and so on.

Thus, visual indicia may be added to each airfoil (e.g., upper surface) to distinguish between multiple extended sections of the flap. The visual indicia corresponding to the individual stretch may be different from one another, whereby the current position of the flap may be determined based on the detected visual indicia. While FIG. 2 shows the division between each of the extension segments 231-234 in dashed lines, it should be appreciated that the dashed lines are not required as visual indicia on the airfoil.

The division between the extension segments 231-234 is related to the flap configuration (e.g., flap size, distance of movement corresponding to each detent, etc.), and may be determined based on the flap configuration.

By way of example and not limitation, in one high lift system, a flap configuration is shown in the following table.

Handle clamping position	Slat position (degree)	Flap position (degree)	Configuration of
				0	0	0	Cruising device
1	20	8	Heavy take-off
				2	20	18	Normal weight take-off
3	25	30	Flying around
				FULL	25	40	Landing

The flap motion mechanism may be of the hinge type, i.e. the flap moves in an arc about a hinge point on the pivot axis by means of its associated rocker arm. The motion sequence of the flap is: the slat extends out first and the flap extends out later; the flap is retracted first and the slat is retracted later. The movement time of the flap is as follows:

Because the flap motion mechanism is hinged, the width (e.g., the dimension in the direction of flap motion) of each extension can be relatively easily calculated.

Let PI (PI) =3.14159, l be the distance of the flap airfoil to the hinge point. The theoretical width d1=2×pi×l× (8/360) of the extension 231 corresponding to the flap clip 1, the theoretical width d2=2×pi×l× (10/360) of the extension 232 corresponding to the flap clip 2, the theoretical width d3=2×pi×l× (12/360) of the extension 233 corresponding to the flap clip 3, and the theoretical width d4=2×pi×l× (10/360) of the extension 234 corresponding to the flap clip 4 (FULL).

Although examples of determining the width of each stretch are given above, it should be appreciated that the stretches 231-234 corresponding to each detent may be determined in any suitable manner.

A unique visual marker may be added to each stretch to distinguish the different stretches 231-234. By way of example and not limitation, the visual indicia may select graphics, letters, numbers, or a combination thereof. The pattern may be a regular shape, such as a triangle, a quadrilateral, a circle, or other polygons, or an irregular shape, such as a cloud or a complex combination pattern. In one example embodiment, a combination of shape and number is selected as a visual marker to describe the position of the flap catch. In a further embodiment, a circle is selected and a number is embedded as the visual marker.

Preferably, the visual indicia may be sized to correspond to the respective extension (i.e., the exposed area of the flap airfoil for each flap configuration). For example, the visual indicia may occupy the entire width of each stretch or a specified percentage or more thereof (e.g., 90% or more, 95% or more, etc. of the width of the stretch). By way of example and not limitation, the diameter of the selected circle may be adapted to the extension length of each flap configuration. In some implementations, the visual indicia may be slightly smaller in size than the entire width of the respective stretch to allow for no connection or no overlap between visual indicia of adjacent stretch.

In one embodiment according to the invention, the visual indicia may have a different color than the airfoil so that the visual indicia on the airfoil can be clearly identified. For example, the visual indicia may have a different color than the airfoil, such as red, violet, black, green, blue, fluorescent, and the like. The visual indicia may be formed with paint, tape, or other materials (e.g., metal, plastic, etc.). The visual indicia may be formed on the airfoil by coating, pasting, plating, or the like. Preferably, the visual indicia may be waterproof, abrasion resistant, and the like.

Fig. 2 shows a schematic diagram of a circle with embedded numbers as visual indicia. The numbers within the visual indicia may have any orientation, such as perpendicular to the axis of rotation of the airfoil. For example, in flap position 1 (circle+1); flap position 2 is indicated by (circle+2); flap stop 3 is indicated by (circle +3); the flap catch 4 is indicated by (circle +4).

If the flap extends to 1 position, the visual mark of the position 1 is displayed on the wing surface of the flap, and the visual marks of the positions 2-4 are blocked. If the flap extends to the 2-position, the visual marks of the 2-position and the 1-position are displayed on the wing surface of the flap, and the visual marks of the 3-4-position are blocked. If the flap extends to 3, the visual indicia of position 3, 2 and 1 are displayed on the flap airfoil, and the visual indicia of position 4 is obscured. If the flap extends to the 4-position, visual indicia of position 4, 3, 2, and 1 are displayed on the flap airfoil. With this push, if the flap extends to N-stop (e.g., N.gtoreq.4), visual indicia of stop N, stop N-1..and stop 1 are displayed on the flap airfoil.

In one embodiment, a single airfoil may include a set of visual indicia on multiple stretches. In another embodiment, a single airfoil may include a first set of visual indicia (e.g., the left visual indicia in FIG. 2) on a plurality of extension segments and a second set of visual indicia (e.g., the right visual indicia in FIG. 2) on the plurality of extension segments, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance. In addition, each set of visual indicia may or may not be aligned in the flap extension direction.

FIG. 3 is a schematic illustration of a visual indicia display with a flap extended according to one embodiment of the invention.

When the flap operating command indicates a certain detent, the flap may be extended outwardly, exposing visual indicia on the extension segments 231-234. By identifying visual indicia on the flap, the current position of the flap can be determined. Since the extension sections 231, 232, 233, 234 are gradually exposed as the detent increases, it can be considered that the extension section 234 is larger than the extension section 233, the extension section 233 is larger than the extension section 232, and the extension section 232 is larger than the extension section 231.

The current position of the airfoil is determined based on visual indicia of the largest stretch (i.e., the stretch nearest the wing) on the airfoil image, and the current position may be determined as either a full position or a partial position. When the maximum extension segment on the airfoil image has a full visual marker, the current capture of the airfoil is the full capture of the maximum extension segment. When the maximum stretch has an incomplete visual mark on the image of the airfoil, the current position of the airfoil is the complete position of the next largest stretch (i.e., the stretch next adjacent to the airfoil) plus the partial position of the largest stretch.

For example, when the flap operation command indicates position 1, the flap may extend outward and expose the extension 231. If a full signature of the maximum extension 231 is detected, the current position of the airfoil may be considered a full position of the maximum extension 231.

In another example, when the flap operation command indicates position 1, the flap may be extended outward to the position shown in FIG. 3, which shows a partial visual indicia of the extension segment 232 (maximum extension segment) and a full visual indicia of the extension segment 231 (next maximum extension segment). Thus, the current position of the airfoil may be determined to be 1.X, where 1 indicates the position of the next largest extension 231 and X indicates the partial position of the extension 232. In one embodiment, the value of X may be determined based on the ratio of the incomplete visual indicia of the stretch segment 232 to the corresponding complete visual indicia of the stretch segment 232. For example, the ratio may be the height of the visual indicia displayed (e.g., the height in the direction of flap movement) divided by the height of the full visual indicia. From this, it may be determined that the value of X is, for example, 0.3, 0.5, 0.8, etc., and accordingly it may be determined that the current clamping position of the airfoil may be determined to be 1.3, 1.5, 1.8, etc. According to one embodiment of the invention, it may be determined whether the current position of the airfoil reaches an expected flap position corresponding to the operating command. If the current and expected flap position differences of one or more airfoils exceed a threshold, a determination may be made that the flap is experiencing an uncommanded failure.

In another embodiment, it may not be necessary to determine the ratio of the partial visual indicia of the stretch section 232 to the full visual indicia. For example, in the event that a gap exists between visual indicia of adjacent extension segments 231, 232 to account for flap tolerance errors, once an incomplete visual indicia of extension segment 232 is detected, the airfoil may be deemed to have not reached an expected flap jam or to have an uncommanded failure.

FIG. 4 is a schematic view of the tilting during extension of a flap according to one embodiment of the invention. The single airfoil may include a first set of visual indicia (shown on the left) and a second set of visual indicia (shown on the left) on the plurality of extension segments, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance. The exposed portions of the two sets of visual indicia are not identical when the airfoil is tilted.

Thus, a first current position of the airfoil (e.g., 1.8) may be identified based on a first set of visual indicia of the airfoil, and a second current position of the airfoil (e.g., 1.3) may be identified based on a second set of visual indicia of the airfoil. If the difference between the first current position and the second current position exceeds the inclination threshold, it may be determined that the airfoil is experiencing an inclination fault.

FIG. 5 is a schematic illustration of an asymmetry occurring during extension of a flap according to an embodiment of the invention. When the extent of extension of the two airfoils is different, an asymmetric fault may occur.

For example, a first current position of the left inner flap may be identified based on the visual indicia of the left inner flap (e.g., 1), and a second current position of the right inner flap may be identified based on the visual indicia of the right inner flap (e.g., 1.7). If the difference between the first current clamping position and the second current clamping position exceeds an asymmetry threshold value, the two airfoils can be determined to have an asymmetry fault.

FIG. 6 is a block diagram of a flap fault monitoring system according to one embodiment of the invention. The flap fault monitoring system may include one or more cameras and a fault monitoring unit (Fault Monitoring Unit, FMU) 610. The fault monitoring unit 610 may be in communication with one or more control computers 601, 602.

The control computers 601, 602 may be flap control computers (SFCCs) or may implement flap control functions or the functions of the flap control computers described above. Furthermore, the control computers 601, 602 monitor flap faults by means of a fault monitoring unit 610. By way of example, and not limitation, a data bus or hard wire connection may be used between the computers 601, 602 and the fault-monitoring unit 610, or both. The data buses may be ARINC429, CAN, RS232/485, etc. The SFCC performs necessary communication with the FMU through a data bus, such as SFCC 'wake-up' instruction to the FMU, and FMU 'flap fault' report to the SFCC.

The FMU may also be connected to two SFCCs via a plurality of hard wires, such as two hard wires for transmitting a "flap tip-over failure" signal, two hard wires for transmitting a "flap asymmetry failure" signal, two hard wires for transmitting an "airfoil non-commanded failure" signal, etc. Additionally or alternatively, these fault signals may be transmitted over a data bus.

One or more cameras may be used to capture images of the flap. The camera may be mounted on the fuselage in a position that captures an image of the flap. In one embodiment, two digital cameras may be used, both of which are mounted on a three-axis pan-tilt, which is mounted on a vertical tail structure of the aircraft. The purpose of using the cradle head is to eliminate the influence of the vibration of the aircraft body on the shooting quality.

The two cameras may have the following basic features:

a) Frame rate, preferably supporting full-speed transmission of 50/60 fps;

b) The image size should be guaranteed to be 800 x 600 and above;

c) The security level of protection should be guaranteed at IP60 and above.

The main features of the two cameras may differ as follows:

a) Imaging capabilities are different:

one can be a fog-penetrating camera, has basic optical fog penetrating characteristics and backlight shooting capability, and is mainly used in daytime with good illumination conditions.

An aircraft typically performs a approach procedure at 3000 feet (1000 meters), i.e., begins to deploy flaps and landing gear. At this height, there is often a cloud layer, and suspended particles inside will have strong absorption and scattering effects on light. In this case, the image acquired by the imaging device is blurred, whitened, low in contrast, and the details of the image are seriously lost. It is therefore necessary to select a fog-penetrating camera.

The other can be a starlight level camera, which is mainly used for night conditions with poor illumination conditions. The two cameras can be switched by themselves according to the lowest illumination.

In one embodiment, the basic index of the fog-penetrating level camera is as follows: the 1/1.8' progressive scanning CMOS device is adopted, the 3D digital noise reduction is realized, the 120dB wide dynamic range is realized, and the optics and the algorithm are adopted to penetrate fog. The basic index of the starlight level camera is as follows: and a 4/3' progressive scanning CMOS device and a 20-time optical zoom lens are adopted, when the illumination condition is 0.012LUX, the color picture is converted into a black-and-white picture, and the infrared compensation is 50 meters.

B) The photosensitive devices are different: one may use a CCD (charge coupled device) photosensitive device and the other may use a COMS (complementary oxygen metal semiconductor) photosensitive device; or two cameras adopt CCD photosensitive devices with different sizes; or two cameras adopt CMOS photosensitive devices with different sizes;

c) Imaging colors are different: one may be a black and white camera and one may be a color camera;

d) Pixels are different: the pixels of one camera can be 200 ten thousand or less, and the pixels of one camera can be 200 ten thousand or more.

Although cameras having different imaging capabilities are listed above, in particular practice one camera or more of the same cameras may be employed.

The fault monitoring unit 610 may include an interface component 612 (e.g., an input/output I/O module), an image processing component 614, a detent determination component 616, a fault status determination component 618, a start-up component 611, and the like. The fault monitoring unit 610 may also include a power module, memory, etc. For example, the power module may be connected to a 28V dc bus on board the aircraft to convert 28V 600hz dc to dc of different magnitudes to power the modules and chips inside the FMU, as well as the camera. The fault monitoring unit 610 may be implemented using a computer, a processor, a server, a controller, or the like. In other embodiments, the fault monitoring unit 610 may be integrated into one or more control computers.

The interface component 612 can be responsible for receiving and transmitting data, such as implemented by an FPGA. In one embodiment, the interface component 612 may be configured to:

a) Connected to the camera 1,2, receives image frames transmitted by the camera with a capture period of 20 milliseconds or less;

b) Connected to the control computers 601, 602, the processing results of the fault monitoring unit 610 (e.g. in the form of bus data and discrete signals, respectively) are sent to each control computer;

c) And ensuring the integrity of the data.

1) The interface FPGA can store 5V, 3.3V, 7.5V, 2.5V and ground reference voltages, which are periodically read and verified by the control chip.

2) The interface FPGA periodically generates a pseudo-random number sequence and sends the pseudo-random number sequence to the control chip, and the control chip returns the pseudo-random number sequence to the interface FPGA. If the returned sequence is the same as the original sequence, the interface FPGA will reset the watchdog FPGA of the chip. Otherwise, the watchdog is triggered, causing the fault-monitoring unit 610 to enter a fail-safe state.

The image processing assembly 614 may process the flap image captured by the camera to extract visual indicia on the flap airfoil. Because of the effects of various factors such as body shake, natural light, weather conditions, etc., a certain degree of interference and noise are inevitably introduced into the acquired image, and the image processing component 614 optionally eliminates these adverse factors in preprocessing, and equalizes the light of the image through links such as image equalization and image enhancement. In the image enhancement link, algorithms such as image defogging, low illumination enhancement and the like can be included.

After the feature color extraction and image denoising of the image in the link, the image processing component 614 may use a pre-configured algorithm to extract the visual mark on the flap airfoil. For example, depending on the shape and/or character of the visual indicia employed, the image processing component 614 may be preconfigured with an appropriate algorithm to determine whether there are corresponding one or more visual indicia (full or partial visual indicia) in the captured flap image. The image processing assembly 614 may provide one or more visual indicia of the identification of each airfoil of the flap. The image processing component 614 may also provide relative positional relationships of the visual indicia to facilitate determination of the visual indicia of the maximum stretch.

By way of example and not limitation, where visual features of circular embedded numbers are employed, image processing component 614 may employ Canny edge detection operators to extract edge information and then find circular areas using Hough transforms to complete identification of image features. In one embodiment, the feature colors may use RGB, HSV, YUV or other color spaces. The HSV color model is insensitive to illumination factors, the illumination influence can be reduced to the greatest extent, and the physical meaning of the model accords with the visual characteristics of people, so that the HSV color model can be optimized.

If there are two cameras capturing redundant flap images, the image processing assembly 614 should be able to obtain nearly identical image features (visual indicia) for the images captured by the first and second cameras.

The position determination component 616 can determine the current position of each airfoil based upon the visual indicia of that airfoil identified by the image processing component 614.

In one embodiment, a circle may be used to represent a first detent, a rectangle a second detent, a hexagon a third detent, a five-star a fourth detent, and so on. If the image processing component 614 identifies the presence of a complete circle and a complete rectangle on an airfoil, the stuck position determination component 616 may determine that the airfoil is in the second stuck position. If the image processing component 614 identifies the presence of a circle, rectangle, and a portion of hexagon on an airfoil, the capture determination component 616 may determine that the airfoil is in a portion of a third capture. Further, the detent determination component 616 can also determine a particular scale value for the portion of the third detent based on a display scale of the portion of the hexagon (e.g., a ratio of a height thereof to a height of the complete hexagon). For example, if the visual indicia of the third clip exhibits 30% or 60%, the clip determining component 616 may determine that the airfoil is at the 2.3 or 2.3 clip.

In one embodiment, where circles plus numbers are used to represent corresponding stretches, the stall determination component 616 may determine the stall at which the airfoil is located based on the numbers in the identified circles.

The fault condition determining assembly 618 may be configured to determine whether the flap is faulty based on the determined current position of each airfoil.

In one embodiment, a single airfoil includes a first set of visual indicia on a plurality of extension segments and a second set of visual indicia on the plurality of extension segments, wherein the first set of visual indicia is spaced apart from the second set of visual indicia by at least a specified distance. If the difference between the first current position of the airfoil identified based on the first set of visual indicia and the second current position of the airfoil identified based on the second set of visual indicia exceeds a tip-out threshold, the fault condition determination component 618 may determine that the airfoil is subject to a tip-out fault.

In one embodiment, the fault condition determining component 618 may determine that an asymmetric fault occurred with the first airfoil and the second airfoil if a first current stuck position identified based on the visual indicia of the first airfoil and a second current stuck position identified based on the visual indicia of the second airfoil differ by more than an asymmetric threshold.

In one embodiment, the fault condition determination component 618 may determine an expected flap position corresponding to an operating instruction generated by the control computer for controlling flap movement. If the current position of one or more airfoils differs from the expected flap position by more than a threshold value, the fault status determination component 618 can determine that the flap is experiencing a non-commanded fault. For example, the control computers 601 and/or 602 may communicate the operating instructions or their corresponding expected flap positions to the fault monitoring unit 610 (e.g., the interface component 612), whereby the fault status determination component 618 may be aware of the expected flap positions of the flaps. The fault condition determination component 618 can determine whether a flap is faulty by comparing the identified current position of the flap to an expected flap position.

If the fault status determination component 618 determines that a flap is faulty, flap fault information can be reported to the control computer 601 and/or the control computer 602. In another embodiment, the fault status determination component 618 can report the results of the fault monitoring of the flap (e.g., normal, fault, status information, etc.) to the control computer.

In one embodiment of the invention, the camera and/or fault-monitoring unit 610 (e.g., components 614-618) may be in a dormant or standby state when flap fault monitoring is not required, such as when the wing remains retracted during a cruise process. In a sleep or standby state, the initiation component 611 may cooperate with the interface component 612 to detect whether a trigger event has occurred and wake up one or more cameras and/or other components of the fault monitoring unit 610 upon occurrence of the trigger event. For example, the activation component 611 may activate the camera (and optionally also components 614-618) in response to one or more of the following: the aircraft enters a take-off or approach stage; flap lever movement; or a command to perform flap fault monitoring is received.

In one embodiment, the activation component 611 may activate the appropriate camera based on ambient light conditions. For example, a fog-level camera is activated when light conditions are good, and a starlight-level camera is activated when light conditions are poor. In other embodiments, the activation component 611 may also activate or switch the camera based on ambient light changes, time changes, whether the taken picture meets the requirements, and so forth.

The camera and fault monitoring unit 610 may continuously monitor wing faults after start-up. For example, the camera may take wing pictures (frames) at a specified sampling rate, and the fault monitoring unit 610 may process each frame to determine whether a flap fault has occurred. After the fault monitoring unit 610 determines that the flap stops moving for a certain time threshold, or after the flap operating instructions occur for a certain time threshold, the actuation assembly 611 may cause the cameras and assemblies 614-618 to enter a sleep or standby state again.

In one embodiment of the invention, the detent determination component 616 and the fault condition determination component 618 can be implemented in combination, such as with a CPU. The CPU may be divided into two channels, a control channel and a monitor channel. Each channel may be implemented by a core in a dual or multi-core processor (Multi Core Processor), or by a separate chip.

The control channel is used as a calculation link to process a positive code form comprising image characteristic data; the monitoring channel is used as a checking link to process the complementary code form of the image characteristic data. And the two channels respectively determine the current clamping position of the flap according to the received image characteristic data and judge whether the flap fails or not.

The two channels send the results of the determination to the interface component 612. The interface component 612 performs a comparison after performing a unification process on the coding formats of the two sets of output results. If the results of the comparison are consistent, the interface component 612 sends the results to the two control computers as system output results.

If none of the consecutive multiple data frames (e.g., 3 data frames) between two lanes is successful, the monitoring lane may trigger fail-safe logic to first notify both control computers of the status of the lane (e.g., via ARINC429 bus) and then lock the failure monitoring unit 610. For example, the ARINC429 bus data SSM bit sent out is all set to "FAILURE WARNING". The control computer can immediately stop the flap from moving and lock the flap in the current position after receiving the signal.

In one embodiment of the invention, after the aircraft is powered on, the camera and the FMU enter a standby state after the powering on self-test is completed, and wait for an SFCC wake-up instruction. Once one of the SFCCs monitors that the flap lever is clear of the last detent, the FMU may be awakened. The FMU may then activate different cameras depending on the minimum light level or time. If the fog level camera fails, the FMU activates the starlight level camera.

The image processing component should change from the "standby" state to the "active" state within a certain time threshold Ttrans. This time threshold Ttrans should be less than the time from acquisition of the flap handle signal by the SFCC to determination of the handle operation command. The SFCC can send the information of the handle clamping position to the FMU, and the FMU can correspondingly adjust the threshold value for judging the fault.

After receiving the image transmitted by the camera, the FMU analyzes the image feature changes of all frames within a certain time threshold Tth one by one, and informs the SFCC of the feature change results (and/or faults). The determination of this time threshold Tth may take into account the following factors:

a) The time required for processing the image, determining the clamping position and judging the fault is recorded as Ta;

b) The SFCC records Tb from the time of receiving fault information to the time of generating a locking instruction for a power drive device (PDU) braking device;

c) The time from responding to SFCC command to completing locking action is marked as Tc;

d) A certain safety margin;

e) The safety factor is generally 1 to 3.

The time threshold tth= (ta+tb+tc) -safety margin.

If the FMU determines that a flap failure (e.g., pitch, asymmetry, non-commanded, etc.) occurs, the SFCC will immediately lock the brake on the power drive, and the airfoil will be locked in the current position and no longer move. If the high lift system is equipped with a wingtip brake, the SFCC will lock both the brake of the power drive and the wingtip brake.

The determination of the airfoil inclination threshold Pth1 or the airfoil asymmetry threshold Pth2 may take into account the following factors:

a) The angle at which the flap moves over time (ta+tb+tc);

b) The flap allows for a maximum angle of inclination or an asymmetric angle;

c) Tolerances of mechanical parts, including play of torsion tubes, bearing supports, ball screw actuators, etc., and stiffness deformation causing flap deflection angles;

d) A certain safety margin;

e) The safety factor is generally 1 to 3.

Thus, the airfoil inclination threshold may be:

pth1=maximum angle of inclination allowed for the flap-angle of movement of the flap within (ta+tb+tc) -safety margin.

The airfoil asymmetry threshold may be:

Pth2=the maximum asymmetric angle allowed for the flap-the angle at which the flap moves within (ta+tb+tc) -the safety factor.

If the fault condition determination component 618 determines that the difference between the first current clamp position and the second current clamp position of the individual airfoil exceeds the tip-over threshold Pth1, it may be determined that an airfoil tip-over has occurred. If the fault condition determining component 618 determines that the current clamp position of one airfoil differs from the current clamp position of the other airfoil by more than the asymmetry threshold value Pth2, an asymmetric fault may be determined for the first airfoil and the second airfoil.

The FMU may process image frames of each airfoil sequentially or concurrently, determine whether each airfoil has a tip-over or non-commanded failure, and determine whether an asymmetrical failure has occurred between the plurality of airfoils.

In one embodiment, for both sets of visual indicia of a monolithic airfoil, the ratio values R _i1 and R _i2 of the height of the two visual indicia of the maximum extension to the height of the intended full visual indicia may be determined to determine if the two visual indicia of the piece of flap differ by more than the pitch threshold Th1. The tilt threshold Th1 may be a threshold obtained by appropriate transformation of Pth1, or may be an appropriate threshold determined in other ways as desired.

In another embodiment, the differential angle of the two-sided visual indicia may also be determined:

P= (flap configuration _i angle of extension-flap configuration _(i-1) angle of extension) ×|r _i1–R_i2 |

The actual inclination angle P _actul = P x K of the airfoil.

Note that: k is a scaling factor and the differential angle P of the two visual markers may be different from the actual angle of inclination Pactul of the airfoil. The two angles are linear. The magnitude of K is related to the distance between the center of the visual mark and the left and right edges of the airfoil, and can be finely adjusted according to actual conditions. If P _actul≥P_th1, it can be determined that the current flap airfoil is tilted.

In one embodiment, for any airfoil, such as the left outer airfoil, the left inner airfoil, the right inner airfoil, and the right outer airfoil, the ratio value R of the visual marker height to the expected full visual marker height may be determined and the difference R difference = ABS (R ₁-R₂) for any two airfoils calculated to determine if the difference is greater than the asymmetry threshold Th2. The tilt threshold Th2 may be a threshold obtained by appropriate transformation of Pth2, or may be an appropriate threshold determined in other ways as desired.

In another embodiment, the angle of difference of the visual indicia of the two airfoils may also be determined:

P difference = (flap configuration _i extension angle-flap configuration _(i-1) extension angle) x R difference;

if the P difference is greater than or equal to Pth2, the corresponding two airfoils are determined to be asymmetric.

In addition, the FMU may transition from an "on" state to a "standby" state after the visual indicia has not changed for a certain time threshold, and may also transition the camera to a dormant or standby state. This time threshold should be slightly greater than the tolerance of the flap movement time, for example 1 second, 2 seconds.

It should be noted that the manner and values of calculation of the various thresholds are given above by way of example only and not limitation. In specific practice, the corresponding threshold values may be set as desired without departing from the scope of the application.

FIG. 7 is a flow chart of a method 700 of flap fault monitoring in accordance with one embodiment of the invention. The method 700 may be performed by the fault monitoring unit described above or may be performed by a control computer, processor, computer, or the like.

After the aircraft is electrified, the camera and the flap fault monitoring unit are electrified and self-inspected firstly, and are in a standby state after finishing, and wait for the instruction of the SFCC.

At step 702, a wake-up signal may be detected. For example, an interface component in the fault monitoring unit may remain operational and receive a wake-up signal. The wake-up signal may be, for example, a command for the aircraft to enter a take-off or approach phase, flap lever movement, or flap fault monitoring, etc. For example, after a double start of the aircraft, the pilot may set the flap as required by standard operating procedures, such as moving the flap handle from a 0-position to a 2-position. And when the SFCC monitors that the flap control handle leaves the 0 clamping position, the flap fault monitoring unit is awakened.

After detecting the wake-up signal, the fault monitoring unit and the camera may be started at step 704. For example, the fault monitoring unit may activate different cameras according to a preset time, activate fog level cameras from 7:00 a.m. to 17 a.m. and activate starlight level cameras for other periods of time.

At step 706, images of a plurality of airfoils of a flap captured by a camera may be received. For example, the camera may monitor eight visual markers on four flaps of the left and right wings. The camera outputs data frames to the image processing assembly at 60Hz, 30 fps.

At step 708, image features in the image corresponding to the visual indicia may be extracted. For example, the image processing assembly may monitor the movement of the flap for, for example, 14±2 seconds. The image processing component extracts image features of the data frame with 150 milliseconds as a time region.

At step 710, a current position of each airfoil may be determined based on the extracted image features. For example, the current position of the airfoil may be determined based on visual indicia of the maximum stretch on the image of the airfoil. When the maximum extension section on the image of the airfoil has a complete visual mark, the current clamping position of the airfoil is the complete clamping position of the maximum extension section; or when the maximum extension has an incomplete visual mark on the image of the airfoil, the current position of the airfoil is the complete position of the next maximum extension plus the partial position of the maximum extension. The partial detent may be determined based on a ratio of the incomplete visual indicia of the maximum stretch to the complete visual indicia of the maximum stretch.

At step 712, it may be determined whether the flap is malfunctioning based on the determined current position of each airfoil. In one example, a single airfoil includes a first set of visual indicia on a plurality of extension segments and a second set of visual indicia on a plurality of extension segments, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance, wherein a tip failure of the airfoil is determined if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip threshold. In one example, an asymmetric fault is determined for a first airfoil of a flap if a first current position difference identified based on a visual signature of the first airfoil of the flap and a second current position difference identified based on a visual signature of the second airfoil of the flap exceeds an asymmetric threshold.

In one example, an expected flap position corresponding to the operating instructions may also be determined based on the operating instructions generated by the control computer for controlling flap movement, and if a current position of one or more airfoils differs from the expected flap position by more than a threshold value, a non-commanded failure of the flap is determined.

If a flap failure occurs, the flap failure may be reported to the control computer at step 714. In other embodiments, flap status may be reported to the control computer whether or not a flap failure has occurred.

If no fault is found, steps 706-712 are continued until the condition is ended. The end condition may be, for example, after the flap has stopped moving for a specified time (e.g., after the flap has reached an 18 degree position for 2 seconds), receipt of an instruction to stop monitoring, etc. The camera and the fault monitoring unit may be switched from an "active" to a "standby" state.

During the take-off phase, after 400 feet from the ground, the pilot will retract the flap, first moving the flap handle from the 2-position to the 1-position, which can wake the fault monitoring unit and camera for the first time in the air, and monitor the flap from 18 degrees back to 8 degrees, and after 2 seconds the flap reaches the 8 degree position, these components can go into a "standby" state.

After the slat is fully retracted to the 0 degree position, the pilot may again move the handle from the 1-to-0-click, which may wake up the fault detection unit and camera a second time in the air and monitor the flap retraction from 8 to 0 degrees, and after 2 seconds the flap reaches the 0 degree position, these components may again enter a "standby" state.

In the approach stage, the pilot releases the flap to the landing configuration before 3000 feet from the ground, and the position of the handle is from 0- >1- >2- >3- >4. The fault monitoring unit and camera will wake up 4 times to monitor the flap motion process.

The pilot will retract the wing slats to the cruise configuration during the taxiing phase after the aircraft has completed landing, and the position of the handles will be from 4- >3- >2- >1- >0. The fault monitoring unit and camera will wake up 4 times to monitor the flap motion process.

The various steps and modules of the methods and apparatus described above may be implemented in hardware, software, or a combination thereof. If implemented in hardware, the various illustrative steps, modules, and circuits described in connection with this disclosure may be implemented or performed with a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic component, a hardware component, or any combination thereof. A general purpose processor may be a processor, microprocessor, controller, microcontroller, state machine, or the like. If implemented in software, the various illustrative steps, modules, described in connection with this disclosure may be stored on a computer readable medium or transmitted as one or more instructions or code. Software modules implementing various operations of the present disclosure may reside in storage media such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, cloud storage, etc. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium, as well as execute corresponding program modules to implement the various steps of the present disclosure. Moreover, software-based embodiments may be uploaded, downloaded, or accessed remotely via suitable communication means. Such suitable communication means include, for example, the internet, world wide web, intranet, software applications, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave and infrared communications), electronic communications, or other such communication means.

The numerical values given in the embodiments are only examples and are not intended to limit the scope of the present invention. Furthermore, as an overall solution, there are other components or steps not listed by the claims or the specification of the present invention. Moreover, the singular designation of a component does not exclude the plural designation of such a component.

It is also noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. Additionally, the order of the operations may be rearranged.

The disclosed methods, apparatus, and systems should not be limited in any way. Rather, the present disclosure encompasses all novel and non-obvious features and aspects of the various disclosed embodiments (both alone and in various combinations and subcombinations with one another). The disclosed methods, apparatus and systems are not limited to any specific aspect or feature or combination thereof, nor do any of the disclosed embodiments require that any one or more specific advantages be present or that certain or all technical problems be solved.

The present invention is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the invention and the scope of the appended claims, which are all within the scope of the invention.

Claims

1. A system for monitoring flap faults, comprising:

A camera configured to capture images of a plurality of airfoils of a flap, each airfoil including a plurality of stretches corresponding to a plurality of flap detents, the plurality of stretches having different visual indicia thereon;

An image processing component configured to extract image features in the image corresponding to the visual indicia;

A clamping position determining component configured to determine a current clamping position of each airfoil based on the image features extracted by the image processing component; and

A fault condition determining assembly configured to determine whether the flap is faulty based on the determined current position of each airfoil,

Wherein a single airfoil includes a first set of visual indicia on the plurality of extension segments and a second set of visual indicia on the plurality of extension segments, the first set of visual indicia being spaced apart from the second set of visual indicia by at least a specified distance,

Wherein the fault condition determination component determines that the airfoil is subject to a tip fault if a first current position of the airfoil identified based on the first set of visual markers differs from a second current position of the airfoil identified based on the second set of visual markers by more than a tip threshold.

2. The system for monitoring flap faults of claim 1, wherein:

the current position of the airfoil is determined based on visual indicia of the maximum stretch in the image of the airfoil,

Wherein when the maximum extension section on the image of the airfoil has a full visual mark, the current position of the airfoil is the full position of the maximum extension section; or alternatively

Wherein when the maximum extension has an incomplete visual marker on the image of the airfoil, the current position of the airfoil is the complete position of the next maximum extension plus a partial position of the maximum extension.

3. The system for monitoring flap faults of claim 2, wherein:

The partial detent is determined based on a ratio of the incomplete visual indicia of the maximum stretch to the complete visual indicia of the maximum stretch.

4. The system for monitoring flap faults of claim 1, wherein:

The fault condition determination component determines that an asymmetric fault has occurred with a first airfoil of the flap if a first current detent identified based on a visual marker of the first airfoil of the flap differs from a second current detent identified based on a visual marker of the second airfoil of the flap by more than an asymmetric threshold.

5. The system for monitoring flap faults of claim 1, wherein:

The fault status determination component determines an expected flap position corresponding to an operating instruction generated by a control computer for controlling flap movement based on the operating instruction,

Wherein the fault condition determination component determines that a non-commanded fault has occurred with the flap if a current position of one or more airfoils differs from the expected flap position by more than a threshold value.

6. The system for monitoring flap faults of claim 1, further comprising an actuation component that actuates the camera in response to one or more of:

The aircraft enters a take-off or approach stage;

Flap lever movement; or alternatively

And carrying out a command of flap fault monitoring.

7. An aircraft, comprising:

A flap coupled to a wing of the aircraft; and

The system for monitoring flap faults of any of claims 1 to 6.

8. A method for monitoring flap faults, comprising:

Receiving images of a plurality of airfoils of a flap captured by a camera, each airfoil including a plurality of extension segments corresponding to a plurality of flap detents, the plurality of extension segments having different visual indicia thereon;

extracting image features corresponding to the visual marks in the image;

Determining a current clip position for each airfoil based on the image features extracted by the image processing component; and

Judging whether the flap fails according to the determined current clamping position of each airfoil surface,

Wherein if a first current position of the airfoil identified based on the first set of visual indicia differs from a second current position of the airfoil identified based on the second set of visual indicia by more than a tip-out threshold, a tip-out fault is determined for the airfoil.

9. The method for monitoring flap faults of claim 8 in which the current position of the airfoil is determined based on visual indicia of a maximum extension on an image of the airfoil,

10. The method for monitoring flap faults of claim 9, wherein:

11. The method for monitoring flap faults of claim 8, wherein:

and if the difference between the first current clamping position identified based on the visual mark of the first airfoil surface of the flap and the second current clamping position identified based on the visual mark of the second airfoil surface of the flap exceeds an asymmetry threshold value, determining that the first airfoil surface and the second airfoil surface have an asymmetry fault.

12. The method for monitoring flap faults of claim 8, further comprising:

determining an expected flap position corresponding to an operation instruction generated by a control computer for controlling flap movement; and

If the current position of one or more airfoils differs from the expected flap position by more than a threshold value, determining that the flap is subject to an uncommanded failure.

13. The method for monitoring flap faults of claim 8 further comprising activating the camera in response to one or more of:

The aircraft enters a take-off or approach stage;

Flap lever movement; or alternatively

And carrying out a command of flap fault monitoring.