CN115079293A - Equipment and method for detecting foreign matters and road surfaces of airfield runway - Google Patents
Equipment and method for detecting foreign matters and road surfaces of airfield runway Download PDFInfo
- Publication number
- CN115079293A CN115079293A CN202210570956.XA CN202210570956A CN115079293A CN 115079293 A CN115079293 A CN 115079293A CN 202210570956 A CN202210570956 A CN 202210570956A CN 115079293 A CN115079293 A CN 115079293A
- Authority
- CN
- China
- Prior art keywords
- infrared
- image
- runway
- foreign matter
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
- G01V8/00—Prospecting or detecting by optical means
- G01V8/10—Detecting, e.g. by using light barriers
- G01V8/20—Detecting, e.g. by using light barriers using multiple transmitters or receivers
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N29/00—Investigating or analysing materials by the use of ultrasonic, sonic or infrasonic waves; Visualisation of the interior of objects by transmitting ultrasonic or sonic waves through the object
- G01N29/04—Analysing solids
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N29/00—Investigating or analysing materials by the use of ultrasonic, sonic or infrasonic waves; Visualisation of the interior of objects by transmitting ultrasonic or sonic waves through the object
- G01N29/34—Generating the ultrasonic, sonic or infrasonic waves, e.g. electronic circuits specially adapted therefor
- G01N29/348—Generating the ultrasonic, sonic or infrasonic waves, e.g. electronic circuits specially adapted therefor with frequency characteristics, e.g. single frequency signals, chirp signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/803—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of input or preprocessed data
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N2291/00—Indexing codes associated with group G01N29/00
- G01N2291/02—Indexing codes associated with the analysed material
- G01N2291/023—Solids
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Analytical Chemistry (AREA)
- Chemical & Material Sciences (AREA)
- Biochemistry (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Geophysics (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Acoustics & Sound (AREA)
- Geophysics And Detection Of Objects (AREA)
Abstract
The invention provides equipment and a method for detecting foreign matters and a pavement of an airport runway, and relates to the technical field of airport pavement safety detection. On one hand, the device can effectively detect the foreign matters on the airport runway in real time through a plurality of detection units arranged at two sides of the airport runway at intervals, can identify and classify the foreign matters, can monitor the pavement health condition of the airport runway through a vibration sensor embedded under the pavement in the detection units, and then transmits related detection result signals to a remote monitoring terminal through a wireless sensor network; on the other hand, the method monitors the foreign matters on the runway and the state of the pavement by acquiring the energy of the first infrared ray and the second infrared ray, the infrared image of the pavement containing the foreign matters on the runway and the vibration frequency data of the pavement and performing corresponding processing.
Description
Technical Field
The invention relates to the technical field of airport pavement detection, in particular to equipment and a method for detecting foreign matters and a pavement on an airport runway.
Background
With the rapid development of economy and the continuous improvement of living standard, taking an airplane for going out becomes a choice for many people. Therefore, the flight safety of aircraft is increasingly receiving attention. Foreign object detection and road surface condition detection in airport runways are the two most important aspects concerning the safety of take-off and landing of airplanes. The airfield runway foreign matter is FOD (ForeignObjectDebris) which is all foreign matters damaging the safety of the airfield runway, and comprises an airplane, an engine connector (a nut, a screw, a gasket, a fuse and the like), a mechanical tool, flying articles (a nail, a private certificate, a pen, a pencil and the like), wild animals, leaves, stones and sand, a pavement material, a wood block, a plastic or polyethylene material, a paper product, ice ballast of a running area and the like. FOD on airport pavement is easily inhaled by engines, causing engine failure and even combustion, which is very serious for the flight safety of airplanes and the safety of passengers' lives and property. On the other hand, the airport pavement is in a stress-strain overlapping change state under the repeated action of load for a long time, and the situations of cracks, damages and the like can occur over the long time, thereby causing influence on the takeoff and landing of the airplane.
In actual life, in airports, manual patrol cars are generally used for carrying out FOD detection and road surface condition detection on airport runways, but when the manual patrol cars are used for checking the ground of the runways, omission and negligence can occur, obstacles dropped by the patrol cars are usually not easy to observe by the patrol cars, regular detection is usually carried out, the interval time is long, and the real-time performance of data is weak.
Disclosure of Invention
The invention aims to provide equipment and a method for detecting foreign matters and a pavement of an airport runway, which can effectively detect the foreign matters on the airport runway in real time, monitor the pavement health condition of the airport runway, reduce the workload of workers and improve the safety of an airplane during taking off and landing.
The embodiment of the invention is realized by the following steps:
in a first aspect, an embodiment of the present application provides an apparatus for detecting foreign objects and a road surface on an airport runway, which includes a plurality of detection units disposed at two sides of the airport runway at intervals, the detection units are connected to a monitoring terminal through a wireless sensor network, and the detection units include: the system comprises an infrared transmitter, an infrared receiver, an infrared camera and a processor which are arranged on a road surface, and a vibration sensor which is buried under the road surface, wherein the infrared transmitter, the infrared receiver, the infrared camera and the vibration sensor are all connected with the processor;
the infrared transmitter is used for periodically radiating infrared rays outwards;
the infrared receiver is used for receiving infrared light reflected by foreign matters on the runway and infrared light in ambient light, processing the infrared light to obtain energy data of the infrared light, and transmitting the energy data to the processor for analysis;
the infrared camera is used for acquiring an infrared image of a road surface containing foreign matters on the runway and transmitting the infrared image to the processor for analysis and identification;
the vibration sensor is used for detecting the vibration frequency of the road surface and transmitting the vibration frequency data to the processor;
the processor is used for analyzing and determining whether the runway foreign matter and the type of the runway foreign matter are detected or not according to the received infrared ray energy data and the infrared image, judging whether the runway surface is abnormal or not according to the received vibration frequency data and generating a result signal, and sending the result signal to the monitoring terminal through the wireless sensor network.
Based on the first aspect, in some embodiments of the invention, the infrared receiver includes a first infrared receiver and a second infrared receiver;
the first infrared light received by the first infrared receiver comprises reflected light obtained by reflecting infrared light radiated outwards by the infrared transmitter by foreign matters on the runway and infrared light in ambient light;
the second infrared light received by the second infrared receiver is infrared light in ambient light;
the processor is used for judging whether the energy difference value of the first infrared light and the second infrared light is larger than a preset threshold value, and if yes, determining that the runway foreign matter is detected and generating a foreign matter signal.
In some embodiments of the invention based on the first aspect, the apparatus further comprises a sound emitting device connected to the processor, the sound emitting device being configured to simulate different frequencies of ultrasound based on an acoustic parametric array technique.
In a second aspect, embodiments of the present application provide a method for airport runway foreign object detection and runway surface detection, which includes:
step S1: controlling the infrared transmitter to periodically radiate infrared rays outwards;
step S2: acquiring energy data of a first infrared ray received by a first infrared receiver and a second infrared ray received by a second infrared receiver, an infrared image of a pavement surface containing a runway foreign matter acquired by an infrared camera and vibration frequency data detected by a vibration sensor;
step S3: judging whether the energy difference value of the first infrared light and the second infrared light is larger than a preset threshold value or not, if so, determining that the foreign matter is detected and generating a foreign matter signal, and if not, generating a foreign matter-free signal;
step S4: processing and identifying the infrared image, determining the type of the runway foreign matter and generating a foreign matter classification result signal;
step S5: judging whether the vibration frequency data exceed a preset safe vibration frequency threshold value, if so, generating a pavement abnormal signal, and if not, generating a pavement normal signal;
step S6: and sending the generated foreign matter signal, the foreign matter-free signal, the foreign matter classification result signal, the road surface abnormal signal and the road surface normal signal to a monitoring terminal through a wireless sensor network.
Based on the second aspect, in some embodiments of the invention, the step of determining whether the energy difference between the first infrared ray and the second infrared ray is greater than a preset threshold specifically includes:
acquiring a receiving voltage value representing the energy value of the first infrared ray and a reference voltage value representing the energy value of the second infrared ray;
and judging whether the receiving voltage value is greater than the reference voltage value, if so, determining that the foreign object on the runway is detected and generating a foreign object signal, and if not, generating a foreign object-free signal.
Based on the second aspect, in some embodiments of the present invention, the step of processing and identifying the infrared image, determining the type of the runway foreign object and generating the foreign object classification result signal includes:
processing the infrared image to obtain a corresponding visual saliency area image;
processing the image of the visual salient region by utilizing an LCM algorithm to obtain a local contrast image;
performing self-adaptive threshold segmentation on the local contrast image to obtain a foreign matter detection result image;
and identifying the foreign matter detection result graph, determining the type of the foreign matter on the runway and generating a foreign matter classification result.
Based on the second aspect, in some embodiments of the present invention, the step of processing the infrared image to obtain the corresponding image of the visually significant area specifically includes:
carrying out graying processing on the infrared image to obtain a grayed image;
calculating and obtaining an area saliency image based on image entropy and an area saliency image based on local similarity according to the gray images;
and fusing the region saliency image based on the image entropy and the region saliency image based on the local similarity to obtain the visual saliency region image.
Based on the second aspect, in some embodiments of the present invention, the step of calculating and obtaining the region saliency image based on the image entropy according to the grayscale image specifically includes:
calculating the information entropy H (x, y) of each pixel point (x, y) in the gray-scale image to obtain an information entropy chart corresponding to the gray-scale image, wherein the calculation formula is as follows:
wherein Ω (x, y) represents a local area around the pixel point (x, y), and the pixel value in the local area is projected onto K intervals, P b (x, y) represents the probability that the pixel value is in the interval b;
and carrying out binarization processing on the information entropy diagram to obtain an area saliency image based on the image entropy.
Based on the second aspect, in some embodiments of the present invention, the step of calculating and obtaining a region saliency image based on local similarity from a grayed image specifically includes:
dividing the gray-scale image into a plurality of regions with the same size;
calculating a similarity value between every two adjacent areas along the horizontal direction, sequentially judging whether the similarity value is higher than a preset standard value, if so, marking the two areas as insignificant areas, and if not, marking the two areas as significant areas to obtain a horizontal area significance image;
calculating a similarity value between every two adjacent areas along the vertical direction, sequentially judging whether the similarity value is higher than a preset standard value, if so, marking the two areas as non-salient areas, otherwise, marking the two areas as salient areas, and obtaining a vertical area saliency image;
and fusing the horizontal area saliency image and the vertical area saliency image to obtain an area saliency image based on local similarity.
Based on the second aspect, in some embodiments of the present invention, the step of calculating the similarity value between each two adjacent regions along the horizontal direction includes:
according to the formula
Calculating a similarity value between two adjacent regions, where p Ω1，Ω2 Representing the similarity of two regions, Ω 1 and Ω 2, F Ω1 (x, y) represents a pixel value in the region of Ω 1, F Ω2 (x, y) represents a pixel value in the Ω 2 region.
Compared with the prior art, the embodiment of the invention has at least the following advantages or beneficial effects:
according to the first aspect, the embodiment of the application provides equipment for detecting foreign matters and detecting a road surface of an airport runway, the foreign matters on the airport runway are effectively detected in real time by a plurality of detection units arranged on two sides of the airport runway at intervals, the foreign matters can be identified and classified, meanwhile, the health condition of the road surface of the airport runway can be monitored by a vibration sensor buried under the road surface in the detection units, then related detection result signals are transmitted to a remote monitoring terminal through a wireless sensor network, a worker can comprehensively consider the whole condition of the airport runway according to the feedback result of each detection unit, and corresponding measures can be taken timely if problems are found, so that the airport runway is guaranteed to be suitable for safe flight. On the whole, the device can effectively monitor the health conditions of foreign matters and the road surfaces of the airfield runway in real time, and can improve the safety of the aircraft during taking off and landing while reducing the workload of workers.
In a second aspect, the embodiment of the present application provides a method for detecting foreign objects and a runway surface, wherein the acquired energy data of the first infrared light and the second infrared light are processed and judged to determine whether a runway foreign object is detected and generate a corresponding result signal, meanwhile, the type of the runway foreign object is determined and a corresponding result signal is generated by performing target detection and identification on an infrared image of the runway surface containing the runway foreign object, in addition, the vibration frequency data of the runway surface is analyzed and judged to determine whether the runway surface is abnormal and generate a corresponding result signal, and finally, all related result signals are transmitted to a remote monitoring terminal, so as to realize real-time monitoring of the health conditions of the runway foreign object and the runway surface.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
Fig. 1 is a schematic diagram illustrating a position distribution of an apparatus for detecting foreign objects and a pavement on an airport runway according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for airfield runway foreign object detection and runway surface detection according to an embodiment of the present invention;
FIG. 3 is a detailed flowchart of the steps of processing and recognizing the infrared images, determining the type of the runway foreign object and generating a foreign object classification result signal in the method for detecting the foreign object and the runway surface of the airport runway according to the embodiment of the present invention;
FIG. 4 is a schematic illustration of a foreign object on an airport runway according to an embodiment of the present invention;
fig. 5 is a diagram illustrating a foreign object detection result obtained by processing an infrared image according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Examples
Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the individual features of the embodiments can be combined with one another without conflict.
In a first aspect, please refer to fig. 1, fig. 1 is a schematic diagram illustrating a position distribution of an apparatus for detecting foreign objects and a runway surface in an airport runway according to an embodiment of the present disclosure. This equipment includes that a plurality of intervals set up the detecting element in airport runway both sides, and detecting element passes through wireless sensor network and is connected with monitor terminal, and above-mentioned detecting element includes: the system comprises an infrared transmitter, an infrared receiver, an infrared camera and a processor which are arranged on a road surface, and a vibration sensor which is buried under the road surface, wherein the infrared transmitter, the infrared receiver, the infrared camera and the vibration sensor are all connected with the processor;
the infrared transmitter is used for periodically radiating infrared rays outwards;
the infrared receiver is used for receiving infrared light reflected by foreign matters on the runway and infrared light in ambient light, processing the infrared light to obtain energy data of the infrared light, and transmitting the energy data to the processor for analysis;
the infrared camera is used for acquiring an infrared image of a road surface containing foreign matters on the runway and transmitting the infrared image to the processor for analysis and identification;
the vibration sensor is used for detecting the vibration frequency of the road surface and transmitting the vibration frequency data to the processor;
the processor is used for analyzing and determining whether the runway foreign matter and the type of the runway foreign matter are detected or not according to the received infrared ray energy data and the infrared image, judging whether the runway surface is abnormal or not according to the received vibration frequency data and generating a result signal, and sending the result signal to the monitoring terminal through the wireless sensor network.
In the technical solution provided in this embodiment, a plurality of detection units may be arranged at intervals on both sides of the runway of the airport, for example, one detection unit may be arranged every 50 m. Each detection unit can monitor the foreign matters and the road surface health conditions of the runways in the adjacent areas, the related detection result data are transmitted to the remote monitoring terminal through the wireless sensor network, the monitoring terminal comprehensively considers the overall situation of the airport runways according to the detection result data of each detection unit, and if a problem is found, an alarm signal is sent out to remind workers to take corresponding measures in time so as to ensure that the airport runways are suitable for safe flight. Furthermore, each detection unit has an ID number, when the detection result data are transmitted through the wireless sensor network node, the ID numbers are transmitted so that the monitoring terminal can identify the detection result data of different detection units, and the area with problems is locked according to the ID numbers, so that the staff can timely process the data, and the obstacle clearing efficiency is improved.
Specifically, the processor controls the infrared transmitter to periodically radiate infrared light outwards, then the reflected infrared light is received by the infrared receiver, corresponding processing is carried out to obtain energy data of the infrared light, the energy data of the infrared light is transmitted to the processor, and the processor analyzes the energy data of the infrared light obtained through reflection to judge whether foreign matters exist on the runway or not. In addition, the foreign matter on the road surface can be detected according to the reflection time of the infrared light (because the reflection time of the emitted infrared light is obviously changed after encountering the foreign matter, whether the foreign matter exists on the runway can be judged by analyzing and comparing the actual reflection time with a standard preset time threshold value). If the foreign matter is detected to exist on the runway, the processor receives the infrared image of the runway surface containing the foreign matter, which is collected by the infrared camera, processes and identifies the infrared image, determines the type of the foreign matter on the runway and generates a corresponding detection result signal. Meanwhile, when an airplane flies over a runway, the vibration sensor buried under the pavement can detect the vibration frequency of the pavement and send the vibration frequency data of the pavement to the processor, and the processor compares the vibration frequency data with a preset safe vibration frequency threshold value to judge whether the pavement is normal and generate a corresponding detection result signal. And finally, the processor sends the related detection result signals to the remote monitoring terminal through the wireless sensor network, and the monitoring terminal comprehensively considers the overall situation of the airport runway according to the detection result signals of each detection unit.
For example, the equipment can be directly powered by the power supply systems of the street lamps on the two sides of the airport runway, so that the space is saved. In addition, the processor may be an integrated circuit chip having signal processing capabilities. The Processor may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), etc.; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
Based on the first aspect, in some embodiments of the invention, the infrared receiver includes a first infrared receiver and a second infrared receiver;
the first infrared light received by the first infrared receiver comprises reflected light obtained by reflecting infrared light radiated outwards by the infrared transmitter by foreign matters on the runway and infrared light in ambient light;
the second infrared light received by the second infrared receiver is infrared light in ambient light;
the processor is used for judging whether the energy difference value of the first infrared light and the second infrared light is larger than a preset threshold value, and if yes, determining that the runway foreign matter is detected and generating a foreign matter signal.
In the technical scheme provided by this embodiment, the first infrared light received by the first infrared receiver of any one detection unit includes reflected light obtained by infrared light emitted by an infrared emitter reflected by a foreign object on the runway and infrared light in ambient light, the second infrared light received by the second infrared receiver is infrared light in the ambient light, the processor calculates an energy difference between the first infrared light and the second infrared light, compares the energy difference with a preset threshold, and determines that a foreign object exists in the runway area corresponding to the detection unit when the energy difference is greater than the preset threshold; and when the energy difference value is smaller than a preset threshold value, judging that no foreign matter exists in the runway area corresponding to the detection unit. And meanwhile, transmitting the result data obtained by judgment to a remote monitoring terminal through a wireless sensor network for comprehensive judgment.
In some embodiments of the invention based on the first aspect, the apparatus further comprises a sound emitting device connected to the processor, the sound emitting device being configured to simulate different frequencies of ultrasound based on an acoustic parametric array technique.
In the technical scheme provided by the embodiment, in actual life, the runway is flat and spacious, and part of birds can fly and nest in the area of the runway, which may affect the safe flight of the airplane, so that the sound emitting device can be installed on the equipment, and the sound of natural enemies of the birds can be simulated by emitting ultrasonic waves with different frequencies, so as to achieve the aim of repelling the birds.
Illustratively, acoustic parametric array techniques may be employed to simulate different frequencies of ultrasound. The acoustic parametric array is an inverted-type array, and is a difference frequency wave, a frequency wave and the like generated by mutual interference of two beams of ultrasonic waves with different frequencies and limited amplitudes coaxially transmitted by a physical transducer in a nonlinear propagation common area. From the array perspective, the common region where the nonlinear action occurs is considered as the sound source emission array of difference frequency wave and frequency wave, i.e. the acoustic parametric array. The acoustic parametric array has strong sharp directivity, wide frequency band and noise resistance, is used for simulating ultrasonic waves with different frequencies, can achieve the aim of repelling birds and ensures the safety of airport runways.
In a second aspect, please refer to fig. 2, fig. 2 is a flowchart illustrating a method for detecting foreign objects and a runway surface of an airport runway according to an embodiment of the present application, the method includes the following steps:
step S1: the infrared transmitter is controlled to periodically radiate infrared light outwards.
In the above step, after the processor is powered on, the processor may control the infrared emitter to be powered on, where the infrared emitter may include at least one infrared light emitting LED.
Step S2: the method comprises the steps of obtaining energy data of first infrared rays received by a first infrared receiver and second infrared rays received by a second infrared receiver, and obtaining infrared images of a pavement surface containing the runway foreign matters and vibration frequency data detected by a vibration sensor, wherein the infrared images are collected by an infrared camera.
In the above step, the first infrared receiver and the second infrared receiver may respectively include at least one infrared receiving LED.
Step S3: and judging whether the energy difference value of the first infrared light and the second infrared light is greater than a preset threshold value, if so, determining that the foreign object on the runway is detected and generating a foreign object signal, and if not, generating a foreign object-free signal.
In the above steps, the first infrared light and the second infrared light both include infrared light in ambient light, and an energy difference between the first infrared light and the second infrared light is an infrared light emitted by an infrared emitter reflected by the runway foreign object included in the first infrared light, so that whether the runway foreign object is detected can be determined by comparing the energy difference with a preset threshold. It should be noted that the preset threshold may be set according to the requirement of the detection tolerance, wherein the higher the preset threshold is, the higher the accuracy rate of determining that the obstacle is detected is, but at the same time, the accuracy rate of determining that the obstacle is not detected is reduced, and different settings may be performed according to actual situations.
Specifically, first, a receiving voltage value indicating an energy value of the first infrared ray and a reference voltage value indicating an energy value of the second infrared ray are obtained; the first infrared receiver and the second infrared receiver can generate photocurrent when infrared light is received, and the stronger the energy of the received infrared light is, the larger the photocurrent is. Therefore, it is possible to provide a received voltage value representing the energy value of the first infrared ray by detecting the magnitude of the photocurrent generated by the first infrared receiver, and to provide a reference voltage value representing the energy value of the second infrared ray by detecting the magnitude of the photocurrent generated by the second infrared receiver. Then, judging whether the receiving voltage value is larger than the reference voltage value, if so, determining that the foreign object on the runway is detected and generating a foreign object signal, and if not, generating a foreign object-free signal; the signal may be a level signal, such as a high level signal indicating that a runway threshold is detected or a low level signal indicating that no runway threshold is detected.
Step S4: and processing and identifying the infrared image, determining the type of the runway foreign matter and generating a foreign matter classification result signal. Referring to fig. 3, the steps specifically include:
step S4-1: processing the infrared image to obtain a corresponding visual saliency area image;
referring to fig. 4 and 5, in the above steps, because the foreign object on the runway is generally small, the runway foreign object target to be detected in the collected infrared image of the runway surface containing the foreign object is often weak, and the foreign object target area may attract the visual attention of people, that is, for human eyes, the weak target area is an area with high significance and large information amount in the infrared image. Therefore, a relatively unimportant area in the infrared image can be removed firstly in a significance measurement mode, the background is suppressed to a certain degree, the foreign object target is highlighted, and then the weak foreign object target is searched from the extracted significant area, so that the processing time of the processor can be effectively reduced, and the overall operation efficiency is improved.
Illustratively, the step of processing the infrared image to obtain a corresponding image of the visually significant region specifically includes:
step S4-1-1: and carrying out graying processing on the infrared image to obtain a grayed image. In the step, the calculation amount can be reduced by carrying out graying processing on the image, and the efficiency of subsequent image processing is improved.
Step S4-1-2: and calculating and obtaining a region saliency image based on the image entropy and a region saliency image based on the local similarity according to the gray-scale image.
In the above steps, the process of calculating and obtaining the region saliency image based on the image entropy is as follows:
firstly, calculating the information entropy H (x, y) of each pixel point (x, y) in the gray-scale image to obtain an information entropy chart corresponding to the gray-scale image, wherein the calculation formula is as follows:
wherein Ω (x, y) represents a local area around the pixel point (x, y), and the pixel value in the local area is projected onto K intervals, P b (x, y) represents the probability that the pixel value is in the section b;
and then, carrying out binarization processing on the information entropy diagram to obtain an area saliency image based on the image entropy.
The regional significance image based on the image entropy only considers the gray level statistical distribution of each region and does not consider the spatial distribution characteristics of pixels. In an actual infrared image, the background often presents certain spatial similarity, the repeated areas cause visual redundancy, and the detection and identification of foreign matters are meaningless, so that the similar areas in the image need to be removed to better measure the visual saliency.
The process of calculating and obtaining the regional saliency image based on local similarity is as follows:
firstly, dividing a gray image into a plurality of regions with the same size; in this step, since the regions with similarity tend to be located closer to each other in the actual image, rather than far apart from each other, the image may be divided into several regions with the same size, and then the similarity between two adjacent regions may be calculated in the horizontal and vertical directions to remove the similar regions.
Then, calculating a similarity value between every two adjacent areas along the horizontal direction, and sequentially judging whether the similarity value is higher than a preset standard value, if so, marking the two areas as non-significant areas, otherwise, marking the two areas as significant areas, and obtaining a horizontal area significance image;
illustratively, it can be according to a formula
Calculating a similarity value between two adjacent regions, wherein rho Ω1，Ω2 Representing the similarity of two regions, Ω 1 and Ω 2, F Ω1 (x, y) represents a pixel value in the region of Ω 1, F Ω2 (x, y) represents a pixel value in the Ω 2 region.
Then, calculating a similarity value between every two adjacent areas along the vertical direction, and sequentially judging whether the similarity value is higher than a preset standard value, if so, marking the two areas as non-significant areas, otherwise, marking the two areas as significant areas, and obtaining a vertical area significance image;
in the above steps, firstly, similarity values are calculated along the horizontal direction, if the similarity value between two adjacent regions is higher than a preset standard value, it is indicated that the two regions have strong similarity and are represented as a visual redundant region, so that the gray values of the two regions are simultaneously set to be 0 and marked as an insignificant region; and then, calculating a similarity value along the vertical direction, and if the similarity value between the two adjacent regions is lower than a preset standard value, indicating that the two regions have low similarity and visually represent a large difference, and therefore, setting the gray values of the two regions to be 1 at the same time and marking the two regions as the significant regions. This process is repeated until each region is subjected to horizontal and vertical direction saliency measurements, respectively, resulting in corresponding horizontal region saliency images and vertical region saliency images.
And finally, fusing the horizontal area saliency image and the vertical area saliency image to obtain an area saliency image based on local similarity.
Step S4-1-3: and fusing the region saliency image based on the image entropy and the region saliency image based on the local similarity to obtain the visual saliency region image.
Illustratively, the fusion formula is:
wherein R is S (x, y) represents a visually significant region,
representing a salient region based on the entropy of the image,
representing salient regions based on local similarity. Note that the visual saliency area obtained here is displayed in the form of a binary image, where 1 represents that the point is highly salient and 0 represents that the point is low salient.
Step S4-2: processing the image of the visual salient region by utilizing an LCM algorithm to obtain a local contrast image;
in the above steps, the LCM algorithm is a pixel-level operation, and a weak foreign object target is highlighted and a background is suppressed by calculating a difference between a region where each pixel point is located and a peripheral region, so that the obtained visually significant region image can be further processed by the LCM algorithm to obtain a local contrast image corresponding to the visually significant region image, and a position where a pixel value in the local contrast image is the largest is a position where the foreign object target is located. The principle of obtaining a local contrast image by LCM algorithm processing belongs to the prior art, and is not described herein again.
Step S4-3: performing self-adaptive threshold segmentation on the local contrast image to obtain a foreign matter detection result image;
in the above steps, the adaptive threshold method can calculate the local threshold according to the brightness distribution of different regions of the image, so that different thresholds can be adaptively calculated for different regions of the image. When the local threshold of each region in the local contrast image is calculated by the adaptive threshold method, the region with the largest local threshold is the foreign object target region to be identified, and then the image is segmented according to the threshold, so that a foreign object detection result map (as shown in fig. 5) corresponding to the foreign object target region to be identified can be obtained.
Step S4-4: and identifying the foreign matter detection result graph, determining the type of the foreign matter on the runway and generating a foreign matter classification result.
In the above steps, the neural network model may be used to identify the foreign object detection result map, determine the type of the runway foreign object, and generate a foreign object classification result. The neural network model is a model which is obtained after a series of historical data (foreign body types) are trained and can be used for foreign body identification and classification, and a specific model construction method and a specific model construction process belong to the prior art and are not described again here.
Step S5: and judging whether the vibration frequency data exceed a preset safe vibration frequency threshold, if so, generating a pavement abnormal signal, and if not, generating a pavement normal signal.
In the above steps, if the road surface has cracks, depressions, etc., when the object passes quickly through the runway, the vibration frequency of the road surface will change obviously, so that the vibration frequency data collected in real time can be compared with the preset safe vibration frequency threshold, and if the vibration frequency data exceeds the preset safe vibration frequency threshold, indicating that the road surface has problems, a road surface abnormal signal is generated; and if the vibration frequency data do not exceed the preset safe vibration frequency threshold value, indicating that the state of the pavement is in a controllable range, generating a normal pavement signal. And then, the generated road surface abnormal signal and the generated road surface normal signal are sent to a monitoring terminal through a wireless sensor network, and if the monitoring terminal receives the road surface abnormal signal, an alarm is sent to remind a worker to check and repair the road surface abnormal signal in time, so that the worker is helped to adjust the arrangement strategy of taking off and landing of the airplane in time. And because every detecting element all has own ID number, so the staff can carry out quick location to the detecting element who sends out the road surface abnormal signal, improves maintenance efficiency.
Step S6: and sending the generated foreign matter signal, the foreign matter-free signal, the foreign matter classification result signal, the road surface abnormal signal and the road surface normal signal to a monitoring terminal through a wireless sensor network.
In the steps, each detection unit sends the relevant detection result data obtained by processing the detection result data to the monitoring terminal through the wireless sensor network, the monitoring terminal comprehensively considers the overall situation of the airport runway according to the detection result data of each detection unit, and if a problem is found, an alarm signal is sent out to remind workers to take corresponding measures in time so as to ensure that the airport runway is suitable for safe flight.
In summary, in one aspect, embodiments of the present application provide an apparatus for detecting foreign objects and a runway surface on an airport runway, the apparatus effectively detects the foreign objects on the airport runway in real time by a plurality of detection units disposed at two sides of the airport runway at intervals, and can identify and classify the foreign objects, meanwhile, the apparatus can monitor the health status of the runway surface of the airport runway by a vibration sensor embedded under the runway surface in the detection unit, and then transmit related detection result signals to a remote monitoring terminal through a wireless sensor network, so that a worker can comprehensively consider the overall situation of the airport runway according to the feedback result of each detection unit, and timely take corresponding measures if a problem is found, so as to ensure that the airport runway is suitable for safe flight. On the whole, the device can effectively monitor the health conditions of foreign matters and the road surfaces of the airfield runway in real time, and can improve the safety of the aircraft during taking off and landing while reducing the workload of workers. On the other hand, the embodiment of the application provides a method for detecting foreign matters on an airport runway and detecting a pavement, which comprises the steps of processing and judging the acquired energy data of a first infrared light ray and a second infrared light ray, determining whether the runway foreign matters are detected and generating corresponding result signals, simultaneously, carrying out target detection and identification on an infrared image of the pavement containing the runway foreign matters, determining the type of the runway foreign matters and generating corresponding result signals, analyzing and judging the vibration frequency data of the pavement, determining whether the pavement is abnormal or not and generating corresponding result signals, and finally transmitting all the related result signals to a remote monitoring terminal to realize real-time monitoring on the health conditions of the airport runway foreign matters and the pavement.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Claims (10)
1. An apparatus for detecting foreign matter and a road surface on an airport runway, comprising a plurality of detection units disposed at both sides of the airport runway at intervals, the detection units being connected with a monitoring terminal through a wireless sensor network, wherein the detection units comprise: the system comprises an infrared transmitter, an infrared receiver, an infrared camera and a processor which are arranged on a road surface, and a vibration sensor which is buried under the road surface, wherein the infrared transmitter, the infrared receiver, the infrared camera and the vibration sensor are all connected with the processor;
the infrared transmitter is used for periodically radiating infrared rays outwards;
the infrared receiver is used for receiving infrared light reflected by foreign matters on the runway and infrared light in ambient light, processing the infrared light to obtain energy data of the infrared light, and transmitting the energy data to the processor for analysis;
the infrared camera is used for acquiring an infrared image of a road surface containing the foreign matters on the runway and transmitting the infrared image to the processor for analysis and identification;
the vibration sensor is used for detecting the vibration frequency of the road surface and transmitting the vibration frequency data to the processor;
the processor is used for analyzing and determining whether the runway foreign matter and the type of the runway foreign matter are detected or not according to the received infrared ray energy data and the infrared image, judging whether the runway surface is abnormal or not according to the received vibration frequency data and generating a result signal, and sending the result signal to the monitoring terminal through the wireless sensor network.
2. The apparatus for airport runway threshold detection and runway pavement detection as recited in claim 1, wherein the infrared receiver comprises a first infrared receiver and a second infrared receiver;
the first infrared light received by the first infrared receiver comprises reflected light obtained by reflecting infrared light radiated outwards by the infrared transmitter by foreign matters on the runway and infrared light in ambient light;
the second infrared light received by the second infrared receiver is infrared light in ambient light;
the processor is used for judging whether the energy difference value of the first infrared light and the second infrared light is larger than a preset threshold value, if so, the runway foreign matter is determined to be detected and a foreign matter signal is generated.
3. The apparatus of claim 1, further comprising a sound emitting device coupled to said processor, said sound emitting device configured to simulate different frequencies of ultrasound based on an acoustic parametric array technique.
4. A method for airfield runway foreign object detection and runway surface detection, comprising the steps of:
step S1: controlling the infrared transmitter to periodically radiate infrared rays outwards;
step S2: acquiring energy data of a first infrared ray received by a first infrared receiver and a second infrared ray received by a second infrared receiver, an infrared image of a pavement surface containing a runway foreign matter acquired by an infrared camera and vibration frequency data detected by a vibration sensor;
step S3: judging whether the energy difference value of the first infrared light and the second infrared light is larger than a preset threshold value or not, if so, determining that the foreign matter is detected and generating a foreign matter signal, and if not, generating a foreign matter-free signal;
step S4: processing and identifying the infrared image, determining the type of the runway foreign matter and generating a foreign matter classification result signal;
step S5: judging whether the vibration frequency data exceeds a preset safe vibration frequency threshold value, if so, generating a pavement abnormal signal, and if not, generating a pavement normal signal;
step S6: and sending the generated foreign matter signal, the foreign matter-free signal, the foreign matter classification result signal, the road surface abnormal signal and the road surface normal signal to a monitoring terminal through a wireless sensor network.
5. The method of claim 4, wherein the step of determining whether the energy difference between the first infrared light and the second infrared light is greater than a predetermined threshold comprises:
acquiring a receiving voltage value representing the energy value of the first infrared ray and a reference voltage value representing the energy value of the second infrared ray;
and judging whether the receiving voltage value is greater than the reference voltage value, if so, determining that the foreign object on the runway is detected and generating a foreign object signal, and if not, generating a foreign object-free signal.
6. The method for airport runway foreign object detection and runway pavement detection as recited in claim 4, wherein the step of processing and identifying the infrared images, determining the type of runway foreign object and generating a foreign object classification result signal comprises:
processing the infrared image to obtain a corresponding visual saliency area image;
processing the image of the visual salient region by utilizing an LCM algorithm to obtain a local contrast image;
performing self-adaptive threshold segmentation on the local contrast image to obtain a foreign matter detection result image;
and identifying the foreign matter detection result graph, determining the type of the foreign matter on the runway and generating a foreign matter classification result.
7. The method of claim 6, wherein the step of processing the infrared images to obtain corresponding visually significant region images comprises:
carrying out graying processing on the infrared image to obtain a grayed image;
calculating and obtaining an area saliency image based on image entropy and an area saliency image based on local similarity according to the gray images;
and fusing the region saliency image based on the image entropy and the region saliency image based on the local similarity to obtain the visual saliency region image.
8. The method for detecting foreign objects and road surfaces on airport runways according to claim 7, wherein the step of calculating and obtaining the area saliency image based on image entropy from the grayed image specifically comprises:
calculating the information entropy H (x, y) of each pixel point (x, y) in the gray-scale image to obtain an information entropy chart corresponding to the gray-scale image, wherein the calculation formula is as follows:
wherein Ω (x, y) represents a local area around the pixel point (x, y), and the pixel value in the local area is projected onto K intervals, P b (x, y) represents the probability that the pixel value is in the section b;
and carrying out binarization processing on the information entropy diagram to obtain an area saliency image based on the image entropy.
9. The method of claim 7, wherein the step of calculating and obtaining a local similarity-based regional saliency image from a grayed out image comprises:
dividing the gray-scale image into a plurality of regions with the same size;
calculating a similarity value between every two adjacent areas along the horizontal direction, sequentially judging whether the similarity value is higher than a preset standard value, if so, marking the two areas as insignificant areas, and if not, marking the two areas as significant areas to obtain a horizontal area significance image;
calculating a similarity value between every two adjacent areas along the vertical direction, sequentially judging whether the similarity value is higher than a preset standard value, if so, marking the two areas as insignificant areas, and if not, marking the two areas as significant areas to obtain a vertical area significance image;
and fusing the horizontal area saliency image and the vertical area saliency image to obtain an area saliency image based on local similarity.
10. The method for airfield runway foreign object detection and runway pavement detection of claim 9, wherein the step of calculating the similarity value between each two adjacent zones in the horizontal direction comprises:
according to the formula
Calculating a similarity value between two adjacent regions, where p Ω1，Ω2 Representing the similarity of two regions, Ω 1 and Ω 2, F Ω1 (x, y) represents a pixel value in the region of Ω 1, F Ω2 (x, y) represents a pixel value in the Ω 2 region.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210570956.XA CN115079293A (en) | 2022-05-24 | 2022-05-24 | Equipment and method for detecting foreign matters and road surfaces of airfield runway |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210570956.XA CN115079293A (en) | 2022-05-24 | 2022-05-24 | Equipment and method for detecting foreign matters and road surfaces of airfield runway |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115079293A true CN115079293A (en) | 2022-09-20 |
Family
ID=83248538
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210570956.XA Withdrawn CN115079293A (en) | 2022-05-24 | 2022-05-24 | Equipment and method for detecting foreign matters and road surfaces of airfield runway |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115079293A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115793086A (en) * | 2023-02-07 | 2023-03-14 | 武汉新楚光电科技发展有限公司 | Optical cable laying environment underground cavity judgment method and system based on optical fiber sensing |
| CN115859075A (en) * | 2022-11-01 | 2023-03-28 | 湖北国际物流机场有限公司 | Aircraft type identification device and method for airport area |
-
2022
- 2022-05-24 CN CN202210570956.XA patent/CN115079293A/en not_active Withdrawn
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115859075A (en) * | 2022-11-01 | 2023-03-28 | 湖北国际物流机场有限公司 | Aircraft type identification device and method for airport area |
| CN115859075B (en) * | 2022-11-01 | 2024-03-26 | 湖北国际物流机场有限公司 | Aircraft model identification device and method for airport area |
| CN115793086A (en) * | 2023-02-07 | 2023-03-14 | 武汉新楚光电科技发展有限公司 | Optical cable laying environment underground cavity judgment method and system based on optical fiber sensing |
| CN115793086B (en) * | 2023-02-07 | 2023-06-06 | 武汉新楚光电科技发展有限公司 | Optical cable laying environment underground cavity judging method and system based on optical fiber sensing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115079293A (en) | Equipment and method for detecting foreign matters and road surfaces of airfield runway | |
| CN108710126B (en) | Automatic target detection and eviction method and system | |
| RU2596246C2 (en) | Observation system and method of detecting contamination or damage of aerodrome with foreign objects | |
| US20210324832A1 (en) | Imaging Array for Bird or Bat Detection and Identification | |
| US8547530B2 (en) | System and method to detect foreign objects on a surface | |
| US11380105B2 (en) | Identification and classification of traffic conflicts | |
| EP3537875B1 (en) | System and method for detecting flying animals | |
| US20220189326A1 (en) | Detection and classification of unmanned aerial vehicles | |
| CN105739335A (en) | Airport bird detection early warning and repelling linkage system | |
| DK201300589A1 (en) | Dynamic alarm zones for bird detection systems | |
| CN105787192B (en) | Information processing method and aircraft | |
| GB201219194D0 (en) | Identification and analysis of aircraft landing sites | |
| CN105116418A (en) | Obstacle detection method and apparatus | |
| EP2926296A2 (en) | Systems and methods to classify moving airplanes in airports | |
| CN110271582B (en) | System and method for monitoring safety of cross-road bridge area | |
| US10303941B2 (en) | Locating light sources using aircraft | |
| CN105607075A (en) | Road safety monitoring method and apparatus thereof | |
| Raj et al. | Wild Animals Intrusion Detection for Safe Commuting in Forest Corridors using AI Techniques | |
| Al Yahyaai et al. | LiDAR based remote sensing system for foreign object debris detection (FODD) | |
| CN112198499B (en) | Airport runway foreign matter monitoring system based on thing networking | |
| CN106199750A (en) | Flying Area in Airport activity foreign matter detection system and detection method thereof | |
| EP3724868A1 (en) | Flight vision system and method for presenting images from the surrounding of an airborne vehicle in a flight vision system | |
| KR102317104B1 (en) | Method for improving accuracy of foreign objects detection by setting detection exclusion zone | |
| Jain et al. | A Comparison of Manual and Automotive FOD Detection Systems at Airport Runways | |
| Ashmi et al. | The Modern Approaches for Identifying Foreign Object Debris (FOD) in Aviation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220920 |
|
| WW01 | Invention patent application withdrawn after publication |