CN114092880A - Airport runway large particle foreign matter detection method based on video analysis - Google Patents

Airport runway large particle foreign matter detection method based on video analysis Download PDF

Info

Publication number
CN114092880A
CN114092880A CN202111336962.0A CN202111336962A CN114092880A CN 114092880 A CN114092880 A CN 114092880A CN 202111336962 A CN202111336962 A CN 202111336962A CN 114092880 A CN114092880 A CN 114092880A
Authority
CN
China
Prior art keywords
runway
module
area
threshold
semantic segmentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111336962.0A
Other languages
Chinese (zh)
Inventor
曾杰
黄�俊
方晓强
张路遥
赵国朋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anhui Civio Information And Technology Co ltd
Original Assignee
Anhui Civio Information And Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anhui Civio Information And Technology Co ltd filed Critical Anhui Civio Information And Technology Co ltd
Priority to CN202111336962.0A priority Critical patent/CN114092880A/en
Publication of CN114092880A publication Critical patent/CN114092880A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a video analysis-based method for detecting large-particle foreign matters on an airport runway, which comprises the following steps: 1) setting a detection area of an airport runway on a video monitoring picture, setting an initial pixel area threshold c _ threshold of large-particle foreign matters, and periodically extracting the monitoring picture as an input image; 2) calculating a detection region pixel Area 1; 3) segmenting the runway Area by using a semantic segmentation algorithm, and calculating the Area2 of pixels belonging to the runway; 4) calculating the difference value of the pixel areas of the Area1 and the Area2 to obtain the pixel Area T _ Area of the foreign matter on the runway; 5) if T _ Area is larger than c _ threshold, large-particle foreign matters exist in the runway; otherwise, no large-particle foreign matters exist in the runway. Compared with the prior art, the method has the advantages of low cost, higher recognition rate, better real-time performance and stronger adaptability.

Description

Airport runway large particle foreign matter detection method based on video analysis
Technical Field
The invention relates to the technical field of airport security monitoring, in particular to a method for detecting large-particle foreign matters on an airport runway based on video analysis.
Background
When the airplane slides on the runway, the foreign matters left on the runway can influence the correct running of the airplane, and the large-particle foreign matters in the runway can possibly cause safety accidents in the flying taking-off and landing processes. Therefore, accurately and timely detect and discover the large-particle foreign matter on the runway, effectively avoid the influence of the large-particle foreign matter on flight safety, and ensure the normal running of the airplane.
At present, large-particle foreign matters on airfield runways are usually detected by adopting a millimeter wave radar, but the millimeter wave radar is high in price and strong in specificity, and is not beneficial to popularization and application in small and medium-sized airports.
Therefore, it is necessary to improve the existing radar detection method and provide a new detection method that satisfies the requirements of cost, accuracy, applicability, etc.
Disclosure of Invention
Aiming at the defects or improvement requirements of the existing method, the invention provides a novel detection method of large-particle foreign matters on an airport runway, which realizes the rapid detection of the foreign matters based on a video analysis technology, effectively improves the reliability and accuracy of the detection, and has the advantages of high recognition rate, strong adaptability and low cost.
In order to achieve the above object, the present invention provides a method for detecting large particulate matters on an airport runway based on video analysis, which comprises the following steps:
1) setting a detection area of an airport runway on a video monitoring picture, setting an initial pixel area threshold c _ threshold of large-particle foreign matters, and periodically extracting the monitoring picture as an input image;
2) calculating a detection region pixel Area 1;
3) segmenting the runway Area by using a semantic segmentation algorithm, and calculating the Area2 of pixels belonging to the runway;
4) calculating the difference value of the pixel areas of the Area1 and the Area2 to obtain the pixel Area T _ Area of the foreign matter on the runway;
5) if T _ Area is larger than c _ threshold, large-particle foreign matters exist in the runway; otherwise, no large-particle foreign matters exist in the runway;
6) if large foreign particles exist, setting the pixel values of the divided runway areas to zero, and keeping the pixel values of other areas unchanged;
7) carrying out binarization operation on the processed image, and extracting a large-particle foreign matter detection frame;
8) and outputting a detection result. In a preferred embodiment, the detection result includes whether the runway has large foreign matters, the outline of the large foreign matters is presented, and the position coordinates of the large foreign matters.
Further, the semantic segmentation algorithm comprises an encoding module and a decoding module, wherein the encoding module uses mobilenetv2 to perform feature extraction, a feature map of the decoding module is fused with mobilenetv2 intermediate information, the feature map of the encoding module provides semantic information, the mobilenetv2 intermediate down-sampling provides detail information, and then the semantic segmentation result is obtained through up-sampling.
Preferably, the quadratic rates of the hole convolution in the semantic segmentation algorithm are 2 and 6, respectively.
Wherein, the network model structure of the semantic segmentation algorithm comprises: the method is characterized in that a lightweight network mobilenetv2 is used as a backbone network, the lightweight network mobilenetv2 is provided with an input and two outputs, one output of mobilenetv2 is respectively provided for a first 1 × 1 convolution module, a hole convolution module and a pooling module, the other output of mobilenetv2 is provided for a second 1 × 1 convolution module, the outputs of the first 1 × 1 convolution module, the hole convolution module and the pooling module are all provided for a third 1 × 1 convolution module to obtain intermediate information, the intermediate information is provided for a down-sampling module, the outputs of the second 1 × 1 convolution module and the down-sampling module are respectively provided for a Concat module to perform information fusion operation, the fusion information is provided for the up-sampling module after passing through a3 × 3 convolution module, and a semantic segmentation result is output through the up-sampling module.
Furthermore, the training process of the semantic segmentation network includes the following steps:
a1) collecting video data of an airport runway under different weather, illumination, visibility, environment and other conditions, and carrying out image preprocessing and data annotation on a data set;
a2) and (3) marking the labeled data set as 6: 3: 1, randomly dividing the training set, the testing set and the verification set;
a3) training the semantic segmentation network after the improvement by using a training set and a testing set to obtain an initial segmentation model;
a4) whether the model meets the segmentation effect is verified through a verification set, the termination of model training is judged by adopting an average cross-over ratio mIou, and the calculation formula is as shown in formula 1:
Figure BDA0003350903560000031
wherein, the total classification is 2 classes of runway and background, pijRepresenting the number of pixels, p, originally belonging to class i but predicted to be class jiiExpressed as a true quantity, and pij、pjiExpressed as the number of false positives and false negatives, respectively.
a5) Setting a threshold Iou _ threshold which meets the intersection ratio, if mIou is greater than Iou _ threshold, stopping model training, and outputting an optimal segmentation model; if the stopping condition is not satisfied, the training parameters, the data set and the learning rate are further changed, and the secondary training is carried out by returning to the step a3) by using the adjusted parameters until the stopping condition is satisfied.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the memory implementing the above method when executing the program.
The invention also provides a processor for running the program, wherein the program executes the steps.
Compared with the prior art, the scheme of the invention has the following beneficial effects: 1) the airport monitoring camera equipment is only adopted for detection, professional equipment and special equipment are not needed, and the equipment cost is greatly reduced; 2) the Deep Lab v3 algorithm is improved to realize better semantic segmentation of the airport runway, and the detection precision is further improved. Compared with the prior art, the method has the advantages of low cost, higher recognition rate, better real-time performance and stronger adaptability.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the example serve to explain the principles of the invention and not to limit the invention.
Fig. 1 is a schematic view illustrating a detection process of large particles in an airport runway according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an improved semantic segmentation network provided by an embodiment of the present invention;
fig. 3 is a schematic diagram of a semantic segmentation network training process according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.
The invention introduces machine vision for detection, and utilizes artificial intelligence technology to guarantee reliability and improve accuracy. The foreign matter detection technology based on video analysis is taken as a core, and the method has the advantages of low cost and strong applicability in application.
The flow schematic diagram of the detection method of large-particle foreign matters on the airport runway provided by the invention is shown in fig. 1, and the method comprises the following steps:
9) setting a detection area of an airport runway on a video monitoring picture, and setting an initial pixel area threshold c _ threshold of large-particle foreign matters; then, regularly extracting a monitoring picture as an input image;
10) calculating a detection region pixel Area 1;
11) segmenting the runway Area by using a semantic segmentation algorithm, and calculating the Area2 of pixels belonging to the runway;
12) calculating the difference value of the pixel areas of the Area1 and the Area2 to obtain the pixel Area T _ Area of the foreign matter on the runway;
13) if T _ Area is larger than c _ threshold, large-particle foreign matters exist in the runway; otherwise, no large-particle foreign matters exist in the runway;
14) if large foreign particles exist, setting the pixel values of the divided runway areas to zero, and keeping the pixel values of other areas unchanged;
15) carrying out binarization operation on the processed image, and extracting a large-particle foreign matter detection frame;
16) and outputting a detection result. In a preferred embodiment, the detection result includes whether the runway has large foreign matters, the outline of the large foreign matters is presented, and the position coordinates of the large foreign matters.
In order to improve the detection accuracy, the invention further improves the Deep Lab v3 algorithm to realize better semantic segmentation of the airport runway. Deep Lab v3 is a semantic segmentation algorithm using a spatial pyramid module and a coding/decoding structure, and richer context information is obtained through pooling operations under different resolutions. The invention specifically improves the Deep Lab v3 semantic segmentation algorithm as follows:
1) by analyzing the data set samples, the influence of the computational complexity on the real-time performance is considered, and the lightweight network mobilenetv2 is adopted as a backbone network to train the segmentation model. The encoding module uses mobilenetv2 for feature extraction, and the blocks are used to extract multi-scale information of the fused image. And fusing the feature graph of the decoding module with the mobilenetv2 intermediate information, providing semantic information by the feature graph of the encoding module, providing detail information by the mobilenetv2 intermediate down-sampling, and up-sampling to obtain a semantic segmentation result.
2) By adopting the cavity convolution with the secondary rates of 2 and 6 respectively, compared with a deep Lab v3 network, the method reduces a layer of cavity convolution calculation, enlarges the receptive field to a certain extent, not only can keep an internal data structure, but also can avoid the loss of characteristic information caused by downsampling, thereby obtaining a characteristic diagram with larger information content. Also, the number of parameters and the amount of computation are reduced by the depth separable convolution.
In conjunction with the improved semantic segmentation network structure diagram shown in fig. 2, the network structure of the semantic segmentation model used in the present invention includes: the method is characterized in that a lightweight network mobilenetv2 is used as a backbone network, the lightweight network mobilenetv2 is provided with an input and two outputs, one output of mobilenetv2 is respectively provided for a first 1 × 1 convolution module, a hole convolution module and a pooling module, the other output of mobilenetv2 is provided for a second 1 × 1 convolution module, the outputs of the first 1 × 1 convolution module, the hole convolution module and the pooling module are all provided for a third 1 × 1 convolution module to obtain intermediate information, the intermediate information is provided for a down-sampling module, the outputs of the second 1 × 1 convolution module and the down-sampling module are respectively provided for a Concat module to perform information fusion operation, the fusion information is provided for the up-sampling module after passing through a3 × 3 convolution module, and a semantic segmentation result is output through the up-sampling module.
Further, a training process of the semantic segmentation network of the present invention is shown in fig. 3, and specifically includes the following steps:
a1) collecting video data of an airport runway under different weather, illumination, visibility, environment and other conditions, and carrying out image preprocessing and data annotation on a data set;
a2) and (3) marking the labeled data set as 6: 3: 1, randomly dividing the training set, the testing set and the verification set;
a3) training the semantic segmentation network after the improvement by using a training set and a testing set to obtain an initial segmentation model;
a4) whether the model meets the segmentation effect is verified through a verification set, the termination of model training is judged by adopting an average cross-over ratio mIou, and the calculation formula is as shown in formula 1:
Figure BDA0003350903560000051
wherein, the total classification is 2 classes of runway and background, pijRepresenting the number of pixels, p, originally belonging to class i but predicted to be class jiiExpressed as a true quantity, and pij、pjiExpressed as the number of false positives and false negatives, respectively.
a5) Setting a threshold Iou _ threshold which meets the intersection ratio, if mIou is greater than Iou _ threshold, stopping model training, and outputting an optimal segmentation model; if the stopping condition is not satisfied, the training parameters, the data set and the learning rate are further changed, and the secondary training is carried out by returning to the step a3) by using the adjusted parameters until the stopping condition is satisfied.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the memory implementing the above method when executing the program.
The invention also provides a processor for running the program, wherein the program executes the steps.
In the technical scheme of the invention, the existing monitoring camera of the airport is used for collecting training pictures and executing foreign body detection, so that the equipment cost is saved; and due to the improvement of the semantic segmentation network, the detection precision and the target detection capability are more excellent (about 30 percent of improvement). Compared with the prior art, the method has the advantages of low cost, higher recognition rate, better real-time performance and stronger adaptability.
Although the embodiments of the present invention have been described above, the above description is only for the convenience of understanding the present invention, and is not intended to limit the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the scope of the present invention should be determined by the following claims.

Claims (10)

1. A video analysis-based method for detecting large-particle foreign matters on an airport runway is characterized by comprising the following steps:
1) setting a detection area of an airport runway on a video monitoring picture, setting an initial pixel area threshold c _ threshold of large-particle foreign matters, and periodically extracting the monitoring picture as an input image;
2) calculating a detection region pixel Area 1;
3) segmenting the runway Area by using a semantic segmentation algorithm, and calculating the Area2 of pixels belonging to the runway;
4) calculating the difference value of the pixel areas of the Area1 and the Area2 to obtain the pixel Area T _ Area of the foreign matter on the runway;
5) if T _ Area is larger than c _ threshold, large-particle foreign matters exist in the runway; otherwise, no large-particle foreign matters exist in the runway.
2. The method according to claim 1, characterized in that the method further comprises the steps of:
6) if large foreign particles exist, setting the pixel values of the divided runway areas to zero, and keeping the pixel values of other areas unchanged;
7) carrying out binarization operation on the processed image, and extracting a large-particle foreign matter detection frame;
8) and outputting a detection result.
3. The method of claim 2, wherein: the detection result comprises whether the runway has large-particle foreign bodies and the position coordinates of the large-particle foreign bodies.
4. The method according to any one of claims 1 to 3, wherein the semantic segmentation algorithm comprises an encoding module and a decoding module, the encoding module uses mobilenetv2 to perform feature extraction, a decoding module feature map is fused with mobilenetv2 intermediate information, a feature map of the encoding module provides semantic information, mobilenetv2 intermediate down-sampling provides detail information, and up-sampling obtains a semantic segmentation result.
5. The method according to claim 4, wherein the quadratic rates of the hole convolution in the semantic segmentation algorithm are 2 and 6, respectively.
6. The method according to any one of claims 1-3, wherein the network model structure of the semantic segmentation algorithm comprises: the method is characterized in that a lightweight network mobilenetv2 is used as a backbone network, the lightweight network mobilenetv2 is provided with an input and two outputs, one output of mobilenetv2 is respectively provided for a first 1 × 1 convolution module, a hole convolution module and a pooling module, the other output of mobilenetv2 is provided for a second 1 × 1 convolution module, the outputs of the first 1 × 1 convolution module, the hole convolution module and the pooling module are all provided for a third 1 × 1 convolution module to obtain intermediate information, the intermediate information is provided for a down-sampling module, the outputs of the second 1 × 1 convolution module and the down-sampling module are respectively provided for a Concat module to perform information fusion operation, the fusion information is provided for the up-sampling module after passing through a3 × 3 convolution module, and a semantic segmentation result is output through the up-sampling module.
7. The method of claim 6, wherein the training process of the semantic segmentation network comprises the following steps:
a1) collecting video data of an airport runway under different weather, illumination, visibility, environment and other conditions, and carrying out image preprocessing and data annotation on a data set;
a2) and (3) marking the labeled data set as 6: 3: 1, randomly dividing the training set, the testing set and the verification set;
a3) training the semantic segmentation network after the improvement by using a training set and a testing set to obtain an initial segmentation model;
a4) whether the model meets the segmentation effect is verified through a verification set, the termination of model training is judged by adopting an average cross-over ratio mIou, and the calculation formula is as shown in formula 1:
Figure FDA0003350903550000021
wherein, the total classification is 2 classes of runway and background, pijRepresenting the number of pixels, p, originally belonging to class i but predicted to be class jiiExpressed as a true quantity, and pij、pjiExpressed as the number of false positives and false negatives, respectively;
a5) setting a threshold Iou _ threshold which meets the intersection ratio, if mIou is greater than Iou _ threshold, stopping model training, and outputting an optimal segmentation model; if the stopping condition is not satisfied, the training parameters, the data set and the learning rate are further changed, and the secondary training is carried out by returning to the step a3) by using the adjusted parameters until the stopping condition is satisfied.
8. The method according to claim 6 or 7, wherein the quadratic rates of the hole convolutions are 2 and 6, respectively.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the memory when executing the program implements the method of any of claims 1-8.
10. A processor for executing a program, wherein the program executes to perform the method of any one of claims 1 to 8.
CN202111336962.0A 2021-11-12 2021-11-12 Airport runway large particle foreign matter detection method based on video analysis Pending CN114092880A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111336962.0A CN114092880A (en) 2021-11-12 2021-11-12 Airport runway large particle foreign matter detection method based on video analysis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111336962.0A CN114092880A (en) 2021-11-12 2021-11-12 Airport runway large particle foreign matter detection method based on video analysis

Publications (1)

Publication Number Publication Date
CN114092880A true CN114092880A (en) 2022-02-25

Family

ID=80300122

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111336962.0A Pending CN114092880A (en) 2021-11-12 2021-11-12 Airport runway large particle foreign matter detection method based on video analysis

Country Status (1)

Country Link
CN (1) CN114092880A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998836A (en) * 2022-06-13 2022-09-02 北京拙河科技有限公司 Panoramic monitoring method and device for airport runway

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114998836A (en) * 2022-06-13 2022-09-02 北京拙河科技有限公司 Panoramic monitoring method and device for airport runway

Similar Documents

Publication Publication Date Title
Chen et al. Vehicle detection in high-resolution aerial images via sparse representation and superpixels
CN113723377B (en) Traffic sign detection method based on LD-SSD network
WO2020103893A1 (en) Lane line property detection method, device, electronic apparatus, and readable storage medium
Zhang et al. Study on traffic sign recognition by optimized Lenet-5 algorithm
AU2020102039A4 (en) A high-precision multi-targets visual detection method in automatic driving scene
Li et al. Parking slot detection on around-view images using DCNN
Saleh et al. Traffic signs recognition and distance estimation using a monocular camera
CN114049572A (en) Detection method for identifying small target
CN113052106A (en) Airplane take-off and landing runway identification method based on PSPNet network
Li et al. Pixel‐Level Recognition of Pavement Distresses Based on U‐Net
CN110991374B (en) Fingerprint singular point detection method based on RCNN
Zhang et al. Detecting small Chinese traffic signs via improved YOLOv3 method
Bousarhane et al. Road signs recognition: state-of-the-art and perspectives
Khan et al. A novel deep learning based anpr pipeline for vehicle access control
CN113158954B (en) Automatic detection method for zebra crossing region based on AI technology in traffic offsite
CN114092880A (en) Airport runway large particle foreign matter detection method based on video analysis
Hua et al. Traffic lights detection and recognition method using deep learning with improved YOLOv5 for autonomous vehicle in ROS2
CN111832463A (en) Deep learning-based traffic sign detection method
Huang et al. Nighttime vehicle detection based on direction attention network and bayes corner localization
Wang et al. Vehicle key information detection algorithm based on improved SSD
Wu et al. Research on asphalt pavement disease detection based on improved YOLOv5s
Merugu et al. Multi lane detection, curve fitting and lane type classification
Gao et al. Semantic segmentation of street view image based on fully convolutional neural networks
Al Khafaji et al. Traffic Signs Detection and Recognition Using A combination of YOLO and CNN
CN114550139A (en) Lane line detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination