CN108184096B - Panoramic monitoring device, system and method for airport running and sliding area - Google Patents
Panoramic monitoring device, system and method for airport running and sliding area Download PDFInfo
- Publication number
- CN108184096B CN108184096B CN201810016035.2A CN201810016035A CN108184096B CN 108184096 B CN108184096 B CN 108184096B CN 201810016035 A CN201810016035 A CN 201810016035A CN 108184096 B CN108184096 B CN 108184096B
- Authority
- CN
- China
- Prior art keywords
- module
- image
- monitoring
- airport
- sliding area
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Abstract
The invention relates to a panoramic monitoring device, a panoramic monitoring system and a panoramic monitoring method for an airport running and sliding area.A plurality of monitoring devices acquire space scene light of the airport running and sliding area through camera lenses, and the light is converted into charges through a CCD (charge coupled device) sensor and is converted into digital signals through an analog-to-digital converter chip; the image acquisition module acquires real-time airport running and sliding area video image data code streams; the image preprocessing module performs feature extraction and segmentation processing on an input video image data code stream, then performs matching superposition on a plurality of acquired images through the image registration module, and then performs splicing fusion on the plurality of acquired images through the image fusion module to form an airport sliding area panoramic image; the object identification module identifies an object with a motion state on the panoramic image of the airport running and sliding area, and the motion object is tracked through the identification and tracking module. The invention realizes panoramic monitoring of the airport running and sliding area, can carry out real-time identification, tracking and early warning, and improves the safety of the airport running and sliding area.
Description
Technical Field
The invention relates to the technical field of panoramic monitoring, in particular to a panoramic monitoring device, a panoramic monitoring system and a panoramic monitoring method for an airport running and sliding area.
Background
Taxiways are defined passages in land airports through which aircraft can taxi on the ground. The taxiway system mainly comprises a main taxiway, an in-out taxiway, a taxiway of an airplane stand, a apron taxiway, an auxiliary taxiway, a taxiway shoulder and a taxiway belt. The airport running area is an area including a taxiway and around the taxiway, the airport running area is related to safe take-off and landing of airplanes, and monitoring of the airport running area is of great significance for guaranteeing safety of lives and properties of passengers.
The traditional video monitoring generally has narrow visual field range and can monitor only limited scenes. Under the condition that a large scene needs to be monitored, a plurality of cameras are often used for monitoring a certain specific area, a plurality of monitoring pictures are provided through the plurality of cameras, the attention of monitoring personnel is often dispersed by the solution, accidental events cannot be found and tracked in time, and safety accidents are easily caused.
Disclosure of Invention
The invention aims to provide a panoramic monitoring device, a panoramic monitoring system and a panoramic monitoring method for an airport running and sliding area.
In order to achieve the purpose, the technical scheme of the invention is as follows: a panoramic monitoring device for an airport running and sliding area comprises a DSP (digital signal processor) and an ARM (advanced RISC machines) processor, and further comprises a camera lens, a CCD (charge coupled device) sensor, a video acquisition module, an image preprocessing module, an image registration module, an image fusion module, an object identification module, an identification and tracking module and an out-of-range monitoring module; the camera lens is connected with the CCD sensor and used for acquiring a space scene of an airport running and sliding area; the CCD sensor is used for converting light rays into charges and converting the charges into digital signals through an analog-to-digital converter chip; the video acquisition module is used for acquiring a real-time video image data code stream transmitted by the CCD sensor; the image preprocessing module is used for performing feature extraction and segmentation processing on the input video image; the image registration module is used for matching and superposing a plurality of acquired images; the image fusion module is used for splicing and fusing the acquired images; the object identification module is connected with the ARM processor and the image fusion module, and is used for identifying an object with a motion state on the airport running and sliding area image fused by the image fusion module; the identification tracking module is used for tracking an object with a motion state in an airport running and sliding area image; the boundary crossing monitoring module is used for monitoring boundary crossing of an object with a motion state in an airport running and sliding area image.
The panoramic monitoring device for the airport running and sliding area further comprises an emergency stop detection module, wherein the emergency stop detection module is connected with the ARM processor, and is used for performing emergency stop detection on the airplane with a motion state on the airport running and sliding area image. The emergency stop detection module extracts characteristic parameters of the airplane with a motion state in the airport running and sliding area, and when the airplane is in an emergency stop state, the characteristic parameters of the airplane on adjacent images are the same, so that the airplane is judged to have an emergency stop accident.
The panoramic monitoring device for the airport running and sliding area further comprises an early warning module, wherein the early warning module is connected with the ARM processor, and is used for carrying out safety early warning when an object with a moving state in an airport running and sliding area image crosses a safety limit or an airplane is in an emergency stop state. The safety limit of the airport running-sliding area is preset inside the parameters of the monitoring device, the image characteristic parameters in the safety limit are consistent when no foreign object intrudes, and when the foreign object intrudes, the image characteristic parameters in the safety limit of the running-sliding area change, and then the safety early warning is carried out by judging that the foreign object intrudes.
The airport sliding area panoramic monitoring device is provided with a Flash chip, a DDR (double data rate) chip and an Ethernet interface, wherein the Flash chip is connected with the ARM processor, the Flash chip is used for storing airport sliding area image data, the DDR chip is connected with the ARM processor, and the DDR chip is used for realizing double-rate synchronous dynamic random storage of the airport sliding area image data; the Ethernet interface is used for connecting the monitoring device to the Internet for network data transmission. Flash chips are non-volatile memories, and blocks of memory cells, called blocks, can be erased and reprogrammed. The frequency of the DDR chip memory can be expressed by a working frequency mode and an equivalent frequency mode, the working frequency is the actual working frequency of the memory particles, and the DDR chip memory can transmit data at the rising edge and the falling edge of a pulse, so that the equivalent frequency of the data transmission is twice of the working frequency. The DDR chip can read/write data at 4 times the speed of the external bus per clock and can run at 4 times the speed of the internal control bus. The data storage requirement of the monitoring device is met.
The airport running-sliding area panoramic monitoring device is provided with the wireless transmission module, the wireless transmission module is connected with the ARM processor, the wireless transmission module adopts an LoRa module, a Bluetooth module or an NB-IoT module, and the wireless transmission module is used for wirelessly transmitting airport running-sliding area image data monitored by the monitoring device. The LoRa module can adopt an SX1278 radio frequency chip of Semtech company, the transmission distance can reach 8km furthest, and the working frequency band is 411-441 MHz; the module supports 4 working modes and air awakening, power consumption can be reduced to the maximum extent, the module has 7 pins, communication with external equipment is achieved through a serial port, the pins M0 and M1 are used for setting the working modes of the module, and the AUX is used for indicating the working state of the module. The NB-IoT module can adopt SIM7000C of SIMCom, the SIM7000C is a support NB-IoT module developed based on a high-pass MDM9206 platform, and the requirements of the panoramic monitoring device in the airport runway area can be met by adopting SMT packaging.
The invention also provides a panoramic monitoring system for the airport running and sliding area, which comprises the monitoring device, a display host, a remote server and a mobile terminal; the number of the monitoring devices is at least 2, the monitoring devices are arranged around an airport running and sliding area and are provided with acquisition ends and monitoring ends, the acquisition ends and the monitoring ends are connected, and each acquisition end comprises a DSP (digital signal processor), a camera lens, a CCD (charge coupled device) sensor, a video acquisition module, an image preprocessing module, an image registration module and an image fusion module; the monitoring end comprises an ARM processor, an object identification module, an identification tracking module and an out-of-range monitoring module; the monitoring end establishes a connection relation with the display host through an Ethernet interface, and the display host acquires and displays panoramic video image data of the airport running and sliding area transmitted by the monitoring end; the remote server establishes a connection relation with the monitoring terminal and the mobile terminal, and acquires image data of the airport running area of the monitoring device and transmits the image data to the mobile terminal.
According to the panoramic monitoring system for the airport running and sliding area, the monitoring end establishes a connection relation with the display host through the wireless transmission module, and the wireless transmission module adopts an LoRa module, a Bluetooth module or an NB-IoT module.
According to the panoramic monitoring system for the airport running and sliding area, the mobile terminal and the monitoring device are connected through the Bluetooth module, and the mobile terminal directly obtains panoramic monitoring data of the airport running and sliding area from the monitoring device.
The invention also provides a panoramic monitoring method for the airport running and sliding area, which is realized by adopting the monitoring device and the monitoring system, and comprises the following steps:
the method comprises the following steps: the monitoring devices acquire space scene light rays of an airport running and sliding area through the camera lens, the light rays are converted into charges through the CCD sensor and are converted into digital signals through the analog-to-digital converter chip;
step two: the CCD sensor transmits a digital signal to the image acquisition module, and the image acquisition module acquires a real-time airport running and sliding area video image data code stream;
step three: the image preprocessing module performs feature extraction and segmentation processing on an input video image data code stream, then performs matching and superposition on a plurality of acquired images through the image registration module, and then performs splicing and fusion on the plurality of acquired images through the image fusion module to form a panoramic image of the airport running and sliding area;
step four: the object identification module identifies an object with a motion state on a panoramic image of the airport running and sliding area, and the motion object is tracked through the identification and tracking module;
step five: the boundary crossing monitoring module carries out boundary crossing monitoring on objects with motion states in the airport running and sliding area, and when the moving objects cross the safety boundary of the airport running and sliding area, the early warning module carries out safety early warning.
In the fourth step of the panoramic monitoring method for the airport running and sliding area, the object identification module identifies that the moving object is an airplane, and when the airplane is in an emergency stop state after moving, the early warning module carries out safety early warning.
The invention has the following advantages: the method adopts a common camera lens, performs characteristic extraction and segmentation processing on an input video image data code stream through an image processing algorithm, matches and superposes a plurality of acquired images, and splices and fuses the images to form a panoramic image of the airport sliding area, thereby realizing panoramic monitoring of the airport sliding area, and performing real-time identification, tracking and early warning on a moving object in the airport sliding area, improving the safety of the airport sliding area and having low monitoring cost.
Drawings
FIG. 1 is a schematic view of a panoramic monitoring device for airport running and sliding areas;
FIG. 2 is a schematic view of a panoramic monitoring system for airport running and sliding areas;
fig. 3 is a flow chart of a panoramic monitoring method for airport running and sliding areas.
Detailed Description
The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
As shown in fig. 1, the panoramic monitoring device for airport running and sliding areas comprises a DSP processor 1 and an ARM processor 2, and further comprises a camera lens 3, a CCD sensor 4, a video acquisition module 5, an image preprocessing module 6, an image registration module 7, an image fusion module 8, an object recognition module 9, an identification and tracking module 10, and an out-of-range monitoring module 11; the camera lens 3 is connected with the CCD sensor 4, and the camera lens 3 is used for acquiring a space scene of an airport running and sliding area; the CCD sensor 4 is connected with the video acquisition module 5, and the CCD sensor 4 is used for converting light rays into charges and converting the charges into digital signals through an analog-to-digital converter chip; the video acquisition module 5 establishes a connection relation with the DSP 1, and the video acquisition module 5 is used for acquiring a real-time video image data code stream transmitted by the CCD sensor 4; the image preprocessing module 6 establishes a connection relation with the video acquisition module 5, and the image preprocessing module 6 is used for performing feature extraction and segmentation processing on an input video image; the image registration module 7 establishes a connection relation with the image preprocessing module 6, and the image registration module 7 is used for matching and superposing a plurality of acquired images; the image fusion module 8 establishes a connection relation with the image registration module 7, and the image fusion module 8 is used for splicing and fusing the acquired images; the object identification module 9 establishes a connection relationship with the ARM processor 2 and the image fusion module 8, and the object identification module 9 is used for identifying an object with a motion state on the airport running and sliding area image fused by the image fusion module 8; the identification tracking module 10 establishes a connection relation with the object identification module 9, and the identification tracking module 10 is used for tracking an object with a motion state in an airport running area image; the border crossing monitoring module 11 establishes a connection relation with the identification tracking module 10, and the border crossing monitoring module 11 is used for monitoring border crossing of an object with a motion state in an airport running and sliding area image.
In an embodiment of the panoramic monitoring device for the airport running and sliding area, the monitoring device further comprises an emergency stop detection module 12, the emergency stop detection module 12 establishes a connection relationship with the ARM processor 2, and the emergency stop detection module 12 is used for performing emergency stop detection on an airplane with a motion state on an airport running and sliding area image. The emergency stop detection module 12 extracts the characteristic parameters of the airplane having a motion state in the airport running and sliding area, and when the airplane has an emergency stop state, the characteristic parameters of the airplane on the adjacent images are the same, so that the airplane is judged to have an emergency stop accident.
In an embodiment of the panoramic monitoring device for the airport running and sliding area, the monitoring device further comprises an early warning module 13, the early warning module 13 establishes a connection relationship with the ARM processor 2, and the early warning module 13 is used for performing safety early warning when an object with a moving state in an airport running and sliding area image crosses a safety limit or in an airplane emergency stop state. The safety limit of the airport running-sliding area is preset with internal parameters of the monitoring device, when no foreign object intrudes, image characteristic parameters in the safety limit are consistent, when the foreign object intrudes, the image characteristic parameters in the safety limit of the running-sliding area change, and then the safety early warning is carried out by judging that the foreign object intrudes.
In an embodiment of the panoramic monitoring device for the airport sliding area, the monitoring device is provided with a Flash chip 14, a DDR chip 15 and an Ethernet interface 16, the Flash chip 14 is connected with the ARM processor 2, the Flash chip 14 is used for storing image data of the airport sliding area, the DDR chip 15 is connected with the ARM processor 2, and the DDR chip 15 is used for realizing double-rate synchronous dynamic random storage of the image data of the airport sliding area; the Ethernet interface 16 establishes a connection relationship with the ARM processor 2, and the Ethernet interface 16 is used for connecting the monitoring device to the Internet for network data transmission. The Flash chip 14 is a non-volatile memory that can erase and reprogram blocks of memory cells called blocks. The frequency of the memory of the DDR chip 15 can be expressed by a working frequency and an equivalent frequency, the working frequency is the actual working frequency of the memory particles, and the DDR memory can transmit data at the rising edge and the falling edge of a pulse, so that the equivalent frequency of the data transmission is twice of the working frequency. The DDR chip 15 can read/write data at 4 times the speed of the external bus per clock and can run at 4 times the speed of the internal control bus. The data storage requirement of the monitoring device is met.
In an embodiment of the panoramic airport sliding area monitoring device, the monitoring device is provided with a wireless transmission module 17, the wireless transmission module 17 establishes a connection relationship with the ARM processor 2, the wireless transmission module 17 adopts an LoRa module, a bluetooth module or an NB-IoT module, and the wireless transmission module 17 is used for wirelessly transmitting airport sliding area image data monitored by the monitoring device. The LoRa module can adopt an SX1278 radio frequency chip of Semtech company, the transmission distance can reach 8km furthest, and the working frequency band is 411-441 MHz; the module supports 4 working modes and air awakening, power consumption can be reduced to the maximum extent, the module has 7 pins, communication with external equipment is achieved through a serial port, the pins M0 and M1 are used for setting the working modes of the module, and the AUX is used for indicating the working state of the module. The NB-IoT module can adopt SIM7000C of SIMCom, the SIM7000C is a support NB-IoT module developed based on a high-pass MDM9206 platform, and the requirements of the panoramic monitoring device in the airport runway area can be met by adopting SMT packaging.
As shown in fig. 2, the present invention further provides a panoramic monitoring system for airport running and sliding area, wherein the monitoring system comprises the monitoring device, the monitoring system further comprises a display host 18, a remote server 19 and a mobile terminal 20; the number of the monitoring devices is at least 2, the monitoring devices are arranged around an airport running and sliding area and are provided with acquisition ends and monitoring ends, the acquisition ends and the monitoring ends are connected, and each acquisition end comprises a DSP (digital signal processor) 1, a camera lens 3, a CCD (charge coupled device) sensor 4, a video acquisition module 5, an image preprocessing module 6, an image registration module 7 and an image fusion module 8; the monitoring end comprises an ARM processor 2, an object identification module 9, an identification tracking module 10 and an out-of-range monitoring module 11; the monitoring end establishes a connection relation with the display host 18 through an Ethernet interface 16, and the display host 18 acquires and displays the panoramic video image data of the airport running and sliding area transmitted by the monitoring end; the remote server 19 establishes a connection relationship with the monitoring terminal and the mobile terminal 20, and the remote server 19 acquires the image data of the airport running area of the monitoring device and transmits the image data to the mobile terminal 20.
In an embodiment of the panoramic monitoring system for airport running and sliding areas, the monitoring end establishes a connection relationship with the display host 18 through a wireless transmission module 17, and the wireless transmission module 17 adopts an LoRa module, a bluetooth module or an NB-IoT module.
In an embodiment of the panoramic monitoring system for airport running and sliding areas, the mobile terminal 20 establishes a connection relationship with the monitoring device through a bluetooth module, and the mobile terminal 20 directly obtains panoramic monitoring data of the airport running and sliding areas from the monitoring device.
Referring to fig. 3, the invention further provides a panoramic monitoring method for airport running and sliding areas, the method is implemented by the monitoring system by using the monitoring device, and the monitoring method comprises the following steps:
s1: the monitoring devices acquire space scene light rays of an airport running and sliding area through the camera lens 3, the light rays are converted into electric charges through the CCD sensor 4 and are converted into digital signals through the analog-to-digital converter chip;
s2: the CCD sensor 4 transmits a digital signal to the image acquisition module, and the image acquisition module acquires a real-time airport running and sliding area video image data code stream;
s3: the image preprocessing module 6 performs feature extraction and segmentation processing on an input video image data code stream, then performs matching and superposition on a plurality of acquired images through the image registration module 7, and then performs splicing and fusion on the plurality of acquired images through the image fusion module 8 to form an airport sliding area panoramic image;
s4: the object identification module 9 identifies an object with a motion state on the panoramic image of the airport running and sliding area, and the identification and tracking module 10 tracks the moving object;
s5: the boundary crossing monitoring module 11 carries out boundary crossing monitoring on objects with motion states in the airport running and sliding area, and when the moving objects cross the safety boundary of the airport running and sliding area, the early warning module 13 carries out safety early warning.
In the panoramic monitoring method S4 for airport running and sliding area, the object recognition module 9 recognizes that the moving object is an airplane, and when the airplane is in an emergency stop state after moving, the early warning module 13 performs a safety early warning.
The invention adopts a common camera lens, and performs characteristic extraction and segmentation processing on an input video image data code stream through an image processing algorithm, wherein the characteristic extraction and segmentation processing can be realized by adopting a combined segmentation algorithm based on superpixels and SVM. The method comprises the steps of carrying out matching superposition on a plurality of acquired images, wherein the image matching superposition adopts a color image registration algorithm based on an improved SURF operator, the SURF characteristic is a color image registration algorithm with unchanged scale and rotation, the steps are basically the same as those of an SIFT algorithm, but the operation speed is increased by 3-5 times, and the SURF algorithm uses an integral image and a box filter and mainly comprises three parts of extracting characteristic points, generating characteristic descriptors and matching the characteristics. And feature point extraction is carried out by carrying out feature point detection on a scale space constructed based on a Gaussian pyramid, and feature points are extracted by a Hessian matrix. In order to ensure the rotation invariance of SURF features in the generated feature descriptors, a main direction is determined for the extracted feature points. Calculating Haar wavelet responses in x and y directions by taking the characteristic point as a circle center and 6s (s is a value of a scale where the characteristic point is located) as a radius, wherein the size of a Haar wavelet template is 4s multiplied by 4 s; gaussian weighting is given to the Haar responses so that responses closer to the feature points are larger; scanning a circle in a pi/3 sector range by taking the characteristic point as a center, and performing accumulated superposition on Haar wavelet responses in all angles to form a new vector; the principal direction is the direction of the maximum of the cumulative overlap-add. After the main direction is determined, a 20s × 20s square window region is constructed with the feature point as the center, the region is divided into 4 × 4 sub-regions, gaussian weight coefficients are given, the sum of Haar wavelet responses dx in the horizontal direction, the sum of Haar wavelet responses dy in the vertical direction, and the sum of absolute values of the Haar wavelet responses are calculated for each sampling point of the region, a four-dimensional vector (Σ dx, Σ dy, Σ dx, Σ dy) is formed in each region, and the descriptor of the feature point is (4 × 4) × 4 ═ 64-dimensional vector Vs (i1, i2, …, i 64). After the feature points of the two images are extracted, feature matching can be performed according to the similarity measurement between the feature points. The euclidean distance between feature points is chosen as the similarity measure. The invention splices and fuses a plurality of images to form the panoramic image of the airport running and sliding area, realizes panoramic monitoring of the airport running and sliding area, can carry out real-time identification, tracking and early warning on moving objects in the airport running and sliding area, improves the safety of the airport running and sliding area and has low monitoring cost.
Although the invention has been described in detail above with reference to a general description and specific examples, it will be apparent to one skilled in the art that modifications or improvements may be made thereto based on the invention. Accordingly, such modifications and improvements are intended to be within the scope of the invention as claimed.
Claims (9)
1. The utility model provides an airport running and sliding district panorama monitoring device, monitoring device includes DSP treater and ARM treater, its characterized in that: the monitoring device also comprises a camera lens, a CCD sensor, a video acquisition module, an image preprocessing module, an image registration module, an image fusion module, an object identification module, an identification tracking module and an out-of-range monitoring module; the camera lens is connected with the CCD sensor and used for acquiring a space scene of an airport running and sliding area; the CCD sensor is used for converting light rays into charges and converting the charges into digital signals through an analog-to-digital converter chip; the video acquisition module is used for acquiring a real-time video image data code stream transmitted by the CCD sensor; the image preprocessing module is used for performing feature extraction and segmentation processing on the input video image; the image registration module is used for matching and superposing a plurality of acquired images; the image fusion module is used for splicing and fusing the acquired images; the object identification module is connected with the ARM processor and the image fusion module, and is used for identifying an object with a motion state on the airport running and sliding area image fused by the image fusion module; the identification tracking module is used for tracking an object with a motion state in an airport running and sliding area image; the boundary crossing monitoring module is used for monitoring boundary crossing of an object with a motion state in an airport running and sliding area image;
matching and superposing the acquired images, wherein the image matching and superposition adopts a color image registration algorithm based on an improved SURF operator; the SURF algorithm uses an integral image and a box filter and comprises three parts of extracting feature points, generating feature descriptors and matching features; feature point extraction is carried out by carrying out feature point detection through a scale space constructed based on a Gaussian pyramid, and feature points are extracted through a Hessian matrix; in order to ensure the rotation invariance of SURF characteristics in the generated characteristic descriptors, determining a main direction for the extracted characteristic points, calculating Haar wavelet responses in x and y directions by taking the characteristic points as circle centers and 6s as radii, and taking the size of a Haar wavelet template as 4s multiplied by 4s, wherein s is a value of the scale of the characteristic points; after extracting the feature points of the two images, performing feature matching according to the similarity measurement between the feature points, and selecting the Euclidean distance between the feature points as the similarity measurement;
the monitoring device also comprises an emergency stop detection module, the emergency stop detection module is connected with the ARM processor, and the emergency stop detection module is used for performing emergency stop detection on the airplane with a motion state on the image of the airport running and sliding area; the emergency stop detection module 12 extracts the characteristic parameters of the airplane having a motion state in the airport running and sliding area, and when the airplane has an emergency stop state, the characteristic parameters of the airplane on the adjacent images are the same, so that the airplane is judged to have an emergency stop accident.
2. The panoramic airport running area monitoring device of claim 1, wherein: the monitoring device further comprises an early warning module, the early warning module is connected with the ARM processor, and the early warning module is used for carrying out safety early warning when an object with a motion state in an airport running and sliding area image crosses a safety limit or in an airplane emergency stop state.
3. The panoramic airport running area monitoring device of claim 1, wherein: the monitoring device is provided with a Flash chip, a DDR chip and an Ethernet interface, the Flash chip is connected with the ARM processor, the Flash chip is used for storing airport sliding area image data, the DDR chip is connected with the ARM processor, and the DDR chip is used for realizing double-rate synchronous dynamic random storage of the airport sliding area image data; the Ethernet interface is used for connecting the monitoring device to the Internet for network data transmission.
4. The panoramic airport running area monitoring device of claim 1, wherein: the monitoring device is provided with a wireless transmission module, the wireless transmission module is connected with the ARM processor, the wireless transmission module adopts a LoRa module, a Bluetooth module or an NB-IoT module, and the wireless transmission module is used for wirelessly transmitting airport running and sliding area image data monitored by the monitoring device.
5. An airport running area panoramic monitoring system, comprising the monitoring device of any one of claims 1 to 4, wherein: the monitoring system also comprises a display host, a remote server and a mobile terminal; the number of the monitoring devices is at least 2, the monitoring devices are arranged around an airport running and sliding area and are provided with acquisition ends and monitoring ends, the acquisition ends and the monitoring ends are connected, and each acquisition end comprises a DSP (digital signal processor), a camera lens, a CCD (charge coupled device) sensor, a video acquisition module, an image preprocessing module, an image registration module and an image fusion module; the monitoring end comprises an ARM processor, an object identification module, an identification tracking module and an out-of-range monitoring module; the monitoring end establishes a connection relation with the display host through an Ethernet interface, and the display host acquires and displays panoramic video image data of the airport running and sliding area transmitted by the monitoring end; the remote server establishes a connection relation with the monitoring terminal and the mobile terminal, and acquires image data of the airport running area of the monitoring device and transmits the image data to the mobile terminal.
6. The panoramic airport running area monitoring system of claim 5, wherein: the monitoring end establishes a connection relation with the display host through a wireless transmission module, and the wireless transmission module adopts an LoRa module, a Bluetooth module or an NB-IoT module.
7. The panoramic airport running area monitoring system of claim 5, wherein: the mobile terminal and the monitoring device establish a connection relationship through a Bluetooth module, and the mobile terminal directly obtains panoramic monitoring data of the airport running and sliding area from the monitoring device.
8. A panoramic airport running area monitoring method using the monitoring device according to any one of claims 1 to 4, and implemented by the monitoring system according to any one of claims 6 to 7, wherein: the monitoring method comprises the following steps:
the method comprises the following steps: the monitoring devices acquire space scene light rays of an airport running and sliding area through the camera lens, the light rays are converted into electric charges through the CCD sensor and are converted into digital signals through the analog-to-digital converter chip;
step two: the CCD sensor transmits a digital signal to the image acquisition module, and the image acquisition module acquires a real-time airport running and sliding area video image data code stream;
step three: the image preprocessing module performs feature extraction and segmentation processing on an input video image data code stream, then performs matching and superposition on a plurality of acquired images through the image registration module, and then performs splicing and fusion on the plurality of acquired images through the image fusion module to form a panoramic image of the airport running and sliding area;
step four: the object identification module identifies an object with a motion state on a panoramic image of the airport running and sliding area, and the motion object is tracked through the identification and tracking module;
step five: the boundary crossing monitoring module carries out boundary crossing monitoring on objects with motion states in the airport running and sliding area, and when the moving objects cross the safety boundary of the airport running and sliding area, the early warning module carries out safety early warning.
9. The panoramic airport running area monitoring method of claim 8, wherein: in the fourth step, the object identification module identifies that the moving object is an airplane, and when the airplane is in an emergency stop state after moving, the early warning module carries out safety early warning.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810016035.2A CN108184096B (en) | 2018-01-08 | 2018-01-08 | Panoramic monitoring device, system and method for airport running and sliding area |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810016035.2A CN108184096B (en) | 2018-01-08 | 2018-01-08 | Panoramic monitoring device, system and method for airport running and sliding area |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108184096A CN108184096A (en) | 2018-06-19 |
| CN108184096B true CN108184096B (en) | 2020-09-11 |
Family
ID=62550123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810016035.2A Expired - Fee Related CN108184096B (en) | 2018-01-08 | 2018-01-08 | Panoramic monitoring device, system and method for airport running and sliding area |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108184096B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111064929A (en) * | 2019-12-16 | 2020-04-24 | 佛山科学技术学院 | Video data acquisition system based on LoRa wireless transmission |
| CN112381797A (en) * | 2020-11-16 | 2021-02-19 | 广东电网有限责任公司肇庆供电局 | Infrared data-based line object information confirmation method |
| CN114912755A (en) * | 2022-04-13 | 2022-08-16 | 西安卓越视讯科技有限公司 | All-weather comprehensive early warning and monitoring method, system and device for airport flight area |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009089281A1 (en) * | 2008-01-11 | 2009-07-16 | Broadband Discovery Systems, Inc. | System and method for conditioning a signal received at a mems based acquisition device |
| CN102253381A (en) * | 2011-04-20 | 2011-11-23 | 上海交通大学 | System and method for automatically detecting foreign object debris (FOD) on airfield runways |
| CN102946528A (en) * | 2012-12-14 | 2013-02-27 | 安徽水天信息科技有限公司 | Airport runway monitoring system based on intelligent video monitoring for whole scenic spot |
| CN103402044A (en) * | 2013-08-07 | 2013-11-20 | 重庆大学 | Target recognition and tracking system based on multi-source video integration |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103400380B (en) * | 2013-07-25 | 2016-11-23 | 河海大学 | The single camera submarine target three-dimensional track analogy method of fusion image matrix offset |
| DE102015207375A1 (en) * | 2015-04-22 | 2016-10-27 | Robert Bosch Gmbh | Method and device for monitoring an area in front of a vehicle |
| WO2017008224A1 (en) * | 2015-07-13 | 2017-01-19 | 深圳市大疆创新科技有限公司 | Moving object distance detection method, device and aircraft |
| CN105427618B (en) * | 2015-12-18 | 2017-08-11 | 清华大学 | Vehicle takes the alarming method for power and system for the band that stops in emergency |
| US10740912B2 (en) * | 2016-05-19 | 2020-08-11 | Intel Corporation | Detection of humans in images using depth information |
-
2018
- 2018-01-08 CN CN201810016035.2A patent/CN108184096B/en not_active Expired - Fee Related
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009089281A1 (en) * | 2008-01-11 | 2009-07-16 | Broadband Discovery Systems, Inc. | System and method for conditioning a signal received at a mems based acquisition device |
| CN102253381A (en) * | 2011-04-20 | 2011-11-23 | 上海交通大学 | System and method for automatically detecting foreign object debris (FOD) on airfield runways |
| CN102946528A (en) * | 2012-12-14 | 2013-02-27 | 安徽水天信息科技有限公司 | Airport runway monitoring system based on intelligent video monitoring for whole scenic spot |
| CN103402044A (en) * | 2013-08-07 | 2013-11-20 | 重庆大学 | Target recognition and tracking system based on multi-source video integration |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108184096A (en) | 2018-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110297498B (en) | Track inspection method and system based on wireless charging unmanned aerial vehicle | |
| US11748898B2 (en) | Methods and system for infrared tracking | |
| CN108184096B (en) | Panoramic monitoring device, system and method for airport running and sliding area | |
| CA2950791C (en) | Binocular visual navigation system and method based on power robot | |
| Li et al. | Automatic bridge crack detection using Unmanned aerial vehicle and Faster R-CNN | |
| CN103413313B (en) | The binocular vision navigation system of electrically-based robot and method | |
| CN103279949B (en) | Based on the multi-camera parameter automatic calibration system operation method of self-align robot | |
| CN105512628A (en) | Vehicle environment sensing system and method based on unmanned plane | |
| CN104808685A (en) | Vision auxiliary device and method for automatic landing of unmanned aerial vehicle | |
| CN105205785A (en) | Large vehicle operation management system capable of achieving positioning and operation method thereof | |
| CN106851229B (en) | Security and protection intelligent decision method and system based on image recognition | |
| CN111178148A (en) | Ground target geographic coordinate positioning method based on unmanned aerial vehicle vision system | |
| CN104902233A (en) | Comprehensive security monitoring system | |
| Luo et al. | Object-aware power line detection using color and near-infrared images | |
| CN110866483A (en) | Dynamic and static combined visual detection and positioning method for foreign matters on airport runway | |
| CN109782793A (en) | Unmanned plane rescue mode and system | |
| CN102722894B (en) | Intelligent video monitoring method based on automatic calibration of camera | |
| CN107481265A (en) | Target method for relocating and device | |
| Van Phat et al. | Deep4air: A novel deep learning framework for airport airside surveillance | |
| CN105447431B (en) | A kind of docking aircraft method for tracking and positioning and system based on machine vision | |
| CN115965578A (en) | Binocular stereo matching detection method and device based on channel attention mechanism | |
| Parker et al. | Live detection of foreign object debris on runways detection using drones and AI | |
| CN205566539U (en) | Airport runway foreign matter detection device based on H. 265, compression is standard | |
| CN117389294A (en) | Partial discharge detection system and method for airborne solar blind ultraviolet camera | |
| CN112529840A (en) | SF6 inflation equipment pressure drop defect image recognition device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200911 Termination date: 20220108 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |