WO1996009207A1 - Systeme autonome d'accostage d'un aeronef pilote par systeme video, procede et dispositif associes - Google Patents

Systeme autonome d'accostage d'un aeronef pilote par systeme video, procede et dispositif associes Download PDF

Info

Publication number
WO1996009207A1
WO1996009207A1 PCT/US1995/011803 US9511803W WO9609207A1 WO 1996009207 A1 WO1996009207 A1 WO 1996009207A1 US 9511803 W US9511803 W US 9511803W WO 9609207 A1 WO9609207 A1 WO 9609207A1
Authority
WO
WIPO (PCT)
Prior art keywords
aircraft
image
windshield
pilot
accordance
Prior art date
Application number
PCT/US1995/011803
Other languages
English (en)
Inventor
Ming Fang
Indranil Chakravarty
Long-Ji Lin
Original Assignee
Siemens Corporate Research, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Corporate Research, Inc. filed Critical Siemens Corporate Research, Inc.
Publication of WO1996009207A1 publication Critical patent/WO1996009207A1/fr

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B64—AIRCRAFT; AVIATION; COSMONAUTICS
    • B64F—GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
    • B64F1/00—Ground or aircraft-carrier-deck installations
    • B64F1/002—Taxiing aids

Definitions

  • the present application relates to vehicle docking systems and, more particularly, to a system that enables a pilot of an aircraft to dock at a gate for passenger or cargo transfer with minimal or no guidance or assistance at all from ground personnel.
  • a docking system monitors the movement of an aircraft as it approaches the gate for deplanning or enplaning, and provides signals to the pilot so that the aircraft is correctly positioned at the gate. Correct positioning of the aircraft requires that the aircraft nose wheel be positioned to within 0.5 meter (0.5m) or about 20 inches (20") of a pre-defined mark on the airfield running surface or tarmac and the aircraft body positioned along a centering line. Typically, different marks are delineated for different types of aircraft. The centering line is also usually indicated on the tarmac to help the pilot taxi the plane correctly to the gate area.
  • a docking system must be able to track an aircraft for at least 20m so that the pilot has enough time and maneuvering room to correct any deviations in aircraft position and/or orientation.
  • a docking system automates this process by providing the pilot with a visual display, visible to the pilot, that is mounted on either the terminal building or close to the docking gate.
  • Present docking systems in use in high-volume European airports are based on either induction based measurements or laser propagation delay ( M time-of-flight”) based measurements to detect the position of the aircraft with respect to the gate.
  • M time-of-flight laser propagation delay
  • a mechanical parallax device is mounted on the display so that a pilot can determine the correct orientation when the parallax device is in alignment with the pilot's view of the display.
  • the system comprises sensor loops, a scanning unit, an operator's panel, a visual display board with central unit and serial interfaces connecting the units.
  • the sensor coils are laid in slots in the apron concrete along the center parking lane.
  • the scanning unit is a microprocessor connected to the sensor loops and to the central unit by a serial channel. The purpose of the scanner is to examine the sensor loops in the apron for faults and to activate the right number and positions of loops.
  • the first three loops to be used are activated. As the aircraft proceeds, a new loop is activated in front of the nose-wheel so that three loops are always in use up to the stop position.
  • Disadvantages of existing systems further include the factors that the installation is permanently fixed; loops have to be imbedded in the tarmac; the system cannot distinguish aircraft and vehicles; each taxi line needs its own system installation and any changes are expensive to implement; and the systems can be sensitive to the wheel structural materials.
  • Laser radar sensors have also been developed. Such systems generally require very exacting installations and are typically hampered by weather conditions such as fog and or snow.
  • an autonomous video-based aircraft docking system comprises a camera arrangement, an image processing unit coupled to the camera arrangement and a communications unit coupled to the image processing unit, the camera arrangement being oriented to obtain an image of an aircraft through a designated portion of its approach for docking, the image processing unit processing the image for detecting in accordance with prestored data a location of a predetermined feature of the image and causing the communications unit to output a signal indicative of the location.
  • the image processing unit is a display unit and the signal indicative of the location comprises a visual display.
  • the predetermined feature comprises a windshield of the aircraft.
  • the predetermined feature comprises a set of characteristic points on the windshield.
  • a computer-implemented method for enabling a pilot of an aircraft to dock at a gate comprises the steps of: obtaining an image of at least a portion of the aircraft from a camera; digitizing and enhancing the image; creating an image bounding box so as to localize therein a windshield of the aircraft by reference to a prestored template; localizing corners of the windshield; verifying in accordance with a predetermined pattern whether the windshield localized correctly represents a true windshield of the aircraft, and if not, returning to the step of digitizing, and if so, processing the image for eliminating outliers; determining from the localized windshield a position for the aircraft; and communicating the position to the pilot.
  • apparatus for communicating position to a pilot of an aircraft approaching a docking station comprises: camera apparatus for deriving an image of at least a portion of the aircraft; computer apparatus for:
  • step (e) verifying the windshield localized in step (b) ;
  • the computer apparatus further compares template information stored in its data base memory on aircraft configuration, based on model selection, and compares the information with the image in forming the bounding box in step (c) .
  • a computer-implemented method for enabling a pilot of an aircraft to dock at a gate comprising the steps of: obtaining an image of at least a portion of the aircraft from a camera; digitizing and enhancing the image; creating an image bounding box by comparing the image with a prestored template of a corresponding portion of a selected aircraft type so as to localize therein a windshield of the aircraft; determining from the localized windshield a position for the aircraft; and communicating the position to the pilot.
  • the step of creating an image bounding box comprises localizing therein a plurality of portions of the windshield and comparing the portions with prestored sub-templates of corresponding portions of the selected aircraft type.
  • the step of creating an image bounding box comprises utilizing a succession of templates of increasingly fine resolution as the aircraft approaches.
  • a computer-implemented method for enabling a pilot of an aircraft to dock at a gate comprises the steps of: obtaining an image of at least a portion of the aircraft from a camera; digitizing and enhancing the image; creating an image bounding box so as to localize therein a windshield of the aircraft; localizing corners of the windshield by comparing the image with a prestored template of a corresponding portion of a selected aircraft type so as to localize therein a windshield of the aircraft; determining from the localized corners a position for the aircraft; and communicating the position to the pilot.
  • the step of creating an image bounding box comprises localizing therein a plurality of portions of the corners and comparing the portions with prestored sub-templates of corresponding portions of a selected aircraft type.
  • the step of creating an image bounding box comprises utilizing a succession of templates of increasingly fine resolution as the aircraft approaches.
  • a computer-implemented method for enabling a pilot of an aircraft to dock at a gate comprises the steps of: obtaining an image of at least a portion of the aircraft from a camera; digitizing and enhancing the image; creating an image bounding box by comparing the image with a prestored template of a corresponding portion of a selected aircraft type and comparing portions of image with prestored subtemplates of corresponding portions of the selected aircraft type so as to localize therein a windshield of the aircraft; determining from the localized windshield a position for the aircraft; and communicating the position to the pilot.
  • Figure 1 shows a system overview in accordance with the invention in a flow chart type of format
  • Figure 2 shows a plan view of a gate geometry in accordance with the invention
  • Figure 3 shows a side view or elevation of a gate geometry in accordance with the invention
  • Figure 4 shows a procedure in accordance with the invention for windshield detection and corner detection
  • Figure 5 shows corner detection in accordance with the invention and the presence of an outlier
  • Figure 6 shows windshield verification in accordance with the invention by spatial relationships of detected corners
  • Figure 7 shows windshield localization by using a resolution reduction factor of 8 for template matching, with a focused and a strongly defocused image
  • Figure 8 shows an example of a windshield template with two subtemplates marked with rectangles for multiple evidence based template matching in accordance with the invention
  • Figure 9 shows windshield localization using global histogram modification in accordance with the invention under uniform lighting change, Figure 9a, 9b, 9c, and 9d, respectively showing conditions of dusk, night, fog, and sunny conditions;
  • Figure 10 shows an illustration of the partitioning concept in accordance with the invention for solving problems caused by nonuniform illumination change
  • Figure 11 shows and example of aircraft docking under non-uniform lighting change, in accordance with the invention
  • Figure 12 shows an illustration of a size sensitive template matching scheme in accordance with the invention.
  • Figure 13 shows an illustration of a size sensitive template matching scheme in accordance with the invention using only three adjacent templates.
  • Figure 1 illustrates basic components of a video-based aircraft docking system in accordance with the present invention.
  • the airport management system notifies the docking system of the aircraft type scheduled for docking at the gate.
  • the docking system continuously checks to see whether an aircraft has entered its field of view. Once an aircraft is detected, the docking system attempts to localize the windshield of the aircraft, by matching the input video image against a computer model of the incoming aircraft.
  • An aircraft data-base is provided that contains a model of each possible aircraft type expected at a gate for docking. Images are preferably acquired by a video sensor, such as a CCD camera.
  • the analog image data is digitized and enhanced for windshield localization (box 1) .
  • the windshield localization process finds an object in the image that most resembles the windshield and then locks the feature by placing a bounding or circumscribing box around it.
  • This procedure uses a template from a data-base to match and localize the windshield (box 2) .
  • the bounding box is passed to the corner localization process (box 3) .
  • the purpose of this process is to perform a detailed analysis of the windshield and locate the corners for (i) verifying the windshield and for (ii) matching the windshield shape against a model or aircraft type verification.
  • Once the windshield is verified (box 4) the windshield.corners are passed to an outlier elimination process (box 5) . This process checks the spatial relationship among the corners and eliminates false matches.
  • the spatial coordinates of the corners are passed for position and orientation determination (box 6) .
  • a succession of frames may be used for measuring the position and orientation.
  • the result of this processing is passed on to a display unit, which shows the current aircraft position and orientation and provides guidance to the pilot (box 7) .
  • the entire process is updated several times a second in order to provide real-time guidance to the pilot.
  • the system in accordance with the present invention uses a reduced resolution image to detect and place a bounding box around the windshield of the incoming aircraft. Once the windshield is detected, the bounding box is used to restrict the search area for corner localization in the image.
  • the corner localization and windshield localization would not be perfect at all times, and therefore the system needs to detect possible outliers or false matches.
  • Outliers can be detected by checking whether each corner has the expected spatial relationships with the others.
  • the result of the outlier processing is shown as set of X-Y coordinate points on the image corresponding to the windshield corners.
  • the central dot in the same figure shows the computed center of the aircraft that needs to be aligned with the center line.
  • the aircraft position and orientation Based on the corner coordinates on the video image, various techniques can be used to determine the aircraft position and orientation. The accuracy obtained is determined by the distance of the video sensor to the aircraft and whether one camera or two cameras are used for distance determination. If two cameras are allocated for each gate, the positional error is estimated to be less than 0.5 meter.
  • the present invention allows satisfactory operation over a wide dynamic range that can accommodate changes in outdoor lighting for 24 hour operation.
  • the present invention can also accommodate static and dynamic shadows that may cover the aircraft, and weather related changes, such as water on the aircraft body, that may result in specular reflections.
  • the basic configuration of the docking system is shown in plan view in Figure 2 and in side view or elevation in Figure 3.
  • the distance from the parked plane to the terminal building is approximately 30.0 meters, for example, 31.8 meters along RR on the right and 27.5 meters along LL on the left line, WI, W2 indicating the nose wheel.
  • a video sensor and display unit 16 is mounted on the terminal building.
  • the video sensor (including a camera) and display unit may be in separate cabinets and constitute separate units.
  • the plane In order to guide the pilot, the plane must be tracked for about 20.0 meters.
  • RA indicates a ramp.
  • FIG. 3 A side view of the geometry is presented in Figure 3.
  • the camera is shown to be mounted at a height HH of 20 meters.
  • a camera tilt angle T 16.6 degrees will allow tracking the nose and windshield for approximately 20.0 meters.
  • Either one or two cameras WW indicates where the nose wheel stops, at a distance XX of about 30 meters from the terminal building, may be used in this setting.
  • the distance of an aircraft can be determined to within 0.5m accuracy.
  • the windshield corners are used for correlation between the two images.
  • distance can also be determined. Both scenarios are herein disclosed.
  • Two key components of the video based aircraft docking system in accordance with the invention are the aircraft windshield and corner localization.
  • implementation details relating to a preferred embodiment will be explained.
  • the windshield of an aircraft is used for tracking. This provides the following advantages:
  • Windshields are a consistent feature across all different aircraft types; that is, all aircrafts have a windshield located at its front;
  • Windshields are in practice always visible while other parts of an aircraft, such as its wings, engines, wheels, or rudder, may be occluded by building infrastructure or service vehicles on the ground; — Contrast. Windshields exhibit good contrast since they are transparent and embedded in an otherwise opaque aircraft body.
  • Windshields have a well-defined fixed geometry for each commercial aircraft type; Windshields are localized by template matching techniques, which are described in further detail below.
  • windshield corners are herein for determining aircraft position and orientation.
  • the use of windshield corners provides the following advantages.
  • the windshield exhibits a large change in scale.
  • the distortion of the windshield size in the image can be as large as 10 times or more from the time a windshield is detected to final docking position.
  • this may result in the use of many corner templates of different sizes In order to compensate for this size distortion. Since a projective transformation preserves straight lines, the use of corners limits the number of templates needed to accommodate scale changes.
  • proper windshield detection in accordance with the invention is resistant to perspective distortion.
  • the approach angle of the aircraft to the gate may vary, it may be viewed in different orientations and several templates in accordance with the invention may be needed. Windshield corners are sufficient to handle variations in approach angle up to 30 degrees.
  • a windshield has a plurality of corners. Once the positions of all corners or just a subset of corners is known, the aircraft position and its orientation can be determined accurately in accordance with the invention. If the error of each corner position is within 3 pixels (after outlier elimination) , the sub-pixel accuracy of aircraft positions can be obtained, simply because the errors can be averaged out when multiple corner locations are combined. Depending on the accuracy requirements and the hardware cost tradeoffs, a choice can be made to use a one-camera system or a two-camera system for each gate area.
  • windshield corners are utilized for verifying if the localized windshield is indeed a windshield.
  • Figure 6 shows windshield verification by spatial relationships of detected corners. Because the aircraft windshield is not entirely visible, the windshield localization does not locate the windshield correctly and a wrong bounding box is drawn. Consequently, within the bounding box, the corner detector finds 8 phony corners. Because these 'corners' clearly do not have the expected spatial relationships, it can be concluded that the windshield has not been recognized and the corner information should not be trusted. Thus, in Figure 6, because the relative positions of the detected corners are completely wrong, the present invention can determine that the object is not a windshield even though the initial template matching might recognize the presence of windows.
  • the aircraft type or class can be determined.
  • the use of windshield corners for detection in accordance with.the invention is reliable.
  • the corner locations can be determined accurately, and when combined, they can determine the aircraft position and orientation accurately and reliably.
  • a small number of corner templates is sufficient to deal with large perspective distortions and approach angles in the course of docking.
  • Template matching is a technique used to detect the presence of an object in an image.
  • the target object If a picture of an object to be detected, herein referred to as the target object, this picture can be stored as a template.
  • a similarity score is-computed reflecting how well the new image matches the template for each possible template location. The location exhibiting a maximal similarity score obtains is considered as.the object location.
  • a common similarity measure between a template t(i) and an input x(i) is computed as
  • i is the index to each pixel in the template or the input
  • p is a similarity metric
  • the time used to find the best match is determined by two factors: the size of the template and the size of the search space (i.e., all of the template locations to be tested) .
  • the template size can be reduced, or.the search space can be reduced, or both can be done.
  • Illumination changes The input images and the template images may be taken in very different illumination conditions such that their pixel-by-pixel differences are very large although they are images of the same object. The primary issue here is how to "normalize" the input image so that the object in the normalized image will exhibit similarity to the one in the template, pixel by pixel, in spite of illumination or weather related changes.
  • Template matching will always find the best match of the target object in the input image. However, the best match found is not guaranteed to be the target object. For instance, even when the input image does not contain the target object, template matching may still determine a "best match". Therefore, it is herein recognized that some method is needed of knowing whether the localized object is indeed the object being looked for or whether it corresponds to a false match.
  • a template matching scheme with high efficiency which addresses basically the efficiency issue of the template matching.
  • This efficient scheme comprises three components: (i) a multi-resolution processing scheme, (ii) a coarse-to-fine search strategy, and (iii) a limited search space based on a priori information available to the system or provided by previous processing steps.
  • a multi-resolution processing scheme for template matching is applicable in accordance with the invention. Since an aircraft should be docked in real time, low-resolution is used for the initial window.localization.
  • the resolution reduction factor for both image and templates is determined by the size of the smallest dimension of the templates used for the matching. According to the experience of the present inventors, it is preferred that the smallest dimension at the low resolution should not be smaller than 8 pixels. This selection is somewhat subjective and it is also influenced by the object to be matched. For the windshield localization for aircraft docking, this number was found to be suitable. The performance of the template matching for windshield localization will be strongly affected if the smallest dimension is smaller than 8, due the lack of information or constraints in that particular small dimension. This determines the resolution reduction factor that should be used for the template matching.
  • the resolution reduction factor should be 8. Once this factor is determined, all of the templates and the image will be down-sampled by a factor of 8, and the matching is then conducted at that resolution. Since this resolution reduction is template-dependent, it will be dynamically changed if the-smallest dimension of the templates for the matching is changed.
  • low resolution means low computational cost, and therefore high processing speed; low resolution makes possible for bigger search area for the matching; low resolution makes the system insensitive to jittering or defocusing of the video sensor; and low resolution permits the system to perform fast noise filtering and/or enhancement, if the video images are relative noisy.
  • Figure 7 shows an example of the windshield localization in the situation where the image is strongly defocused.
  • Figure 7 shows windshield localization by using a resolution reduction factor of 8 for the template matching.
  • Figure 7a is the original image and the image in Figure 7b is strongly defocused. Nevertheless, the windshield localization still provides relatively accurate location of the windshield.
  • prior knowledge is advantageously utilized to constrain the search space.
  • a facile approach to localize an object in a 512 x 512 image is to process the entire image, testing each possible template location to find the best match. This approach is very time-consuming, however.
  • the docking system in accordance with the present invention utilizes prior knowledge to constrain the search space.
  • Windshield localization is also carried out to advantage in accordance with the present invention.
  • the video sensor Once the video sensor is installed and fixed, there is a limited region in the image where the windshield is.expected to appear when the aircraft enters the field of view. Therefore, it is herein recognized that it should only be necessary to perform template matching in this relatively small region, thereby saving much effort.
  • the windshield After the windshield has been localized the first time, there is only a small local region to which the windshield can next move, since the speed of the aircraft is limited. Therefore, by predicting where the aircraft will move to, the search space can be further constrained.
  • Corner localization is also advantageously addressed in accordance with the present invention. Since the corners are on the windshield, there is a limited region where each corner may appear within the bounding box of the windshield. It is herein recognized that it should therefore not be necessary to have to search the entire image for corners, thereby saving additional computational time.
  • Transition from coarse-to-fine search is also accomplished with advantage in accordance with the present invention. Even after the search space is constrained, the number of locations to be considered may still be very large. To further improve efficiency, a coarse-to-fine search strategy is utilized to find the best match within a constrained search space: first, only every fourth (for example) location in the constrained search space is considered as candidate template location. After the best match in this coarse-resolution search space is found, a finer-resolution search is performed within a small bounding box centered at the best match found previously. If the search space is very large, more than 2 levels of search resolutions can be used. Currently, in the preferred embodiment, 1-3 levels of resolution are used, depending on the size of the windshield in the image.
  • a template matching technique is utilized for windshield localization.
  • a known windshield in a scene can be detected by searching for the location of a match between the windshield template and the scene image.
  • Template matching can be conducted.by searching the displacement of the windshield template, where the mismatch energy is minimum or the correlation is maximum.
  • this approach to conventional template matching will not indicate directly whether the object to be detected is present in the scene. If the match score is high, the confidence of the presence of the windshield is high. If the match score is medium or low, then the template matching does not indicate whether this medium or low match score is due to the deformation or defocusing of the object, or due to noise and lighting influences on the object, or due to the absence of the object.
  • it is necessary for the template matching algorithm to have some additional means to determine if it is not known a priori whether the object is present
  • multiple evidence based template matching is utilized.
  • the basic idea of the multiple evidence based template matching is to use not only a whole object template but also some sub-templates (parts) of the this object to conduct multiple matching with the scene image.
  • Such templates are shown in Figure 8, which shows an example of a windshield template with two sub-templates marked with rectangles for multiple evidence based template matching.
  • the circles in Figure 8 represent the localized positions of these three templates with the image. This multiple matching provides the best matches for the whole object and for the parts of the object corresponding to the sub-templates.
  • the detected best-matched positions of the whole and the sub-templates should also have the similar geometrical relations. If the geometrical relations are not as expected, the confidence is low even though the match score may be high. Conversely, when relations are similar to the relations to be expected, the confidence of the presence of the object to be detected is very high, even though the matched scores are medium or low. It is also possible to integrate the windshield and the corner localization together to implement a fast multiple evidence based windshield localization. Since corners of the windshield can be viewed as parts of the windshield, some regular topological relations among the corners and the windshield are to be expected. This property of the windshield corner can be utilized to verify if the localized windshield is indeed a windshield.
  • the illumination change can be uniform or non-uniform.
  • histogram based techniques which can deal with the uniform lighting change.
  • histogram equalization See for example, W.K. Pratt, "Digital Image Processing", Section 12.1 and 12.2, Wiley, New York, 1978 and the afore-mentioned A. K. Jain, "Fundamentals of digital image processing", Prentice-Hall, New Jersey, 1986.
  • histogram equalization does not work very well for windshield/corner localization.
  • the technique found to be very successful is as follows (see A. K. Jain, "Fundamentals of digital image processing", Prentice-Hall, New Jersey, 1986): the darkest 10% (for example) of the pixels are assigned the minimum pixel value 0, and the brightest 10% (for example) of the pixels are assigned the maximum pixel value 255. The other 80% of the pixels are scaled linearly to be between 0 and 255.
  • Histogram modification is a computationally expensive process, if it has to be repeated it for each subimage which is to be matched against a template.
  • each subimage is not individually normalized; there are often many hundred subimages to be considered. Since the search space is known before doing template matching, the bounding box which includes all of the subimages to be considered within this search space can be computed. Then subimage is normalized within this bounding box, and this is done once for all. It is herein recognized that within the bounding box, the subimage can be further partitioned into regions for histogram modification, if necessary.
  • the histogram modification scheme described above can efficiently compensate the effect of the uniform illumination change for the windshield and corner localization, as shown in Figure 9.
  • the windshield localization by using global histogram modification can be seen to deal with uniform lighting change.
  • Successful windshield localization is shown under different lighting conditions: Figure 9a, 9b, 9c, and 9d, respectively show conditions of dusk, night, fog, and sunny conditions.
  • the limitation of this kind histogram modification scheme is that the output image will not be invariant under non-uniform illumination changes.
  • a new partitioning based method in accordance with the invention, for intensity invariant template matching is herein disclosed.
  • This method is aimed to solve the problem of non-uniform illumination change for the situation where the illumination change can be modeled as a slowly occurring change or as a piece-wise function.
  • the basic idea of the technique in accordance with the invention is to divide a given template into several sub-templates, as shown in Figure 10.
  • Figure 10 shows an illustration of the partitioning concept for solving problems caused by non-uniform illumination change. Some sub-templates may be affected by the non-uniform illumination change, but if the majority of the subtemplates are not affected, the matching result will not be affected.
  • a windshield and corner localization algorithm based on this novel partitioning method is implemented and its performance can be seen from an example where some dark shadows and some bright spots are present in the scene, as shown in Figure 11 which shows an example of aircraft docking under non-uniform lighting change.
  • the efficiency issue of the template matching is herein further explained, but with another perspective, that of size or order sensitive template matching.
  • the windshield of an aircraft must be located while at different distances. Due to perspective distortion, the windshield of the same aircraft has different sizes depending on the distance between the plane and the video sensor. Therefore, the template matching for locating an aircraft windshield involves in general several templates with different sizes corresponding to the windshield of the aircraft at different distances, as shown in Figure 11. It is time-consuming for the windshield localization if all of the templates are used for the template matching.
  • the template matching for the docking is size or order sensitive.
  • Figure 12 illustrates size sensitive template matching in accordance with the invention by using only two adjacent templates.
  • the method starts by locating the object at time to only with the two smallest adjacent templates TO and Tl. Since the object at time to has greater distance that at tl, the match score from the template TO is greater than the one from Tl. After a while, say at time tl, as the object is getting bigger, the match score from Tl becomes to be larger than the one from TO, then the next adjacent template T2 and T3 are used for the matching. As the template T3 is active, templates T3 and T4 are used. This scheme allows, at each instant, the use no more than two templates to locate the object whose size is assumed to be changing continuously and monotonically (enlarging or shrinking) .
  • FIG 13 illustrates size sensitive template matching in accordance with the invention by using only three adjacent templates.
  • the template T3 has the best match score, then T3 and two adjacent templates T2 and T4 are used for the matching.
  • T2 becomes to be active (has the best match score), then Tl, T2 and T3 will be used.
  • T3 is getting active again, then T2, T3, and T4 are again used.
  • T4 is active, then the Templates T3, T3, T5 will be used for the tracking.
  • the scheme allows the location of the object whose size is changing continuously, but not necessarily monotonically.
  • the above-mentioned concept can be applied to not only size sensitive but also order sensitive template matching with multiple templates.
  • a camera need not, in fact, be a conventional type of video camera: instead it may be a video sensing arrangement specifically for obtaining the type of video image herein required for further processing.
  • communication to the pilot need not be by visual display: audio communication by voice synthesizer or other means may be employed. Accordingly, it will be understood that such changes and modifications are intended to be within the spirit of the invention whereof the scope is defined by the features of the claims following.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Image Analysis (AREA)

Abstract

Un système autonome d'accostage d'un aéronef, piloté par système vidéo, comprend un ensemble caméra, un ensemble de traitement d'images couplé à l'ensemble caméra et un ensemble de communications couplé à l'ensemble de traitement d'images. L'ensemble caméra est orienté de manière à obtenir une image d'un aéronef par l'intermédiaire d'une partie désignée de son approche d'accostage, l'ensemble de traitement d'images traitant cette image, afin de détecter, en fonction de données mémorisées au préalable, un emplacement d'une caractéristique prédéterminée de l'image et de commander l'émission, par l'ensemble de communications, d'un signal indiquant l'orientation et l'emplacement de l'aéronef.
PCT/US1995/011803 1994-09-19 1995-09-18 Systeme autonome d'accostage d'un aeronef pilote par systeme video, procede et dispositif associes WO1996009207A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US30824994A 1994-09-19 1994-09-19
US08/308,249 1994-09-19

Publications (1)

Publication Number Publication Date
WO1996009207A1 true WO1996009207A1 (fr) 1996-03-28

Family

ID=23193194

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1995/011803 WO1996009207A1 (fr) 1994-09-19 1995-09-18 Systeme autonome d'accostage d'un aeronef pilote par systeme video, procede et dispositif associes

Country Status (1)

Country Link
WO (1) WO1996009207A1 (fr)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2748450A1 (fr) * 1996-05-10 1997-11-14 Sextant Avionique Procede et dispositif de guidage d'une perche de ravitaillement en vol
WO1999014723A1 (fr) * 1997-09-18 1999-03-25 Honeywell Ag Procede pour garer en position exacte des avions
WO1999015406A1 (fr) * 1997-09-22 1999-04-01 Siemens Aktiengesellschaft Dispositif d'accostage pour aerogares
WO1999017263A1 (fr) * 1997-09-30 1999-04-08 Siemens Aktiengesellschaft Procede et dispositif de guidage a assistance automatique d'aeronefs vers une place de stationnement, et systeme de gestion associe
WO1999018555A1 (fr) * 1997-10-06 1999-04-15 Siemens Aktiengesellschaft Procede et dispositif de guidage a assistance automatique d'aeronefs vers un emplacement de stationnement
US6542086B2 (en) 1997-09-22 2003-04-01 Siemens Aktiengesellschaft Docking system for airport terminals
WO2007040448A1 (fr) * 2005-10-04 2007-04-12 Fmt International Trade Ab Procede d'arrimage automatique d'une passerelle passagers ou d'une passerelle de manutention de marchandises, a la porte d'un avion
WO2017137241A1 (fr) * 2016-02-10 2017-08-17 Ifm Electronic Gmbh Système d'arrimage pour véhicules, pourvu d'une caméra 3d et procédé de fonctionnement d'un tel système
US10249203B2 (en) 2017-04-17 2019-04-02 Rosemount Aerospace Inc. Method and system for providing docking guidance to a pilot of a taxiing aircraft
EP3813039A1 (fr) * 2019-10-24 2021-04-28 thyssenkrupp Airport Solutions, S.A. Procédé d'observation d'un trafic au sol à l'intérieur d'un aéroport
DE102022134637A1 (de) 2022-12-22 2024-06-27 Rheinmetall Air Defence Ag Verfahren zur fernsteuerung einer fluggastbrücke

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4009668A1 (de) * 1990-03-26 1991-10-02 Siemens Ag Verfahren und vorrichtung zum positionsgenauen abstellen von flugzeugen
EP0459295A2 (fr) * 1990-05-25 1991-12-04 Toshiba Electronic Systems Co., Ltd. Système de guidage pour le garage d'un avion avec utilisation de l'éclairage anti-collision pour fournir la réfÀ©rence de position
GB2246261A (en) * 1990-07-16 1992-01-22 Roke Manor Research Tracking arrangements and systems
JPH0636200A (ja) * 1992-07-15 1994-02-10 Toshiba Tesco Kk 航空機ドッキングガイダンス装置
JPH06199297A (ja) * 1993-01-06 1994-07-19 Nippon Signal Co Ltd:The 駐機位置計測方法
JPH06199298A (ja) * 1993-01-06 1994-07-19 Nippon Signal Co Ltd:The 駐機位置計測装置
DE4301637A1 (de) * 1993-01-22 1994-08-11 Deutsche Aerospace Verfahren zum Andocken eines Flugzeuges an eine Fluggastbrücke eines Flughafengebäudes

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4009668A1 (de) * 1990-03-26 1991-10-02 Siemens Ag Verfahren und vorrichtung zum positionsgenauen abstellen von flugzeugen
EP0459295A2 (fr) * 1990-05-25 1991-12-04 Toshiba Electronic Systems Co., Ltd. Système de guidage pour le garage d'un avion avec utilisation de l'éclairage anti-collision pour fournir la réfÀ©rence de position
GB2246261A (en) * 1990-07-16 1992-01-22 Roke Manor Research Tracking arrangements and systems
JPH0636200A (ja) * 1992-07-15 1994-02-10 Toshiba Tesco Kk 航空機ドッキングガイダンス装置
JPH06199297A (ja) * 1993-01-06 1994-07-19 Nippon Signal Co Ltd:The 駐機位置計測方法
JPH06199298A (ja) * 1993-01-06 1994-07-19 Nippon Signal Co Ltd:The 駐機位置計測装置
DE4301637A1 (de) * 1993-01-22 1994-08-11 Deutsche Aerospace Verfahren zum Andocken eines Flugzeuges an eine Fluggastbrücke eines Flughafengebäudes

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 018, no. 264 (P - 1740) 19 May 1994 (1994-05-19) *
PATENT ABSTRACTS OF JAPAN vol. 018, no. 559 (M - 1692) 25 October 1994 (1994-10-25) *

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6324295B1 (en) * 1996-05-10 2001-11-27 Sextant Avionique Guiding method and device for in-flight-refuelling line
WO1997043180A1 (fr) * 1996-05-10 1997-11-20 Sextant Avionique Procede et dispositif de guidage d'une perche de ravitaillement en vol
FR2748450A1 (fr) * 1996-05-10 1997-11-14 Sextant Avionique Procede et dispositif de guidage d'une perche de ravitaillement en vol
WO1999014723A1 (fr) * 1997-09-18 1999-03-25 Honeywell Ag Procede pour garer en position exacte des avions
WO1999015406A1 (fr) * 1997-09-22 1999-04-01 Siemens Aktiengesellschaft Dispositif d'accostage pour aerogares
US6542086B2 (en) 1997-09-22 2003-04-01 Siemens Aktiengesellschaft Docking system for airport terminals
US6389334B1 (en) 1997-09-30 2002-05-14 Siemens Aktiengesellschaft Process and device for automatically supported guidance of aircraft to a parking position and management system therefor
WO1999017263A1 (fr) * 1997-09-30 1999-04-08 Siemens Aktiengesellschaft Procede et dispositif de guidage a assistance automatique d'aeronefs vers une place de stationnement, et systeme de gestion associe
US6362750B1 (en) * 1997-10-06 2002-03-26 Siemens Ag Process and device for automatically supported guidance of aircraft to a parking position
WO1999018555A1 (fr) * 1997-10-06 1999-04-15 Siemens Aktiengesellschaft Procede et dispositif de guidage a assistance automatique d'aeronefs vers un emplacement de stationnement
WO2007040448A1 (fr) * 2005-10-04 2007-04-12 Fmt International Trade Ab Procede d'arrimage automatique d'une passerelle passagers ou d'une passerelle de manutention de marchandises, a la porte d'un avion
US8645004B2 (en) 2005-10-04 2014-02-04 Fmt International Trade Ab Method for automated docking of a passenger bridge or a goods handling bridge to a door of an aircraft
WO2017137241A1 (fr) * 2016-02-10 2017-08-17 Ifm Electronic Gmbh Système d'arrimage pour véhicules, pourvu d'une caméra 3d et procédé de fonctionnement d'un tel système
US10249203B2 (en) 2017-04-17 2019-04-02 Rosemount Aerospace Inc. Method and system for providing docking guidance to a pilot of a taxiing aircraft
EP3813039A1 (fr) * 2019-10-24 2021-04-28 thyssenkrupp Airport Solutions, S.A. Procédé d'observation d'un trafic au sol à l'intérieur d'un aéroport
WO2021078689A1 (fr) * 2019-10-24 2021-04-29 thyssenkrupp Airport Solutions, S.A. Procédé d'observation d'un trafic de sol dans un aéroport
DE102022134637A1 (de) 2022-12-22 2024-06-27 Rheinmetall Air Defence Ag Verfahren zur fernsteuerung einer fluggastbrücke

Similar Documents

Publication Publication Date Title
US10290219B2 (en) Machine vision-based method and system for aircraft docking guidance and aircraft type identification
US10255520B2 (en) System and method for aircraft docking guidance and aircraft type identification
Soheilian et al. Detection and 3D reconstruction of traffic signs from multiple view color images
JP3619628B2 (ja) 走行環境認識装置
US7684590B2 (en) Method of recognizing and/or tracking objects
DK2109065T3 (en) Aircraft Identification and Docking Management System
KR101569919B1 (ko) 차량의 위치 추정 장치 및 방법
JP2000509343A (ja) 空中給油ラインの誘導方法および装置
US20030225487A1 (en) Method of guiding an aircraft in the final approach phase and a corresponding system
US5762292A (en) Apparatus for identification and tracking of objects
CN106651963B (zh) 一种用于驾驶辅助系统的车载摄像头的安装参数标定方法
CN110443819B (zh) 一种单轨列车的轨道检测方法和装置
WO1996009207A1 (fr) Systeme autonome d'accostage d'un aeronef pilote par systeme video, procede et dispositif associes
JP4095033B2 (ja) 駐機誘導システムにおけるセンターライン識別の方法及びシステム
WO2020065093A1 (fr) Procédé de fonctionnement d'un système de guidage d'amarrage au niveau d'un stand d'aéroport
CN105654084B (zh) 基于激光器的车牌定位方法、装置及系统
CN105447431A (zh) 一种基于机器视觉的入坞飞机跟踪定位方法及系统
US20230267753A1 (en) Learning based system and method for visual docking guidance to detect new approaching aircraft types
CN105335764B (zh) 一种入坞飞机机型识别验证系统及方法
Wolkow et al. Model-based threshold and centerline detection for aircraft positioning during landing approach
Gong et al. A survey of techniques for detection and tracking of airport runways
CN105447496A (zh) 一种基于机器视觉的入坞飞机机型识别验证方法和系统
JP2001101420A (ja) 障害物検出装置及びその方法
Nair et al. Detecting unexpected moving obstacles that appear in the path of a navigating robot
CN112036399B (zh) 飞机舱门识别方法及装置、电子设备以及存储介质

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): JP

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH DE DK ES FR GB GR IE IT LU MC NL PT SE

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
122 Ep: pct application non-entry in european phase