US20080098538A1 - Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor - Google Patents
Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor Download PDFInfo
- Publication number
- US20080098538A1 US20080098538A1 US11/589,737 US58973706A US2008098538A1 US 20080098538 A1 US20080098538 A1 US 20080098538A1 US 58973706 A US58973706 A US 58973706A US 2008098538 A1 US2008098538 A1 US 2008098538A1
- Authority
- US
- United States
- Prior art keywords
- aircraft
- passenger boarding
- boarding bridge
- engaging end
- doorway
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000000034 method Methods 0.000 title claims description 46
- 230000033001 locomotion Effects 0.000 claims description 34
- 238000003384 imaging method Methods 0.000 claims description 12
- 230000003287 optical effect Effects 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 11
- 230000001131 transforming effect Effects 0.000 claims description 9
- 238000003032 molecular docking Methods 0.000 claims description 8
- 230000000007 visual effect Effects 0.000 claims description 8
- 230000001154 acute effect Effects 0.000 claims description 3
- 238000006073 displacement reaction Methods 0.000 claims 2
- 238000001228 spectrum Methods 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 18
- 238000012545 processing Methods 0.000 description 11
- 238000013459 approach Methods 0.000 description 7
- 239000003973 paint Substances 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 4
- JLQUFIHWVLZVTJ-UHFFFAOYSA-N carbosulfan Chemical compound CCCCN(CCCC)SN(C)C(=O)OC1=CC=CC2=C1OC(C)(C)C2 JLQUFIHWVLZVTJ-UHFFFAOYSA-N 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 230000029305 taxis Effects 0.000 description 3
- 235000010724 Wisteria floribunda Nutrition 0.000 description 2
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 description 2
- 229910052782 aluminium Inorganic materials 0.000 description 2
- 239000000919 ceramic Substances 0.000 description 2
- 230000008867 communication pathway Effects 0.000 description 2
- 230000001934 delay Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000001681 protective effect Effects 0.000 description 2
- 241001272996 Polyphylla fullo Species 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000003028 elevating effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64F—GROUND OR AIRCRAFT-CARRIER-DECK INSTALLATIONS SPECIALLY ADAPTED FOR USE IN CONNECTION WITH AIRCRAFT; DESIGNING, MANUFACTURING, ASSEMBLING, CLEANING, MAINTAINING OR REPAIRING AIRCRAFT, NOT OTHERWISE PROVIDED FOR; HANDLING, TRANSPORTING, TESTING OR INSPECTING AIRCRAFT COMPONENTS, NOT OTHERWISE PROVIDED FOR
- B64F1/00—Ground or aircraft-carrier-deck installations
- B64F1/30—Ground or aircraft-carrier-deck installations for embarking or disembarking passengers
- B64F1/305—Bridges extending between terminal building and aircraft, e.g. telescopic, vertically adjustable
- B64F1/3055—Bridges extending between terminal building and aircraft, e.g. telescopic, vertically adjustable with hinged head interface between aircraft and passenger bridge
Definitions
- the instant invention relates generally to passenger boarding bridges, and more particularly to a vision system and method for automated passenger boarding bridges.
- passenger boarding bridges are used which are telescopically extensible and the height of which is adjustable.
- an apron drive bridge includes a plurality of adjustable modules, including: a rotunda, a telescopic tunnel, a bubble section, a cab, and elevating columns with wheel carriage.
- Other common types of passenger boarding bridges include radial drive bridges and over-the-wing (OTW) bridges. These types of passenger boarding bridges are adjustable, for instance to compensate for different sized aircraft and to compensate for imprecise parking of aircraft at an airport terminal.
- the procedure for aligning the passenger boarding bridge with the doorway of an aircraft has been a time consuming and labor intensive operation.
- the pilot taxis the aircraft along a lead-in line to a final parking position within a gate area.
- the lead-in line is a physical marker that is painted onto the tarmac, and is used for guiding the aircraft along a predetermined path to a final parking position. Additional markings in the form of stop lines, one for each type of aircraft, are provided at predetermined positions along the lead-in line.
- the pilot's view of the tarmac surface from the cockpit of an aircraft is limited. This is particularly true for larger aircraft, such as for instance a Boeing 747-X00.
- the pilot has relied upon instructions that are provided by a human ground marshal together with up to two “wing walkers” to follow the lead-in line.
- stop bars are located on a pole that is fixedly mounted to the ground surface, including appropriate stop bars for each type of aircraft using the gate.
- a tractor or tug is used to tow the aircraft along the lead-in line to its final parking position.
- VDGS Visual Docking Guidance System
- the electronic display device is mounted at a location that makes it highly visible to the pilot when viewed from the cockpit of an aircraft.
- the instructions include a combination of alphanumeric characters and symbols, which the pilot uses to guide the aircraft precisely to the final parking position for the particular type of aircraft.
- the high capital cost of the VDGS system is offset by reduced labor costs and the efficiency that results from stopping the aircraft more precisely than is possible under the guidance of a human ground marshal.
- a manual bridge alignment system requires that a human operator is present to perform the alignment operation each time an aircraft arrives. Delays occur when the human operator is not standing-by to perform the alignment operation as soon as the aircraft comes to a stop. In addition, human operators are prone to errors that may result in the passenger boarding bridge being driven into the aircraft or into a piece of ground service equipment. Such collisions involving the passenger boarding bridge are costly and also result in delays. In order to avoid causing a collision, human operators tend to err on the side of caution and drive the passenger boarding bridge slowly and cautiously.
- WO 96/08411 filed Sep. 14, 1995 in the name of Anderberg, describes a semi-automated system for controlling the movement of a passenger boarding bridge.
- a central computer transmits information relating to the type of the approaching aircraft to a local computer of the passenger boarding bridge.
- the local computer accesses a database and retrieves information relating to the positions of the doors for the type of aircraft that has landed, as well as information relating to the expected final parking position for the type of aircraft.
- the local computer uses the retrieved information to determine an absolute position of the door to which the bridge is to be aligned.
- the passenger boarding bridge is moved under computer control to a position that is close to the determined position of the door, for example within 2-10 meters.
- the bridge is preset to this position before the aircraft stops moving.
- WO 01/34467 filed Nov. 8, 2000 also in the name of Anderberg, teaches that the above system is reliable only for movement to a position that is close to the parked aircraft. Thus, the bridge has to be operated manually during the remaining 2-10 meters of its movement.
- the WO 01/34467 reference also teaches an improvement to the above system, in which electromagnetic sensors are disposed along the outboard end of the passenger boarding bridge for transmitting a set of electromagnetic pulses in different directions and for detecting electromagnetic pulses after reflection from an aircraft. Based on the elapsed time between transmitting and detecting the electromagnetic pulses in different directions, a profile of distance as a function of direction is obtained.
- the system is likely to fail if the targets become obscured due to dirt, scuffs, tearing or the build up of snow, etc.
- Another disadvantage of the system that is described by Spencer et al. is that the targets are imaged continuously as the bridge moves into alignment with the doorway of the aircraft. Continuous imaging, and the associated processing of image data, is necessary in order to update the alignment data so as to compensate for initial low accuracy of the system as well as unexpected movement of the bridge during alignment, such as for instance movement resulting from uneven ground surface adjacent the passenger boarding bridge. Accordingly, the bridge is required to move a significant distance at relatively slow speed, which may unacceptably increase alignment time.
- a vision system for use with an automated control system of a passenger boarding bridge, the automated control system for aligning an aircraft-engaging end of the passenger boarding bridge with a doorway of an aircraft that is stopped within a defined parking space adjacent to the passenger boarding bridge, the vision system comprising: an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge for capturing image data relating to a portion of the aircraft that is proximate an expected stopping location of the doorway; an inclinometer for determining tilt data relating to a sensed deviation of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane; a distance sensor disposed proximate the aircraft-engaging end of the passenger boarding bridge for sensing a distance to the portion of the aircraft that is proximate the expected stopping location of the doorway; a memory element having template image data stored retrievably therein, the template image data comprising at least a template image including a feature that is indicative of the location of
- a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge comprising: positioning the passenger boarding bridge relative to the aircraft, such that the aircraft-engaging end of the passenger boarding bridge is adjacent to an expected stopping location of the doorway of the aircraft; capturing image data using an imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the image data including features that are located within a portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to a current orientation of the aircraft-engaging end of the passenger boarding bridge when the aircraft-engaging end of the passenger boarding bridge is positioned adjacent to the expected stopping location of the doorway of the aircraft; retrieving template image data relating to the doorway of the aircraft; transforming the template image data based on the sensed orientation data; comparing
- a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft comprising: using a visual docking guidance system (VDGS), guiding the aircraft to a predetermined stopping position adjacent to the passenger boarding bridge; positioning the aircraft-engaging end of the passenger boarding bridge at a known position for capturing an image of a portion of the aircraft that is proximate an expected stopping location of the doorway; using an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, capturing an image of the portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data relating to tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane, and relating to distance between the aircraft-engaging end of the passenger boarding bridge and the portion of the aircraft that is proximate the expected stopping location of the doorway; retrieving a template image from a memory element, the template image relating to the
- a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding. bridge comprising: moving the aircraft-engaging end of the passenger boarding bridge to a known position adjacent to an expected stopping location of the doorway of the aircraft; capturing first image data using a first imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the first image data relating to a first portion of the aircraft that is proximate the expected stopping location of the doorway; capturing second image data using a second imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, the second image data relating to a second portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to an orientation of the aircraft-engaging end of the passenger boarding bridge after moving to the known position; retrieving template
- FIG. 1 is a simplified top view showing a passenger boarding bridge that is equipped with an apparatus according to an embodiment of the instant invention, the passenger boarding bridge located at a photo position for the specific type and sub-type of the parked aircraft;
- FIG. 2 is a simplified block diagram showing the components of the camera enclosure that is disposed at the aircraft engaging end of the passenger boarding bridge of FIG. 1 , according to a first embodiment of the instant invention
- FIG. 3 is a simplified block diagram showing the connections between components of a vision system according to the first embodiment of the instant invention
- FIG. 4 is a simplified block diagram showing the components of the camera enclosure that is disposed at the aircraft engaging end of the passenger boarding bridge of FIG. 1 , according to a second embodiment of the instant invention
- FIG. 5 is a simplified block diagram showing the connections between components of a vision system according to the second embodiment of the instant invention.
- FIG. 6 is a simplified flow diagram of a method according to an embodiment of the instant invention.
- FIG. 7 is a simplified flow diagram of another method according to an embodiment of the instant invention.
- FIG. 8 is a simplified flow diagram of another method according to an embodiment of the instant invention.
- FIG. 9 is a simplified block diagram showing the connections between components of a vision system according to a third embodiment of the instant invention.
- FIG. 10 is a simplified block diagram showing the connections between components of a vision system according to a fourth embodiment of the instant invention.
- FIG. 1 shown is a simplified top view of a passenger boarding bridge that is equipped with an apparatus according to a first embodiment of the instant invention, the passenger boarding bridge being located at a photo position for the specific type and sub-type of the parked aircraft.
- the aircraft 100 is stopped within or proximate a parking space that is defined adjacent to the passenger boarding bridge 102 .
- the passenger boarding bridge 102 includes a passageway 104 extending between a terminal building 106 and a pivotal cabin 108 .
- the cabin 108 is open at an aircraft-engaging end 110 thereof.
- a controller 112 of an automated bridge alignment system is provided within the cabin 108 of the passenger boarding bridge 102 .
- the controller 112 is disposed within another portion of the passenger boarding bridge 102 , or within terminal building 106 .
- the controller 112 is in communication with, and provides instruction signals to, not illustrated mechanisms of the automated bridge alignment system, which includes mechanisms for adjusting the length and the angular orientation of the passageway 104 relative to the terminal building 106 , for tilting and pivoting the cabin 108 relative to the passageway 104 , for vertically displacing the cabin 108 relative to the ground surface etc.
- the controller 112 is also in communication with a vision system of the passenger boarding bridge 102 .
- the vision system includes distributed components, some of which are optional, and which are disposed in and about the cabin 108 of passenger boarding bridge 102 . In the specific example that is shown in FIG.
- the components of the vision system include a camera enclosure 114 , an optional laser range finder 116 , an image data processor 118 , and a memory element 120 . Collectively, the components of the vision system are used to identify the location of doorway 122 along the lateral surface of the aircraft 100 , as will be discussed in greater detail in the following sections. Also shown in FIG. 1 is a visual docking guidance system (VDGS) 124 as is well known in the art, including a sensing portion for sensing approach of the aircraft 100 toward the parking space, and a display portion for providing instructions in the form of symbols and/or alphanumeric characters, the instructions for use by the pilot while guiding the aircraft toward the parking space.
- VDGS visual docking guidance system
- FIG. 2 shown is a simplified block diagram of the components that are housed within the camera enclosure of FIG. 1 , according to the first embodiment of the instant invention.
- the camera enclosure 114 is mounted within the back wall of cabin 108 , facing toward the open end 110 with a downward angle of approximately 200 relative to the floor surface of cabin 108 .
- the camera enclosure 114 contains an imager 126 .
- the imager 126 comprises a CMOS image sensor, such as for instance a digital camera available from Allied Vision Technologies.
- the imager 126 is a digital CCD camera.
- the imager 126 is provided in the form of a digital camera that is capable of imaging near infrared (near IR) and/or ultraviolet (UV) radiation, such as for instance a Fuji S3 UVIR.
- the imager 126 has a main optical axis 128 that is approximately perpendicular to the lateral surface of aircraft 100 when the passenger boarding bridge 102 is in the photo position for that type of aircraft, and when the aircraft 100 is parked properly at the parking space.
- the camera enclosure 114 also houses an inclinometer 130 , such as for instance a ceramic electrolytic tilt sensor, which is offset 20° upwards relative to the imager 126 . Accordingly, the inclinometer 130 is approximately parallel to the floor of the cabin 108 .
- a distance sensor such as for instance a laser range finder (LRF) 132 also is housed within the camera enclosure 114 .
- LRF laser range finder
- the LRF 132 is disposed above the imager 126 and is also oriented with a downward angle of approximately 20° relative to the floor surface of cabin 108 .
- the camera enclosure 114 includes a transparent protective window so as to allow imaging of the aircraft whilst protecting the delicate inner components from environmental influences, such as the weather, etc.
- the imager 126 includes a wide-angle lens.
- other types of distance sensors are also envisaged for use with the embodiments of the instant invention.
- the controller 112 is in communication with each one of the VDGS 124 , the LRF 132 and the image data processor 118 .
- the controller 112 is also in communication with the memory element 120 or is in communication with a different memory element (not shown).
- the image data processor 118 is in communication with the controller 112 , the memory element 120 , the imager 126 and the inclinometer 130 .
- VDGS 124 Upon landing, the aircraft 100 taxis along the ground surface toward its assigned gate. The arrival time and gate assignment for the aircraft 100 are known in advance, and are available via the flight information and display system (FIDS) of the airport. As the aircraft approaches the assigned gate, VDGS 124 initiates a procedure for sensing the location of the aircraft and for guiding the aircraft from the sensed location to a parking space that is adjacent to the passenger boarding bridge 102 of the assigned gate. The manner in which VDGS 124 operates is well known to one of skill in the art and will not be described in further detail in this document. Suffice it to say, VDGS 124 identifies the type and sub-type of the aircraft during approach to the parking space, and achieves an aircraft parking precision of at least plus or minus 30 cm from the expected stopping position for the parking space.
- FIDS flight information and display system
- data that is indicative of the type and sub-type of the aircraft 100 is provided to the controller 112 , which then provides a control signal for moving the passenger boarding bridge 102 into a photo position for that particular type and sub-type of aircraft.
- the data is provided manually via an interface having a different button for each type and sub-type of aircraft or having a series of alphanumeric buttons.
- a sensor is used to sense the type and sub-type of the aircraft 100 and sensor data is provided to the controller 112 .
- the type and sub-type data is provided from the VDGS 124 or from the FIDS of the airport.
- the photo position is defined such that the camera enclosure 114 is approximately aligned with the doorway of the aircraft if the aircraft stops precisely at the parking space that is adjacent to the passenger boarding bridge 102 . Since different aircraft types and sub-types have different fuselage configurations, that is to say, the doorways are located at different positions and heights along the side of the aircraft, most different aircraft types and sub-types are expected to require a unique photo position.
- the passenger boarding bridge optionally is moved to the photo position either before or after the aircraft 100 has come to a stop at the parking space.
- the LRF 132 is used to sense the distance to the aircraft, which is done under the control of controller 112 .
- the LRF 132 is disposed above or adjacent to the imager 126 , such that the measured distance corresponds to the actual distance from the camera enclosure 114 to the aircraft 100 .
- Controller 112 compares the distance value that is returned by LRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances the LRF 132 does not return a valid distance value.
- the controller 112 moves the passenger boarding bridge 102 into a back-up photo position, and LRF 132 is used in a second attempt to sense the distance to the aircraft 100 .
- the bridge is moved from the photo position to the back-up photo position by reversing a portion of the movement that brought the bridge into the original photo position.
- the bridge Since the bridge is merely reversing its original course in order to arrive at the back-up photo position, there is very little risk of causing damage to the aircraft or ground service equipment. If a valid distance value is obtained from the back-up photo position, then the automated alignment operation proceeds absent human intervention.
- the controller 112 provides distance value data to the image data processor 118 , as well as data relating to the type and sub-type of the aircraft. An image is then captured using the imager 126 , and image data relating to the captured image is provided to the image data processor 118 . In addition, tilt data relating to the pitch and roll of the cabin 108 is measured using the inclinometer 130 and is also provided to the image data processor 118 . The image data processor 118 then processes the image data relating to the captured image, making use of the distance data and the tilt data.
- the image data processing algorithm is based upon a comparison of features that are contained within the captured image against a small database of template images stored within memory element 120 for each different type and sub-type of aircraft.
- the database includes a plurality of template images of the outline of the doorway, and shows distinctive features such as the handle of the door, the window of the door, the doorway base plate, etc.
- each image of the plurality of template images is obtained from a known optimal viewing position.
- paint scheme features are included in at least some images of the plurality of template images.
- the template images Prior to comparison, the template images are transformed using the distance data that is provided by the LRF 132 , and using tilt data that is provided by inclinometer 130 , respectively.
- the transformed template images are then compared to the captured image, and horizontal and vertical adjustment values are determined. For instance, it is determined by how much the passenger boarding bridge would have to move in the horizontal and vertical directions to cause the features in the real world images to overlap with the features in the transformed template images.
- a rotational movement of the cabin 108 is determined. The rotational movement ensures that the aircraft-engaging end 110 of the cabin 108 is properly aligned with the aircraft fuselage, such that large gaps do not exist therebetween and pose a hazard to passengers or crew.
- the horizontal and vertical adjustment values may be determined a plurality of times, each time using a different feature of the aircraft to make the determination.
- the image data processor 118 After image data processing is complete, the image data processor 118 returns two values to the controller 112 for specifying the x-axis (horizontal) and y-axis (vertical) adjustment to the passenger boarding bridge 100 that is necessary for aligning the aircraft-engaging end 110 of the cabin 108 precisely with the doorway 122 of aircraft 100 .
- a value relating to the rotational movement of the cabin 108 also is provided to the controller 112 .
- the controller 112 receives the two values from the image data processor 118 , and retrieves a bridge specific offset value from memory element 120 . For instance, a bridge specific offset value is stored within memory element 120 for each different photo position and back-up photo position.
- the bridge specific offset value is added to the y-axis adjustment value, so as to correct for slope or unevenness of the apron surface below the passenger boarding bridge 102 .
- the controller 112 determines control signals for moving the passenger boarding bridge 102 , and provides the control signals to mechanisms of the automated bridge alignment system for adjusting the height and the angular orientation of the passageway 104 .
- a control signal is also provided for rotationally adjusting the cabin 108 . Thereafter, the length of the passageway 104 is extended in order to complete the alignment operation with the doorway 122 .
- the imager 126 is provided in the form of a digital camera that is capable of imaging near IR and/or UV.
- a digital camera that is capable of imaging heat signatures around the door or windows of the aircraft.
- the IR signature of the window in the door is different than that of the aluminum skin of the aircraft fuselage.
- use of IR and/or UV imagers may make the paint scheme of the aircraft, which varies considerably from airline to airline, less of an issue.
- FIG. 4 shown is a simplified block diagram of the components that are housed within the camera enclosure of FIG. 1 , according to a second embodiment of the instant invention.
- the camera enclosure 114 is mounted within the back wall of cabin 108 , facing toward the open end 110 with a downward angle of approximately 20° relative to the floor surface of cabin 108 .
- the camera enclosure 114 contains two imagers 126 and 134 .
- one or both of the two imagers 126 and 134 comprises a CMOS image sensor, such as for instance a digital camera available from Allied Vision Technologies.
- one or both of the two imagers 126 and 134 is a digital CCD camera.
- At least one of the two imagers 126 and 134 is provided in the form of a digital camera that is capable of imaging near IR and/or UV, such as for instance a Fuji S3 UVIR.
- the imager 126 has a main optical axis 128 that is approximately perpendicular to the lateral surface of aircraft 100 when the passenger boarding bridge 102 is in the photo position for that type of aircraft, and when the aircraft 100 is properly parked at the parking space.
- the imager 134 is disposed adjacent to the imager 126 and has a main optical axis 136 that is directed approximately 17° away from the main optical axis 128 of imager 126 .
- the main optical axes 128 and 136 of the two imagers 126 and 134 respectively, form an acute angle of approximately 17° facing toward the aircraft 100 .
- the camera enclosure 114 also houses an inclinometer 130 , such as for instance a ceramic electrolytic tilt sensor, which is offset 20° upwards relative to the imagers 126 and 134 . Accordingly, the inclinometer 130 is approximately parallel to the floor of the cabin 108 .
- a distance sensor such as for instance a laser range finder (LRF) 132 also is housed within the camera enclosure 114 . In the specific example that is shown in FIG. 1 , the LRF 132 is disposed above the imagers 126 and 134 and is also oriented with a downward angle of approximately 20° relative to the floor surface of cabin 108 .
- LRF laser range finder
- the camera enclosure 114 includes a transparent protective window so as to allow imaging of the aircraft whilst protecting the delicate inner components from environmental influences, such as the weather, etc.
- a transparent protective window so as to allow imaging of the aircraft whilst protecting the delicate inner components from environmental influences, such as the weather, etc.
- other types of distance sensors are also envisaged for use with the embodiments of the instant invention.
- FIG. 5 shown is simplified block diagram indicating the communication pathways between the components of the vision system and the components of the passenger boarding bridge control system, according to the second embodiment of the instant invention.
- FIG. 5 includes a second LRF 116 , which is not housed within the camera enclosure 114 but rather is mounted separately within the back wall of cabin 108 , and approximately parallel to the floor surface of cabin 108 .
- the second LRF 116 is not mounted approximately parallel to the floor surface of cabin 108 .
- the controller 112 is in communication with each one of the VDGS 124 , the first LRF 132 , the second LRF 116 and the image data processor 118 .
- the controller 112 is also in communication with the memory element 120 or is in communication with a different memory element (not shown).
- the image data processor 118 is in communication with the controller 112 , the memory element 120 , the first imager 126 , the second imager 134 , and with the inclinometer 130 .
- VDGS 124 Upon landing, the aircraft 100 taxis along the ground surface toward its assigned gate. The arrival time and gate assignment for aircraft 100 are known in advance, and are entered into the flight information and display system (FIDS) of the airport. As the aircraft approaches the assigned gate, VDGS 124 initiates a procedure for sensing the location of the aircraft and for guiding the aircraft from the sensed location to a parking space that is adjacent to the passenger boarding bridge 102 of the assigned gate. The manner in which VDGS 124 operates is well known to one of skill in the art and will not be described in further detail in this document. Suffice it to say, VDGS 124 identifies the type and sub-type of the aircraft during approach to the parking space, and achieves an aircraft parking precision of at least plus or minus 30 cm from the expected stopping position for the parking space.
- FIDS flight information and display system
- data that is indicative of the type and sub-type of the aircraft 100 is provided to the controller 112 , which then provides a control signal for moving the passenger boarding bridge 102 into a photo position for that particular type and sub-type of aircraft.
- the data is provided manually via an interface having a different button for each type and sub-type of aircraft or having a series of alphanumeric buttons.
- a sensor is used to sense the type and sub-type of the aircraft 100 and sensor data is provided to the controller 112 .
- the type and sub-type data is provided from the VDGS or from the FIDS of the airport.
- the photo position is defined such that the camera enclosure 114 is approximately aligned with the doorway of the aircraft if the aircraft stops precisely at the parking space that is adjacent to the passenger boarding bridge 102 . Since different aircraft types and sub-types have different fuselage configurations, that is to say, the doorways are located at different positions and heights along the side of the aircraft, most different aircraft types and sub-types will require a unique photo position.
- the passenger boarding bridge optionally is moved to the photo position either before or after the aircraft has come to a stop at the parking space.
- the LRFs 132 and 116 are used to sense the distance to the aircraft 100 , which is done under the control of controller 112 .
- the LRF 132 is disposed above or adjacent to the imagers 126 and 134 , such that the measured distance corresponds to the actual distance from the camera enclosure 114 to the aircraft 100 .
- Controller 112 compares the distance value that is returned by LRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances the LRF 132 does not return a valid distance value.
- the controller 112 compares the distance value that is measured using the second LRF 116 to a second range of expected values. If the distance value that is measured by the second LRF 116 falls within the second range of expected values, then it is considered to be valid for use during alignment. Since the second LRF 116 is not disposed above or adjacent to the imagers 126 and 134 , it is necessary to scale the distance that is returned by the second LRF 116 in order to determine the actual distance to the aircraft relative to the camera enclosure 114 .
- the automated alignment operation is aborted and a human operator is paged to complete the alignment manually.
- the bridge is moved into a back-up photo position for the determined type and sub-type of the aircraft, and fresh distance values are measured using the first LRF 132 and the second LRF 116 .
- the bridge is moved from the photo position to the back-up photo position by reversing a portion of the movement that brought the bridge into the photo position. Since the bridge is merely reversing its original course in order to arrive at the back-up photo position, there is very little risk of causing damage to the aircraft or ground service equipment. If a valid distance value is obtained from the back-up photo position, then the automated alignment operation proceeds absent human intervention.
- the controller 112 provides distance value data to the image data processor 118 , as well as data relating to the type and sub-type of the aircraft. An image is then captured using each of the two imagers 126 and 134 , and image data relating to each captured image is provided to the image data processor 118 . In addition, tilt data relating to the pitch and roll of the cabin 108 is measured using the inclinometer 130 and is also provided to the image data processor 118 . The image data processor then processes separately the image data relating to each of the two captured images, making use of the distance data and the tilt data.
- the image data processing algorithm is based upon a comparison of features that are contained within the captured images against a small database of template images stored within memory element 120 for each different type and sub-type of aircraft.
- the database includes a plurality of template images of the outline of the doorway, and shows distinctive features such as the handle of the door, the window of the door, the doorway base plate, etc.
- paint scheme features are included in at least some of the plurality of template images. Each image of the plurality of template images is obtained from a known optimal viewing position.
- the template images Prior to comparison, the template images are transformed using distance data provided by one of the LRFs 132 and 1 16 , and using tilt data provided by inclinometer 130 , respectively.
- the transformed template images are then compared separately to the two captured images, and horizontal and vertical adjustment values are determined. For instance, it is determined by how much the passenger boarding bridge would have to move in the horizontal and vertical directions to cause the features in the real world images to overlap with the features in the transformed template images.
- a rotational movement of the cabin 108 is determined. The rotational movement ensures that the aircraft-engaging end 110 of the cabin 108 is properly aligned with the aircraft fuselage, such that large gaps do not exist therebetween and pose a hazard to passengers or crew.
- the horizontal and vertical adjustment values may be determined a plurality of times for each captured image, each time using a different feature of the aircraft to make the determination. The determination of which one of the two captured images yields the best values may then be determined statistically.
- the image data processor 118 After image data processing is complete, the image data processor 118 returns two values to the controller 112 for specifying the x-axis (horizontal) and y-axis (vertical) adjustment to the passenger boarding bridge that is necessary for aligning the aircraft-engaging end 110 of the cabin 108 precisely with the doorway 122 of aircraft 100 .
- a value relating to the rotational movement of the cabin 108 also is provided to the controller 112 .
- the controller 112 receives the two values from the image data processor 118 , and retrieves a bridge specific offset value from memory element 120 . For instance, a bridge specific offset value is stored within memory element 120 for each different photo position.
- the bridge specific offset value is added to the y-axis value, and corrects for slope or unevenness of the apron surface below the passenger boarding bridge 102 . Based upon the corrected two values, the controller 112 determines control signals for moving the passenger boarding bridge 102 , and provides the control signals to mechanisms of the automated bridge alignment system for adjusting the height and the angular orientation of the passageway 104 . A control signal is also provided for rotationally adjusting the cabin 108 . Thereafter, the length of the passageway 104 is extended in order to complete the alignment operation with the doorway 122 .
- the two values that are returned to controller 112 are based upon only one or the other of the two captured images. If neither one of the two captured images is determined to be suitable for use then an error message is generated, and bridge alignment is performed in a manual fashion. This may occur, for instance, when the aircraft is improperly parked and therefore neither image contains features that are indicative of the doorway 122 .
- a confidence score is associated with the values that are obtained using each of the two captured images, and those values having the highest confidence score are used for aligning the passenger boarding bridge 102 .
- At least one of the imagers 126 and 134 optionally is provided in the form of a digital camera that is capable of imaging near IR and/or UV.
- a digital camera that is capable of imaging near IR and/or UV.
- Such an imager is capable of imaging heat signatures around the door or windows of the aircraft.
- the IR signature of the window in the door is different than that of the aluminum skin of the aircraft fuselage.
- use of IR and/or UV imagers may make the paint scheme of the aircraft, which varies considerably from airline to airline, less of an issue.
- the method is for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge.
- the passenger boarding bridge is positioned relative to the aircraft, such that the aircraft-engaging end of the passenger boarding bridge is adjacent to an expected stopping location of the doorway of the aircraft.
- the passenger boarding bridge is said to be at a photo position, which is a predefined position for capturing image data relating to a specific type and sub-type of aircraft.
- image data is captured using an imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge.
- the captured image data includes features that are located within a portion of the aircraft that is proximate the expected stopping location of the doorway.
- orientation data including tilt data is sensed, the orientation data relating to a current orientation of the aircraft-engaging end of the passenger boarding bridge when the aircraft-engaging end of the passenger boarding bridge is positioned adjacent to the expected stopping location of the doorway of the aircraft.
- template image data relating to the doorway of the aircraft is retrieved from a memory element.
- the template image data is transformed based on the sensed orientation data.
- the captured image data is compared with the transformed template image data, to determine instruction data relating to a horizontal movement and a vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
- the instruction data is provided to an automated control system of the passenger boarding bridge.
- the horizontal movement and the vertical movement are performed for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
- there is also a rotational movement of the cabin 108 which ensures proper alignment of the aircraft-engaging end 110 of the cabin 108 with the aircraft fuselage.
- FIG. 7 shown is a simplified flow diagram of a method according to an embodiment of the instant invention.
- the method is for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft.
- a visual docking guidance system VDGS
- the aircraft-engaging end of the passenger boarding bridge is positioned at a known position for capturing an image of a portion of the aircraft that is proximate an expected stopping location of the doorway.
- an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge is used to capture an image of the portion of the aircraft that is proximate the expected stopping location of the doorway.
- orientation data is sensed, the orientation data relating to tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane, and relating to a distance between the aircraft-engaging end of the passenger boarding bridge and the portion of the aircraft that is proximate the expected stopping location of the doorway.
- a template image is retrieved from a memory element, the template image relating to the doorway of the aircraft.
- the template image is transformed based on the sensed orientation data.
- instruction data is determined based on a comparison between the transformed template image and the captured image, the instruction data for moving the aircraft-engaging end of the passenger boarding bridge into an aligned condition with the doorway of the aircraft.
- the aircraft-engaging end of the passenger boarding bridge is moved in accordance with the instruction data, so as to align the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
- FIG. 8 shown is a simplified flow diagram of a method according to an embodiment of the instant invention.
- the method is for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge.
- step 800 moving the aircraft-engaging end of the passenger boarding bridge to a known position adjacent to an expected stopping location of the doorway of the aircraft; capturing first image data using a first imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the first image data relating to a first portion of the aircraft that is proximate the expected stopping location of the doorway; capturing second image data using a second imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, the second image data relating to a second portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to an orientation of the aircraft-engaging end of the passenger boarding bridge after moving to the known position; retrieving template image data relating to the doorway of the aircraft; transforming the template image data based on the sensed orientation data; comparing separately the captured first image data and the captured second image data with the transformed template image data, so as to determine independently first alignment data and second alignment data,
- At least two operating modes may be envisaged for those embodiments of the instant invention that include only a single imager 126 .
- the single imager 126 comprises a wide-angle lens that is capable of imaging a portion of an aircraft that is sufficiently large to reliably capture features that are useful for locating the doorway. It is optional but advantageous to obtain distance values using two separate LRFs when operating in the first operating mode.
- a second operating mode the bridge is moved to a first photo position and the single imager 126 is used to capture a first image, then the bridge is moved to a second photo position and the single imager 126 is used to capture a second image. Since the bridge is necessarily moved from one photo position to another photo position when operating in the second operating mode, reliable operation may be achieved even using only a single LRF.
- FIG. 9 is a simplified block diagram showing the connections between components of a vision system according to another embodiment of the instant invention.
- the camera enclosure 114 is identical to the one that is described supra with reference to FIG. 2 .
- the connections are also similar to the ones described with reference to FIG. 3 , except that a second LRF 116 is provided.
- FIG. 9 shows a system including an imager 126 and two separate laser range finders 132 and 116 .
- the LRFs 132 and 116 are used to sense the distance to the aircraft 100 , which is done under the control of controller 112 .
- Controller 112 compares the distance value that is returned by LRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances the LRF 132 does not return a valid distance value. For instance, if the LRF 132 coincidentally is directed toward a window of the aircraft or toward an area of blue or other dark colored paint, then the distance value that is returned by LRF 132 will fall outside the range of expected values and is not likely to be valid. In such an instance the controller 112 compares the distance value that is measured using the second LRF 116 to a second range of expected values.
- the distance value that is measured by the second LRF 116 falls within the second range of expected values, then it is considered to be valid for use during alignment. If neither the first LRF 132 nor the second LRF 116 returns a valid distance value, then the automated alignment operation is aborted and a human operator is paged to complete the alignment manually.
- FIG. 10 is a simplified block diagram showing the connections between components of a vision system according to yet another embodiment of the instant invention.
- the vision system of FIG. 10 is similar to the vision system that was described with reference to FIGS. 2 and 3 , except that LRF 132 within camera enclosure 114 is omitted, and LRF 116 is provided at a location within the cabin 108 that is spatially separated from the camera enclosure 114 .
Landscapes
- Engineering & Computer Science (AREA)
- Architecture (AREA)
- Civil Engineering (AREA)
- Structural Engineering (AREA)
- Mechanical Engineering (AREA)
- Aviation & Aerospace Engineering (AREA)
- Image Analysis (AREA)
Abstract
Description
- The instant invention relates generally to passenger boarding bridges, and more particularly to a vision system and method for automated passenger boarding bridges.
- In order to make aircraft passengers comfortable, and in order to transport them between an airport terminal building and an aircraft in such a way that they are protected from the weather and from other environmental influences, passenger boarding bridges are used which are telescopically extensible and the height of which is adjustable. For instance, an apron drive bridge includes a plurality of adjustable modules, including: a rotunda, a telescopic tunnel, a bubble section, a cab, and elevating columns with wheel carriage. Other common types of passenger boarding bridges include radial drive bridges and over-the-wing (OTW) bridges. These types of passenger boarding bridges are adjustable, for instance to compensate for different sized aircraft and to compensate for imprecise parking of aircraft at an airport terminal.
- Historically, the procedure for aligning the passenger boarding bridge with the doorway of an aircraft has been a time consuming and labor intensive operation. First, the pilot taxis the aircraft along a lead-in line to a final parking position within a gate area. Typically, the lead-in line is a physical marker that is painted onto the tarmac, and is used for guiding the aircraft along a predetermined path to a final parking position. Additional markings in the form of stop lines, one for each type of aircraft, are provided at predetermined positions along the lead-in line. Thus, when the nose gear of a particular type of aircraft stops precisely at the stop line for that type of aircraft, then the aircraft is known to be at its final parking position. Of course, the pilot's view of the tarmac surface from the cockpit of an aircraft is limited. This is particularly true for larger aircraft, such as for instance a Boeing 747-X00. Typically, the pilot has relied upon instructions that are provided by a human ground marshal together with up to two “wing walkers” to follow the lead-in line. Optionally, stop bars are located on a pole that is fixedly mounted to the ground surface, including appropriate stop bars for each type of aircraft using the gate. Alternatively, a tractor or tug is used to tow the aircraft along the lead-in line to its final parking position.
- More recently, sophisticated Visual Docking Guidance Systems have been developed to perform the function of the human ground marshal or tug operator. In particular, a Visual Docking Guidance System (VDGS) senses the aircraft as it approaches the final parking position and provides instructions to the pilot via an electronic display device. The electronic display device is mounted at a location that makes it highly visible to the pilot when viewed from the cockpit of an aircraft. Typically, the instructions include a combination of alphanumeric characters and symbols, which the pilot uses to guide the aircraft precisely to the final parking position for the particular type of aircraft. The high capital cost of the VDGS system is offset by reduced labor costs and the efficiency that results from stopping the aircraft more precisely than is possible under the guidance of a human ground marshal.
- Of course, even when the aircraft is stopped precisely at the final parking position for that type of aircraft, still there is the matter of moving the passenger boarding bridge into an aligned relationship with a doorway of the parked aircraft. In the case of an apron drive bridge this may involve extending the bridge by 10 to 20 meters or more from a stowed position. Unfortunately, driving the bridge over such a long distance is time consuming because often the rate at which the bridge is moved is limited so as to reduce the risk of colliding with ground service vehicles or personnel, and to avoid causing serious damage to the aircraft in the event of a collision therewith. Manual, semi-automated and automated bridge alignment systems are known for moving the passenger boarding bridge relative to the parked aircraft.
- A manual bridge alignment system requires that a human operator is present to perform the alignment operation each time an aircraft arrives. Delays occur when the human operator is not standing-by to perform the alignment operation as soon as the aircraft comes to a stop. In addition, human operators are prone to errors that may result in the passenger boarding bridge being driven into the aircraft or into a piece of ground service equipment. Such collisions involving the passenger boarding bridge are costly and also result in delays. In order to avoid causing a collision, human operators tend to err on the side of caution and drive the passenger boarding bridge slowly and cautiously.
- Semi-automated bridge alignment systems are also known, whereby the bridge is moved rapidly to a preset position under the control of a computer. WO 96/08411, filed Sep. 14, 1995 in the name of Anderberg, describes a semi-automated system for controlling the movement of a passenger boarding bridge. When an aircraft has landed, a central computer transmits information relating to the type of the approaching aircraft to a local computer of the passenger boarding bridge. The local computer accesses a database and retrieves information relating to the positions of the doors for the type of aircraft that has landed, as well as information relating to the expected final parking position for the type of aircraft. The local computer uses the retrieved information to determine an absolute position of the door to which the bridge is to be aligned. Accordingly, the passenger boarding bridge is moved under computer control to a position that is close to the determined position of the door, for example within 2-10 meters. Optionally, the bridge is preset to this position before the aircraft stops moving.
- WO 01/34467, filed Nov. 8, 2000 also in the name of Anderberg, teaches that the above system is reliable only for movement to a position that is close to the parked aircraft. Thus, the bridge has to be operated manually during the remaining 2-10 meters of its movement. The WO 01/34467 reference also teaches an improvement to the above system, in which electromagnetic sensors are disposed along the outboard end of the passenger boarding bridge for transmitting a set of electromagnetic pulses in different directions and for detecting electromagnetic pulses after reflection from an aircraft. Based on the elapsed time between transmitting and detecting the electromagnetic pulses in different directions, a profile of distance as a function of direction is obtained. From the measured distance versus direction profile and the information that is stored in the computer, it is then possible to maneuver the bridge the rest of the way from the preset position to the door of the parked aircraft. Unfortunately, when the aircraft fails to stop at the expected final parking position, the preset position will be misaligned with the actual position of the aircraft door, and human intervention will be required in order to complete the alignment operation.
- Other automated systems have been proposed, such as for instance an automated passenger boarding bridge that uses video cameras in the control of the bridge as described by Schoenberger et al. in U.S. Pat. No. 5,226,204. The system uses video cameras to capture images of an aircraft with which the bridge is to be aligned, which images are provided to a computer for image processing. An object of the image processing is to locate doors along the lateral surface of the aircraft facing the passenger boarding bridge. The bridge is then moved automatically along a direction toward a predetermined door of the parked aircraft. Unfortunately, the system that is described in U.S. Pat. No. 5,226,204 suffers from several disadvantages. For instance, a sophisticated image processing system is required in order to locate a doorway along the side of an aircraft from a distance of up to 15 meters or more. In addition, the “blob” approach and edge detection image processing techniques that are mentioned in the reference are computationally expensive and inherently slow. It is a further disadvantage that factors such as the weather, ambient lighting conditions, markings on the aircraft and the presence of intervening ground support vehicles may also become very significant over such large distances. Furthermore, the bridge is still required to move a significant distance after the aircraft has come to a stop, which increases the time that is needed to complete the alignment and poses a hazard to ground service vehicles and personnel.
- Yet another type of system is disclosed in published United States patent application 2005/0198750 A1, filed Feb. 26, 2003 in the name of Spencer et al. In particular, reflective targets are affixed to the outside surface of an aircraft around the doorway to which the passenger boarding bridge is to be aligned. A plurality of cameras disposed aboard the passenger boarding bridge is used to image the targets during the alignment procedure, and provides image data to a “computer means” for processing thereby. While the use of reflective targets for identifying the doorway of the aircraft is advantageous in that it simplifies image processing, never-the-less airlines are reluctant to apply targets to their aircraft and furthermore regulatory approval may be necessary in order to do so. In addition, the system is likely to fail if the targets become obscured due to dirt, scuffs, tearing or the build up of snow, etc. Another disadvantage of the system that is described by Spencer et al. is that the targets are imaged continuously as the bridge moves into alignment with the doorway of the aircraft. Continuous imaging, and the associated processing of image data, is necessary in order to update the alignment data so as to compensate for initial low accuracy of the system as well as unexpected movement of the bridge during alignment, such as for instance movement resulting from uneven ground surface adjacent the passenger boarding bridge. Accordingly, the bridge is required to move a significant distance at relatively slow speed, which may unacceptably increase alignment time.
- Accordingly, there is a long-standing and unfulfilled need for a bridge alignment system that is capable of safely and reliably aligning a passenger boarding bridge with an aircraft, absent intervention by a human operator. It would be advantageous to provide a system that overcomes at least some of the above-mentioned disadvantages of the prior art.
- In accordance with an aspect of the instant invention there is provided a vision system for use with an automated control system of a passenger boarding bridge, the automated control system for aligning an aircraft-engaging end of the passenger boarding bridge with a doorway of an aircraft that is stopped within a defined parking space adjacent to the passenger boarding bridge, the vision system comprising: an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge for capturing image data relating to a portion of the aircraft that is proximate an expected stopping location of the doorway; an inclinometer for determining tilt data relating to a sensed deviation of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane; a distance sensor disposed proximate the aircraft-engaging end of the passenger boarding bridge for sensing a distance to the portion of the aircraft that is proximate the expected stopping location of the doorway; a memory element having template image data stored retrievably therein, the template image data comprising at least a template image including a feature that is indicative of the location of the doorway of the aircraft; and, an image data processor for transforming the template image data based on the tilt data and the sensed distance, for determining a correlation between the feature that is indicative of the location of the doorway of the aircraft in the transformed template image data and a corresponding feature in the captured image data, and for determining alignment data for use in aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft based on the determined correlation.
- In accordance with another aspect of the instant invention there is provided a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge, the method comprising: positioning the passenger boarding bridge relative to the aircraft, such that the aircraft-engaging end of the passenger boarding bridge is adjacent to an expected stopping location of the doorway of the aircraft; capturing image data using an imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the image data including features that are located within a portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to a current orientation of the aircraft-engaging end of the passenger boarding bridge when the aircraft-engaging end of the passenger boarding bridge is positioned adjacent to the expected stopping location of the doorway of the aircraft; retrieving template image data relating to the doorway of the aircraft; transforming the template image data based on the sensed orientation data; comparing the captured image data with the transformed template image data, to determine instruction data relating to a horizontal movement and a vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft; providing the instruction data to an automated control system of the passenger boarding bridge; and, under the control of the automated control system, performing the horizontal movement and the vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
- In accordance with another aspect of the instant invention there is provided a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, comprising: using a visual docking guidance system (VDGS), guiding the aircraft to a predetermined stopping position adjacent to the passenger boarding bridge; positioning the aircraft-engaging end of the passenger boarding bridge at a known position for capturing an image of a portion of the aircraft that is proximate an expected stopping location of the doorway; using an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, capturing an image of the portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data relating to tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane, and relating to distance between the aircraft-engaging end of the passenger boarding bridge and the portion of the aircraft that is proximate the expected stopping location of the doorway; retrieving a template image from a memory element, the template image relating to the doorway of the aircraft; transforming the template image based on the sensed orientation data; based on a comparison between the transformed template image and the captured image, determining instruction data for moving the aircraft-engaging end of the passenger boarding bridge into an aligned condition with the doorway of the aircraft from the known position; and, under the control of an automated control system of the passenger boarding bridge, moving the aircraft-engaging end of the passenger boarding bridge in accordance with the instruction data, so as to align the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft.
- In accordance with another aspect of the instant invention there is provided a method for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding. bridge, the method comprising: moving the aircraft-engaging end of the passenger boarding bridge to a known position adjacent to an expected stopping location of the doorway of the aircraft; capturing first image data using a first imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the first image data relating to a first portion of the aircraft that is proximate the expected stopping location of the doorway; capturing second image data using a second imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, the second image data relating to a second portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to an orientation of the aircraft-engaging end of the passenger boarding bridge after moving to the known position; retrieving template image data relating to the doorway of the aircraft; transforming the template image data based on the sensed orientation data; comparing separately the captured first image data and the captured second image data with the transformed template image data, so as to determine independently first alignment data and second alignment data, respectively; selecting one of the first alignment data and the second alignment data based on a predefined selection criterion; and, aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft based upon the selected one of the first alignment data and the second alignment data.
- Exemplary embodiments of the invention will now be described in conjunction with the following drawings, in which similar reference numbers designate similar items:
-
FIG. 1 is a simplified top view showing a passenger boarding bridge that is equipped with an apparatus according to an embodiment of the instant invention, the passenger boarding bridge located at a photo position for the specific type and sub-type of the parked aircraft; -
FIG. 2 is a simplified block diagram showing the components of the camera enclosure that is disposed at the aircraft engaging end of the passenger boarding bridge ofFIG. 1 , according to a first embodiment of the instant invention; -
FIG. 3 is a simplified block diagram showing the connections between components of a vision system according to the first embodiment of the instant invention; -
FIG. 4 is a simplified block diagram showing the components of the camera enclosure that is disposed at the aircraft engaging end of the passenger boarding bridge ofFIG. 1 , according to a second embodiment of the instant invention; -
FIG. 5 is a simplified block diagram showing the connections between components of a vision system according to the second embodiment of the instant invention; -
FIG. 6 is a simplified flow diagram of a method according to an embodiment of the instant invention; -
FIG. 7 is a simplified flow diagram of another method according to an embodiment of the instant invention; -
FIG. 8 is a simplified flow diagram of another method according to an embodiment of the instant invention; -
FIG. 9 is a simplified block diagram showing the connections between components of a vision system according to a third embodiment of the instant invention; and, -
FIG. 10 is a simplified block diagram showing the connections between components of a vision system according to a fourth embodiment of the instant invention. - The following description is presented to enable a person skilled in the art to make and use the invention, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and the scope of the invention. Thus, the present invention is not intended to be limited to the embodiments disclosed, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
- Referring to
FIG. 1 , shown is a simplified top view of a passenger boarding bridge that is equipped with an apparatus according to a first embodiment of the instant invention, the passenger boarding bridge being located at a photo position for the specific type and sub-type of the parked aircraft. Theaircraft 100 is stopped within or proximate a parking space that is defined adjacent to thepassenger boarding bridge 102. Thepassenger boarding bridge 102 includes apassageway 104 extending between aterminal building 106 and apivotal cabin 108. Thecabin 108 is open at an aircraft-engagingend 110 thereof. Acontroller 112 of an automated bridge alignment system is provided within thecabin 108 of thepassenger boarding bridge 102. Optionally, thecontroller 112 is disposed within another portion of thepassenger boarding bridge 102, or withinterminal building 106. Thecontroller 112 is in communication with, and provides instruction signals to, not illustrated mechanisms of the automated bridge alignment system, which includes mechanisms for adjusting the length and the angular orientation of thepassageway 104 relative to theterminal building 106, for tilting and pivoting thecabin 108 relative to thepassageway 104, for vertically displacing thecabin 108 relative to the ground surface etc. Thecontroller 112 is also in communication with a vision system of thepassenger boarding bridge 102. The vision system includes distributed components, some of which are optional, and which are disposed in and about thecabin 108 ofpassenger boarding bridge 102. In the specific example that is shown inFIG. 1 , the components of the vision system include acamera enclosure 114, an optionallaser range finder 116, animage data processor 118, and amemory element 120. Collectively, the components of the vision system are used to identify the location ofdoorway 122 along the lateral surface of theaircraft 100, as will be discussed in greater detail in the following sections. Also shown inFIG. 1 is a visual docking guidance system (VDGS) 124 as is well known in the art, including a sensing portion for sensing approach of theaircraft 100 toward the parking space, and a display portion for providing instructions in the form of symbols and/or alphanumeric characters, the instructions for use by the pilot while guiding the aircraft toward the parking space. - Referring now to
FIG. 2 , shown is a simplified block diagram of the components that are housed within the camera enclosure ofFIG. 1 , according to the first embodiment of the instant invention. In this specific and non-limiting example, thecamera enclosure 114 is mounted within the back wall ofcabin 108, facing toward theopen end 110 with a downward angle of approximately 200 relative to the floor surface ofcabin 108. Thecamera enclosure 114 contains animager 126. By way of a non-limiting example, theimager 126 comprises a CMOS image sensor, such as for instance a digital camera available from Allied Vision Technologies. Optionally, theimager 126 is a digital CCD camera. Optionally, theimager 126 is provided in the form of a digital camera that is capable of imaging near infrared (near IR) and/or ultraviolet (UV) radiation, such as for instance a Fuji S3 UVIR. Theimager 126 has a mainoptical axis 128 that is approximately perpendicular to the lateral surface ofaircraft 100 when thepassenger boarding bridge 102 is in the photo position for that type of aircraft, and when theaircraft 100 is parked properly at the parking space. - The
camera enclosure 114 also houses aninclinometer 130, such as for instance a ceramic electrolytic tilt sensor, which is offset 20° upwards relative to theimager 126. Accordingly, theinclinometer 130 is approximately parallel to the floor of thecabin 108. A distance sensor such as for instance a laser range finder (LRF) 132 also is housed within thecamera enclosure 114. In the specific example that is shown inFIG. 1 , theLRF 132 is disposed above theimager 126 and is also oriented with a downward angle of approximately 20° relative to the floor surface ofcabin 108. Optionally, thecamera enclosure 114 includes a transparent protective window so as to allow imaging of the aircraft whilst protecting the delicate inner components from environmental influences, such as the weather, etc. Optionally theimager 126 includes a wide-angle lens. Of course, other types of distance sensors are also envisaged for use with the embodiments of the instant invention. - Referring now to
FIG. 3 , shown is simplified block diagram indicating the communication pathways between the components of the vision system and the components of the passenger boarding bridge control system. Thecontroller 112 is in communication with each one of theVDGS 124, theLRF 132 and theimage data processor 118. Optionally, thecontroller 112 is also in communication with thememory element 120 or is in communication with a different memory element (not shown). Theimage data processor 118 is in communication with thecontroller 112, thememory element 120, theimager 126 and theinclinometer 130. - Operation of the vision system is described in the following paragraphs, with specific reference being made to the various components that are shown in
FIGS. 1-3 . Upon landing, theaircraft 100 taxis along the ground surface toward its assigned gate. The arrival time and gate assignment for theaircraft 100 are known in advance, and are available via the flight information and display system (FIDS) of the airport. As the aircraft approaches the assigned gate,VDGS 124 initiates a procedure for sensing the location of the aircraft and for guiding the aircraft from the sensed location to a parking space that is adjacent to thepassenger boarding bridge 102 of the assigned gate. The manner in whichVDGS 124 operates is well known to one of skill in the art and will not be described in further detail in this document. Suffice it to say,VDGS 124 identifies the type and sub-type of the aircraft during approach to the parking space, and achieves an aircraft parking precision of at least plus or minus 30 cm from the expected stopping position for the parking space. - Separately, data that is indicative of the type and sub-type of the
aircraft 100 is provided to thecontroller 112, which then provides a control signal for moving thepassenger boarding bridge 102 into a photo position for that particular type and sub-type of aircraft. For instance, the data is provided manually via an interface having a different button for each type and sub-type of aircraft or having a series of alphanumeric buttons. Optionally, a sensor is used to sense the type and sub-type of theaircraft 100 and sensor data is provided to thecontroller 112. Further optionally, the type and sub-type data is provided from theVDGS 124 or from the FIDS of the airport. The photo position is defined such that thecamera enclosure 114 is approximately aligned with the doorway of the aircraft if the aircraft stops precisely at the parking space that is adjacent to thepassenger boarding bridge 102. Since different aircraft types and sub-types have different fuselage configurations, that is to say, the doorways are located at different positions and heights along the side of the aircraft, most different aircraft types and sub-types are expected to require a unique photo position. The passenger boarding bridge optionally is moved to the photo position either before or after theaircraft 100 has come to a stop at the parking space. - Once the
passenger boarding bridge 102 stops at the photo position and theaircraft 100 stops at the parking space, theLRF 132 is used to sense the distance to the aircraft, which is done under the control ofcontroller 112. TheLRF 132 is disposed above or adjacent to theimager 126, such that the measured distance corresponds to the actual distance from thecamera enclosure 114 to theaircraft 100.Controller 112 compares the distance value that is returned byLRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances theLRF 132 does not return a valid distance value. For instance, if theLRF 132 coincidentally is directed toward a window of the aircraft or toward an area of blue or other dark colored paint, then the distance value that is returned byLRF 132 will fall outside the range of expected values and is not likely to be valid. In such an instance manual alignment of the passenger boarding bridge may be necessary. Optionally, thecontroller 112 moves thepassenger boarding bridge 102 into a back-up photo position, andLRF 132 is used in a second attempt to sense the distance to theaircraft 100. For instance, the bridge is moved from the photo position to the back-up photo position by reversing a portion of the movement that brought the bridge into the original photo position. Since the bridge is merely reversing its original course in order to arrive at the back-up photo position, there is very little risk of causing damage to the aircraft or ground service equipment. If a valid distance value is obtained from the back-up photo position, then the automated alignment operation proceeds absent human intervention. - The
controller 112 provides distance value data to theimage data processor 118, as well as data relating to the type and sub-type of the aircraft. An image is then captured using theimager 126, and image data relating to the captured image is provided to theimage data processor 118. In addition, tilt data relating to the pitch and roll of thecabin 108 is measured using theinclinometer 130 and is also provided to theimage data processor 118. Theimage data processor 118 then processes the image data relating to the captured image, making use of the distance data and the tilt data. - The image data processing algorithm is based upon a comparison of features that are contained within the captured image against a small database of template images stored within
memory element 120 for each different type and sub-type of aircraft. The database includes a plurality of template images of the outline of the doorway, and shows distinctive features such as the handle of the door, the window of the door, the doorway base plate, etc. In particular, each image of the plurality of template images is obtained from a known optimal viewing position. Optionally, paint scheme features are included in at least some images of the plurality of template images. - Prior to comparison, the template images are transformed using the distance data that is provided by the
LRF 132, and using tilt data that is provided byinclinometer 130, respectively. The transformed template images are then compared to the captured image, and horizontal and vertical adjustment values are determined. For instance, it is determined by how much the passenger boarding bridge would have to move in the horizontal and vertical directions to cause the features in the real world images to overlap with the features in the transformed template images. In addition, a rotational movement of thecabin 108 is determined. The rotational movement ensures that the aircraft-engagingend 110 of thecabin 108 is properly aligned with the aircraft fuselage, such that large gaps do not exist therebetween and pose a hazard to passengers or crew. The horizontal and vertical adjustment values may be determined a plurality of times, each time using a different feature of the aircraft to make the determination. - After image data processing is complete, the
image data processor 118 returns two values to thecontroller 112 for specifying the x-axis (horizontal) and y-axis (vertical) adjustment to thepassenger boarding bridge 100 that is necessary for aligning the aircraft-engagingend 110 of thecabin 108 precisely with thedoorway 122 ofaircraft 100. Of course, a value relating to the rotational movement of thecabin 108 also is provided to thecontroller 112. Thecontroller 112 receives the two values from theimage data processor 118, and retrieves a bridge specific offset value frommemory element 120. For instance, a bridge specific offset value is stored withinmemory element 120 for each different photo position and back-up photo position. The bridge specific offset value is added to the y-axis adjustment value, so as to correct for slope or unevenness of the apron surface below thepassenger boarding bridge 102. Based upon the corrected two values, thecontroller 112 determines control signals for moving thepassenger boarding bridge 102, and provides the control signals to mechanisms of the automated bridge alignment system for adjusting the height and the angular orientation of thepassageway 104. A control signal is also provided for rotationally adjusting thecabin 108. Thereafter, the length of thepassageway 104 is extended in order to complete the alignment operation with thedoorway 122. - As discussed supra, optionally the
imager 126 is provided in the form of a digital camera that is capable of imaging near IR and/or UV. Such an imager is capable of imaging heat signatures around the door or windows of the aircraft. For instance, the IR signature of the window in the door is different than that of the aluminum skin of the aircraft fuselage. Furthermore, use of IR and/or UV imagers may make the paint scheme of the aircraft, which varies considerably from airline to airline, less of an issue. - Referring now to
FIG. 4 , shown is a simplified block diagram of the components that are housed within the camera enclosure ofFIG. 1 , according to a second embodiment of the instant invention. In this specific and non-limiting example, thecamera enclosure 114 is mounted within the back wall ofcabin 108, facing toward theopen end 110 with a downward angle of approximately 20° relative to the floor surface ofcabin 108. Thecamera enclosure 114 contains two 126 and 134. By way of a non-limiting example, one or both of the twoimagers 126 and 134 comprises a CMOS image sensor, such as for instance a digital camera available from Allied Vision Technologies. Optionally, one or both of the twoimagers 126 and 134 is a digital CCD camera. Optionally, at least one of the twoimagers 126 and 134 is provided in the form of a digital camera that is capable of imaging near IR and/or UV, such as for instance a Fuji S3 UVIR. Theimagers imager 126 has a mainoptical axis 128 that is approximately perpendicular to the lateral surface ofaircraft 100 when thepassenger boarding bridge 102 is in the photo position for that type of aircraft, and when theaircraft 100 is properly parked at the parking space. Theimager 134 is disposed adjacent to theimager 126 and has a mainoptical axis 136 that is directed approximately 17° away from the mainoptical axis 128 ofimager 126. In other words, the main 128 and 136 of the twooptical axes 126 and 134, respectively, form an acute angle of approximately 17° facing toward theimagers aircraft 100. - The
camera enclosure 114 also houses aninclinometer 130, such as for instance a ceramic electrolytic tilt sensor, which is offset 20° upwards relative to the 126 and 134. Accordingly, theimagers inclinometer 130 is approximately parallel to the floor of thecabin 108. A distance sensor such as for instance a laser range finder (LRF) 132 also is housed within thecamera enclosure 114. In the specific example that is shown inFIG. 1 , theLRF 132 is disposed above the 126 and 134 and is also oriented with a downward angle of approximately 20° relative to the floor surface ofimagers cabin 108. Optionally, thecamera enclosure 114 includes a transparent protective window so as to allow imaging of the aircraft whilst protecting the delicate inner components from environmental influences, such as the weather, etc. Of course, other types of distance sensors are also envisaged for use with the embodiments of the instant invention. - Referring now to
FIG. 5 , shown is simplified block diagram indicating the communication pathways between the components of the vision system and the components of the passenger boarding bridge control system, according to the second embodiment of the instant invention.FIG. 5 includes asecond LRF 116, which is not housed within thecamera enclosure 114 but rather is mounted separately within the back wall ofcabin 108, and approximately parallel to the floor surface ofcabin 108. Optionally, thesecond LRF 116 is not mounted approximately parallel to the floor surface ofcabin 108. Thecontroller 112 is in communication with each one of theVDGS 124, thefirst LRF 132, thesecond LRF 116 and theimage data processor 118. Optionally, thecontroller 112 is also in communication with thememory element 120 or is in communication with a different memory element (not shown). Theimage data processor 118 is in communication with thecontroller 112, thememory element 120, thefirst imager 126, thesecond imager 134, and with theinclinometer 130. - Operation of the vision system is described in the following paragraphs, with specific reference being made to the various components that are shown in
FIG. 1 andFIGS. 4-5 . Upon landing, theaircraft 100 taxis along the ground surface toward its assigned gate. The arrival time and gate assignment foraircraft 100 are known in advance, and are entered into the flight information and display system (FIDS) of the airport. As the aircraft approaches the assigned gate,VDGS 124 initiates a procedure for sensing the location of the aircraft and for guiding the aircraft from the sensed location to a parking space that is adjacent to thepassenger boarding bridge 102 of the assigned gate. The manner in whichVDGS 124 operates is well known to one of skill in the art and will not be described in further detail in this document. Suffice it to say,VDGS 124 identifies the type and sub-type of the aircraft during approach to the parking space, and achieves an aircraft parking precision of at least plus or minus 30 cm from the expected stopping position for the parking space. - Separately, data that is indicative of the type and sub-type of the
aircraft 100 is provided to thecontroller 112, which then provides a control signal for moving thepassenger boarding bridge 102 into a photo position for that particular type and sub-type of aircraft. For instance, the data is provided manually via an interface having a different button for each type and sub-type of aircraft or having a series of alphanumeric buttons. Optionally, a sensor is used to sense the type and sub-type of theaircraft 100 and sensor data is provided to thecontroller 112. Further optionally, the type and sub-type data is provided from the VDGS or from the FIDS of the airport. The photo position is defined such that thecamera enclosure 114 is approximately aligned with the doorway of the aircraft if the aircraft stops precisely at the parking space that is adjacent to thepassenger boarding bridge 102. Since different aircraft types and sub-types have different fuselage configurations, that is to say, the doorways are located at different positions and heights along the side of the aircraft, most different aircraft types and sub-types will require a unique photo position. The passenger boarding bridge optionally is moved to the photo position either before or after the aircraft has come to a stop at the parking space. - Once the
passenger boarding bridge 102 stops at the photo position and theaircraft 100 stops at the parking space, the 132 and 116 are used to sense the distance to theLRFs aircraft 100, which is done under the control ofcontroller 112. TheLRF 132 is disposed above or adjacent to the 126 and 134, such that the measured distance corresponds to the actual distance from theimagers camera enclosure 114 to theaircraft 100.Controller 112 compares the distance value that is returned byLRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances theLRF 132 does not return a valid distance value. For instance, if theLRF 132 coincidentally is directed toward a window of the aircraft or toward an area of blue or other dark colored paint, then the distance value that is returned byLRF 132 will fall outside the range of expected values and is not likely to be valid. In such an instance thecontroller 112 compares the distance value that is measured using thesecond LRF 116 to a second range of expected values. If the distance value that is measured by thesecond LRF 116 falls within the second range of expected values, then it is considered to be valid for use during alignment. Since thesecond LRF 116 is not disposed above or adjacent to the 126 and 134, it is necessary to scale the distance that is returned by theimagers second LRF 116 in order to determine the actual distance to the aircraft relative to thecamera enclosure 114. If neither thefirst LRF 132 nor thesecond LRF 116 returns a valid distance value, then the automated alignment operation is aborted and a human operator is paged to complete the alignment manually. Optionally, the bridge is moved into a back-up photo position for the determined type and sub-type of the aircraft, and fresh distance values are measured using thefirst LRF 132 and thesecond LRF 116. For instance, the bridge is moved from the photo position to the back-up photo position by reversing a portion of the movement that brought the bridge into the photo position. Since the bridge is merely reversing its original course in order to arrive at the back-up photo position, there is very little risk of causing damage to the aircraft or ground service equipment. If a valid distance value is obtained from the back-up photo position, then the automated alignment operation proceeds absent human intervention. - The
controller 112 provides distance value data to theimage data processor 118, as well as data relating to the type and sub-type of the aircraft. An image is then captured using each of the two 126 and 134, and image data relating to each captured image is provided to theimagers image data processor 118. In addition, tilt data relating to the pitch and roll of thecabin 108 is measured using theinclinometer 130 and is also provided to theimage data processor 118. The image data processor then processes separately the image data relating to each of the two captured images, making use of the distance data and the tilt data. - More specifically, the image data processing algorithm is based upon a comparison of features that are contained within the captured images against a small database of template images stored within
memory element 120 for each different type and sub-type of aircraft. The database includes a plurality of template images of the outline of the doorway, and shows distinctive features such as the handle of the door, the window of the door, the doorway base plate, etc. Optionally, paint scheme features are included in at least some of the plurality of template images. Each image of the plurality of template images is obtained from a known optimal viewing position. - Prior to comparison, the template images are transformed using distance data provided by one of the
LRFs 132 and 1 16, and using tilt data provided byinclinometer 130, respectively. The transformed template images are then compared separately to the two captured images, and horizontal and vertical adjustment values are determined. For instance, it is determined by how much the passenger boarding bridge would have to move in the horizontal and vertical directions to cause the features in the real world images to overlap with the features in the transformed template images. In addition, a rotational movement of thecabin 108 is determined. The rotational movement ensures that the aircraft-engagingend 110 of thecabin 108 is properly aligned with the aircraft fuselage, such that large gaps do not exist therebetween and pose a hazard to passengers or crew. The horizontal and vertical adjustment values may be determined a plurality of times for each captured image, each time using a different feature of the aircraft to make the determination. The determination of which one of the two captured images yields the best values may then be determined statistically. - After image data processing is complete, the
image data processor 118 returns two values to thecontroller 112 for specifying the x-axis (horizontal) and y-axis (vertical) adjustment to the passenger boarding bridge that is necessary for aligning the aircraft-engagingend 110 of thecabin 108 precisely with thedoorway 122 ofaircraft 100. Of course, a value relating to the rotational movement of thecabin 108 also is provided to thecontroller 112. Thecontroller 112 receives the two values from theimage data processor 118, and retrieves a bridge specific offset value frommemory element 120. For instance, a bridge specific offset value is stored withinmemory element 120 for each different photo position. The bridge specific offset value is added to the y-axis value, and corrects for slope or unevenness of the apron surface below thepassenger boarding bridge 102. Based upon the corrected two values, thecontroller 112 determines control signals for moving thepassenger boarding bridge 102, and provides the control signals to mechanisms of the automated bridge alignment system for adjusting the height and the angular orientation of thepassageway 104. A control signal is also provided for rotationally adjusting thecabin 108. Thereafter, the length of thepassageway 104 is extended in order to complete the alignment operation with thedoorway 122. - Although both images are processed fully, the two values that are returned to
controller 112 are based upon only one or the other of the two captured images. If neither one of the two captured images is determined to be suitable for use then an error message is generated, and bridge alignment is performed in a manual fashion. This may occur, for instance, when the aircraft is improperly parked and therefore neither image contains features that are indicative of thedoorway 122. Optionally, a confidence score is associated with the values that are obtained using each of the two captured images, and those values having the highest confidence score are used for aligning thepassenger boarding bridge 102. - As discussed supra, at least one of the
126 and 134 optionally is provided in the form of a digital camera that is capable of imaging near IR and/or UV. Such an imager is capable of imaging heat signatures around the door or windows of the aircraft. For instance, the IR signature of the window in the door is different than that of the aluminum skin of the aircraft fuselage. Furthermore, use of IR and/or UV imagers may make the paint scheme of the aircraft, which varies considerably from airline to airline, less of an issue.imagers - Referring now to
FIG. 6 , shown is a simplified flow diagram of a method according to an embodiment of the instant invention. In particular, the method is for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge. Atstep 600 the passenger boarding bridge is positioned relative to the aircraft, such that the aircraft-engaging end of the passenger boarding bridge is adjacent to an expected stopping location of the doorway of the aircraft. The passenger boarding bridge is said to be at a photo position, which is a predefined position for capturing image data relating to a specific type and sub-type of aircraft. Atstep 602 image data is captured using an imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge. The captured image data includes features that are located within a portion of the aircraft that is proximate the expected stopping location of the doorway. Atstep 604 orientation data including tilt data is sensed, the orientation data relating to a current orientation of the aircraft-engaging end of the passenger boarding bridge when the aircraft-engaging end of the passenger boarding bridge is positioned adjacent to the expected stopping location of the doorway of the aircraft. Atstep 606 template image data relating to the doorway of the aircraft is retrieved from a memory element. Atstep 608 the template image data is transformed based on the sensed orientation data. Atstep 610 the captured image data is compared with the transformed template image data, to determine instruction data relating to a horizontal movement and a vertical movement for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft. Atstep 612 the instruction data is provided to an automated control system of the passenger boarding bridge. Atstep 614, under the control of the automated control system, the horizontal movement and the vertical movement are performed for aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft. As described supra, in addition to the horizontal and vertical movement there is also a rotational movement of thecabin 108, which ensures proper alignment of the aircraft-engagingend 110 of thecabin 108 with the aircraft fuselage. - Referring now to
FIG. 7 , shown is a simplified flow diagram of a method according to an embodiment of the instant invention. In particular, the method is for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft. At step 700 a visual docking guidance system (VDGS) is used to guide the aircraft to a predetermined stopping position adjacent to the passenger boarding bridge. Atstep 702 the aircraft-engaging end of the passenger boarding bridge is positioned at a known position for capturing an image of a portion of the aircraft that is proximate an expected stopping location of the doorway. Atstep 704 an imager disposed proximate the aircraft-engaging end of the passenger boarding bridge is used to capture an image of the portion of the aircraft that is proximate the expected stopping location of the doorway. Atstep 706 orientation data is sensed, the orientation data relating to tilt of the aircraft-engaging end of the passenger boarding bridge relative a horizontal reference plane, and relating to a distance between the aircraft-engaging end of the passenger boarding bridge and the portion of the aircraft that is proximate the expected stopping location of the doorway. At step 708 a template image is retrieved from a memory element, the template image relating to the doorway of the aircraft. Atstep 710 the template image is transformed based on the sensed orientation data. Atstep 712 instruction data is determined based on a comparison between the transformed template image and the captured image, the instruction data for moving the aircraft-engaging end of the passenger boarding bridge into an aligned condition with the doorway of the aircraft. Atstep 714, under the control of an automated control system of the passenger boarding bridge, the aircraft-engaging end of the passenger boarding bridge is moved in accordance with the instruction data, so as to align the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft. - Referring now to
FIG. 8 , shown is a simplified flow diagram of a method according to an embodiment of the instant invention. In particular, the method is for aligning an aircraft-engaging end of a passenger boarding bridge with a doorway of an aircraft, the aircraft stopped within a defined parking space adjacent to the passenger boarding bridge. At step 800 moving the aircraft-engaging end of the passenger boarding bridge to a known position adjacent to an expected stopping location of the doorway of the aircraft; capturing first image data using a first imager that is disposed proximate the aircraft-engaging end of the passenger boarding bridge, the first image data relating to a first portion of the aircraft that is proximate the expected stopping location of the doorway; capturing second image data using a second imager disposed proximate the aircraft-engaging end of the passenger boarding bridge, the second image data relating to a second portion of the aircraft that is proximate the expected stopping location of the doorway; sensing orientation data including tilt data, the orientation data relating to an orientation of the aircraft-engaging end of the passenger boarding bridge after moving to the known position; retrieving template image data relating to the doorway of the aircraft; transforming the template image data based on the sensed orientation data; comparing separately the captured first image data and the captured second image data with the transformed template image data, so as to determine independently first alignment data and second alignment data, respectively; selecting one of the first alignment data and the second alignment data based on a predefined selection criterion; and aligning the aircraft-engaging end of the passenger boarding bridge with the doorway of the aircraft based upon the selected one of the first alignment data and the second alignment data. - At least two operating modes may be envisaged for those embodiments of the instant invention that include only a
single imager 126. In a first operating mode thesingle imager 126 comprises a wide-angle lens that is capable of imaging a portion of an aircraft that is sufficiently large to reliably capture features that are useful for locating the doorway. It is optional but advantageous to obtain distance values using two separate LRFs when operating in the first operating mode. In a second operating mode the bridge is moved to a first photo position and thesingle imager 126 is used to capture a first image, then the bridge is moved to a second photo position and thesingle imager 126 is used to capture a second image. Since the bridge is necessarily moved from one photo position to another photo position when operating in the second operating mode, reliable operation may be achieved even using only a single LRF. -
FIG. 9 is a simplified block diagram showing the connections between components of a vision system according to another embodiment of the instant invention. InFIG. 9 , thecamera enclosure 114 is identical to the one that is described supra with reference toFIG. 2 . The connections are also similar to the ones described with reference toFIG. 3 , except that asecond LRF 116 is provided. Accordingly,FIG. 9 shows a system including animager 126 and two separate 132 and 116. As discussed with reference tolaser range finders FIG. 5 , once thepassenger boarding bridge 102 stops at the photo position and theaircraft 100 stops at the parking space, the 132 and 116 are used to sense the distance to theLRFs aircraft 100, which is done under the control ofcontroller 112.Controller 112 compares the distance value that is returned byLRF 132 to a range of expected values. If the distance value falls within the range of expected values then the distance value is considered to be valid. However, in some instances theLRF 132 does not return a valid distance value. For instance, if theLRF 132 coincidentally is directed toward a window of the aircraft or toward an area of blue or other dark colored paint, then the distance value that is returned byLRF 132 will fall outside the range of expected values and is not likely to be valid. In such an instance thecontroller 112 compares the distance value that is measured using thesecond LRF 116 to a second range of expected values. If the distance value that is measured by thesecond LRF 116 falls within the second range of expected values, then it is considered to be valid for use during alignment. If neither thefirst LRF 132 nor thesecond LRF 116 returns a valid distance value, then the automated alignment operation is aborted and a human operator is paged to complete the alignment manually. -
FIG. 10 is a simplified block diagram showing the connections between components of a vision system according to yet another embodiment of the instant invention. The vision system ofFIG. 10 is similar to the vision system that was described with reference toFIGS. 2 and 3 , except thatLRF 132 withincamera enclosure 114 is omitted, andLRF 116 is provided at a location within thecabin 108 that is spatially separated from thecamera enclosure 114. - Numerous other embodiments may be envisaged without departing from the spirit and scope of the invention.
Claims (34)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11/589,737 US20080098538A1 (en) | 2006-10-31 | 2006-10-31 | Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor |
| CA002606644A CA2606644A1 (en) | 2006-10-31 | 2007-10-12 | Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US11/589,737 US20080098538A1 (en) | 2006-10-31 | 2006-10-31 | Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| US20080098538A1 true US20080098538A1 (en) | 2008-05-01 |
Family
ID=39328402
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US11/589,737 Abandoned US20080098538A1 (en) | 2006-10-31 | 2006-10-31 | Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor |
Country Status (2)
| Country | Link |
|---|---|
| US (1) | US20080098538A1 (en) |
| CA (1) | CA2606644A1 (en) |
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100097183A1 (en) * | 2008-10-20 | 2010-04-22 | Pollema Joshua R | System and Method for Coupling a Component to a Vehicle |
| CN101840629A (en) * | 2010-05-05 | 2010-09-22 | 蒂森克虏伯机场系统(中山)有限公司 | Monitoring system and method of aerobridge |
| CN103895874A (en) * | 2012-12-27 | 2014-07-02 | 深圳中集天达空港设备有限公司 | Boarding bridge emergency evacuation electronic control system and emergency evacuation method thereof |
| US20150198955A1 (en) * | 2013-12-24 | 2015-07-16 | Ratier Figeac | Control device for opening and closing a door aircraft |
| US20150276427A1 (en) * | 2014-03-31 | 2015-10-01 | Unibase Information Corp. | Aircraft guidance system having reviewing and video synchronization abilities, and reviewing and video synchronization device of the same |
| JP2015189251A (en) * | 2014-03-27 | 2015-11-02 | 新明和工業株式会社 | Passenger loading bridge |
| WO2016099285A1 (en) * | 2014-12-18 | 2016-06-23 | Marine Aluminum As | Control system and method of landing an end portion of a freely projecting elongated element, and use of an image processor for generating of control parameters for the control system |
| US20180074474A1 (en) * | 2016-09-09 | 2018-03-15 | Accenture Global Solutions Limited | Automated loading bridge positioning using shapes associated with a vehicle |
| US10384805B2 (en) | 2016-03-21 | 2019-08-20 | Adb Safegate Sweden Ab | Optimizing range of aircraft docking system |
| WO2019229105A1 (en) * | 2018-05-30 | 2019-12-05 | thyssenkrupp Airport Solutions, S.A. | Method for automated docking a passenger boarding bridge to an aircraft |
| US10613507B2 (en) | 2016-09-09 | 2020-04-07 | Accenture Global Solutions Limited | Automated loading bridge positioning using encoded decals |
| JP2020147284A (en) * | 2017-07-13 | 2020-09-17 | 新明和工業株式会社 | Passenger boarding bridge |
| EP3653515A4 (en) * | 2017-07-13 | 2020-11-04 | ShinMaywa Industries, Ltd. | PASSENGER BRIDGE |
| CN112046781A (en) * | 2020-09-11 | 2020-12-08 | 深圳中集天达空港设备有限公司 | Boarding bridge collision avoidance method and boarding bridge collision avoidance system |
| WO2020244591A1 (en) | 2019-06-04 | 2020-12-10 | Shenzhen Cimc-Tianda Airport Support Ltd. | Method for docking boarding bridge with aircraft, electronic equipment and storage medium |
| CN113439058A (en) * | 2019-02-14 | 2021-09-24 | 全日空马达服务株式会社 | Aircraft passenger boarding bridge with access control function |
| US20220161942A1 (en) * | 2020-11-25 | 2022-05-26 | Honeywell International Inc. | Airport gate visual docking guidance system digital twin |
| JPWO2023007770A1 (en) * | 2021-07-29 | 2023-02-02 | ||
| US20230109013A1 (en) * | 2020-03-30 | 2023-04-06 | Shinmaywa Industries, Ltd. | Passenger boarding bridge |
| US20230202683A1 (en) * | 2020-05-12 | 2023-06-29 | Shinmaywa Industries, Ltd. | Passenger boarding bridge |
| US12275539B2 (en) * | 2019-03-25 | 2025-04-15 | Shinmaywa Industries, Ltd. | Method of controlling traveling of passenger boarding bridge |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107264830B (en) * | 2017-07-31 | 2023-06-02 | 威海广泰空港设备股份有限公司 | Active safety docking aircraft system for aviation ground equipment |
| CN110641721B (en) * | 2019-10-16 | 2021-02-02 | 北京天睿空间科技股份有限公司 | Boarding bridge parking method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5226204A (en) * | 1991-02-12 | 1993-07-13 | Subtechnique, Inc. | Tele-robotic passenger loading bridge control system |
| US5950266A (en) * | 1996-07-10 | 1999-09-14 | Thyssen Stearns, Inc. | Method and apparatus for connecting a passenger boarding bridge to a movable body |
| US6552327B2 (en) * | 1999-11-09 | 2003-04-22 | Fmt International Trade Ab | Method and apparatus for positioning one end of a movable bridge in relation to a door of a craft |
| US6724314B2 (en) * | 2001-12-26 | 2004-04-20 | Dew Engineering And Development Limited | Automated passenger loading bridge |
| US6742210B2 (en) * | 2002-05-07 | 2004-06-01 | Dew Engineering And Development Limited | Beacon docking system for automatically aligning a passenger loading bridge to a doorway of an aircraft |
| US6907635B2 (en) * | 2002-05-07 | 2005-06-21 | Dew Engineering And Development Limited | Beacon docking system with visual guidance display |
| US20050198750A1 (en) * | 2002-02-27 | 2005-09-15 | Indal Technologies Inc. | Imaging system for a passenger bridge or the like for docking automatically with an aircraft |
| US7093314B2 (en) * | 2002-05-07 | 2006-08-22 | Dew Engineering And Development Limited | Beacon docking system with visual guidance display |
| US20070119003A1 (en) * | 2005-11-25 | 2007-05-31 | Dew Engineering And Development Limited | Method for aligning a passenger boarding bridge to a doorway of an aircraft |
-
2006
- 2006-10-31 US US11/589,737 patent/US20080098538A1/en not_active Abandoned
-
2007
- 2007-10-12 CA CA002606644A patent/CA2606644A1/en not_active Abandoned
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5226204A (en) * | 1991-02-12 | 1993-07-13 | Subtechnique, Inc. | Tele-robotic passenger loading bridge control system |
| US5950266A (en) * | 1996-07-10 | 1999-09-14 | Thyssen Stearns, Inc. | Method and apparatus for connecting a passenger boarding bridge to a movable body |
| US6552327B2 (en) * | 1999-11-09 | 2003-04-22 | Fmt International Trade Ab | Method and apparatus for positioning one end of a movable bridge in relation to a door of a craft |
| US6724314B2 (en) * | 2001-12-26 | 2004-04-20 | Dew Engineering And Development Limited | Automated passenger loading bridge |
| US20050198750A1 (en) * | 2002-02-27 | 2005-09-15 | Indal Technologies Inc. | Imaging system for a passenger bridge or the like for docking automatically with an aircraft |
| US6742210B2 (en) * | 2002-05-07 | 2004-06-01 | Dew Engineering And Development Limited | Beacon docking system for automatically aligning a passenger loading bridge to a doorway of an aircraft |
| US6757927B2 (en) * | 2002-05-07 | 2004-07-06 | Dew Engineering And Development Limited | Beacon docking system for automatically aligning a passenger loading bridge to a doorway of an aircraft |
| US6907635B2 (en) * | 2002-05-07 | 2005-06-21 | Dew Engineering And Development Limited | Beacon docking system with visual guidance display |
| US7093314B2 (en) * | 2002-05-07 | 2006-08-22 | Dew Engineering And Development Limited | Beacon docking system with visual guidance display |
| US20070119003A1 (en) * | 2005-11-25 | 2007-05-31 | Dew Engineering And Development Limited | Method for aligning a passenger boarding bridge to a doorway of an aircraft |
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100097183A1 (en) * | 2008-10-20 | 2010-04-22 | Pollema Joshua R | System and Method for Coupling a Component to a Vehicle |
| US8552836B2 (en) * | 2008-10-20 | 2013-10-08 | The Boeing Company | System and method for coupling a component to a vehicle |
| CN101840629A (en) * | 2010-05-05 | 2010-09-22 | 蒂森克虏伯机场系统(中山)有限公司 | Monitoring system and method of aerobridge |
| CN103895874A (en) * | 2012-12-27 | 2014-07-02 | 深圳中集天达空港设备有限公司 | Boarding bridge emergency evacuation electronic control system and emergency evacuation method thereof |
| US9651951B2 (en) * | 2013-12-24 | 2017-05-16 | Ratier Figeac | Control device for opening and closing an aircraft door |
| US20150198955A1 (en) * | 2013-12-24 | 2015-07-16 | Ratier Figeac | Control device for opening and closing a door aircraft |
| JP2015189251A (en) * | 2014-03-27 | 2015-11-02 | 新明和工業株式会社 | Passenger loading bridge |
| US20150276427A1 (en) * | 2014-03-31 | 2015-10-01 | Unibase Information Corp. | Aircraft guidance system having reviewing and video synchronization abilities, and reviewing and video synchronization device of the same |
| US9459119B2 (en) * | 2014-03-31 | 2016-10-04 | Unibase Information Corp. | Aircraft guidance system having reviewing and video synchronization abilities, and reviewing and video synchronization device of the same |
| WO2016099285A1 (en) * | 2014-12-18 | 2016-06-23 | Marine Aluminum As | Control system and method of landing an end portion of a freely projecting elongated element, and use of an image processor for generating of control parameters for the control system |
| US10384805B2 (en) | 2016-03-21 | 2019-08-20 | Adb Safegate Sweden Ab | Optimizing range of aircraft docking system |
| US20180074474A1 (en) * | 2016-09-09 | 2018-03-15 | Accenture Global Solutions Limited | Automated loading bridge positioning using shapes associated with a vehicle |
| US10908580B2 (en) * | 2016-09-09 | 2021-02-02 | Accenture Global Solutions Limited | Devices, systems, and methods for automated loading bridge positioning using shapes associated with a vehicle |
| US10613507B2 (en) | 2016-09-09 | 2020-04-07 | Accenture Global Solutions Limited | Automated loading bridge positioning using encoded decals |
| US10875666B2 (en) | 2017-07-13 | 2020-12-29 | Shinmaywa Industries, Ltd. | Passenger boarding bridge |
| EP3653515A4 (en) * | 2017-07-13 | 2020-11-04 | ShinMaywa Industries, Ltd. | PASSENGER BRIDGE |
| JP2020147284A (en) * | 2017-07-13 | 2020-09-17 | 新明和工業株式会社 | Passenger boarding bridge |
| EP4516682A3 (en) * | 2018-05-30 | 2025-04-23 | TK Airport Solutions, S.A. | Method for automated docking a passenger boarding bridge to an aircraft |
| EP4516682A2 (en) | 2018-05-30 | 2025-03-05 | TK Airport Solutions, S.A. | Method for automated docking a passenger boarding bridge to an aircraft |
| WO2019229105A1 (en) * | 2018-05-30 | 2019-12-05 | thyssenkrupp Airport Solutions, S.A. | Method for automated docking a passenger boarding bridge to an aircraft |
| CN112996725A (en) * | 2018-05-30 | 2021-06-18 | 蒂森克虏伯机场解决方案股份公司 | Method for automatically docking a passenger boarding bridge to an aircraft |
| CN113439058A (en) * | 2019-02-14 | 2021-09-24 | 全日空马达服务株式会社 | Aircraft passenger boarding bridge with access control function |
| US12275539B2 (en) * | 2019-03-25 | 2025-04-15 | Shinmaywa Industries, Ltd. | Method of controlling traveling of passenger boarding bridge |
| EP3980336A4 (en) * | 2019-06-04 | 2023-06-14 | Shenzhen Cimc-tianda Airport Support Ltd. | METHOD FOR DOCKING A BOARDING BRIDGE WITH AN AIRCRAFT, ELECTRONIC EQUIPMENT AND INFORMATION MEDIA |
| US12448149B2 (en) | 2019-06-04 | 2025-10-21 | Shenzhen Cimc-Tianda Airport Support Ltd. | Method for docking boarding bridge with aircraft, electronic equipment and storage medium |
| WO2020244591A1 (en) | 2019-06-04 | 2020-12-10 | Shenzhen Cimc-Tianda Airport Support Ltd. | Method for docking boarding bridge with aircraft, electronic equipment and storage medium |
| US20230109013A1 (en) * | 2020-03-30 | 2023-04-06 | Shinmaywa Industries, Ltd. | Passenger boarding bridge |
| US12404037B2 (en) * | 2020-03-30 | 2025-09-02 | Shinmaywa Industries, Ltd. | Passenger boarding bridge |
| US20230202683A1 (en) * | 2020-05-12 | 2023-06-29 | Shinmaywa Industries, Ltd. | Passenger boarding bridge |
| CN112046781A (en) * | 2020-09-11 | 2020-12-08 | 深圳中集天达空港设备有限公司 | Boarding bridge collision avoidance method and boarding bridge collision avoidance system |
| US20220161942A1 (en) * | 2020-11-25 | 2022-05-26 | Honeywell International Inc. | Airport gate visual docking guidance system digital twin |
| US20230202681A1 (en) * | 2020-11-25 | 2023-06-29 | Honeywell International Inc. | Airport gate visual docking guidance system digital twin |
| US12012226B2 (en) * | 2020-11-25 | 2024-06-18 | Honeywell International Inc. | Airport gate visual docking guidance system digital twin |
| US20240278934A1 (en) * | 2020-11-25 | 2024-08-22 | Honeywell International Inc. | Airport gate visual docking guidance system digital twin |
| US11584546B2 (en) * | 2020-11-25 | 2023-02-21 | Honeywell International Inc. | Airport gate visual docking guidance system digital twin |
| US12428169B2 (en) * | 2020-11-25 | 2025-09-30 | Honeywell International Inc. | Airport gate visual docking guidance system digital twin |
| JP7449454B2 (en) | 2021-07-29 | 2024-03-13 | 新明和工業株式会社 | Detection system for detecting the landing area of an aircraft |
| WO2023007770A1 (en) * | 2021-07-29 | 2023-02-02 | 新明和工業株式会社 | Detection system for detecting boarding section of aircraft |
| JPWO2023007770A1 (en) * | 2021-07-29 | 2023-02-02 |
Also Published As
| Publication number | Publication date |
|---|---|
| CA2606644A1 (en) | 2008-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2606644A1 (en) | Vision system for automatically aligning a passenger boarding bridge with a doorway of an aircraft and method therefor | |
| CN100399356C (en) | Imaging system for passenger bridge or similar device automatically parked with airplane | |
| US6907635B2 (en) | Beacon docking system with visual guidance display | |
| US7093314B2 (en) | Beacon docking system with visual guidance display | |
| US6724314B2 (en) | Automated passenger loading bridge | |
| US6742210B2 (en) | Beacon docking system for automatically aligning a passenger loading bridge to a doorway of an aircraft | |
| CA2606691A1 (en) | Method for aligning one end of a passenger boarding bridge with a doorway of an aircraft | |
| JP6209354B2 (en) | How to identify an aircraft in relation to aircraft parking at a stand | |
| US20180354650A1 (en) | Automatic passenger boarding bridge docking system | |
| US7120959B2 (en) | Automated elevational adjustment of passenger loading bridge | |
| US6772464B2 (en) | Method and apparatus for aligning an aircraft with a passenger loading bridge | |
| US20210253271A1 (en) | Method of automated undocking a passenger boarding bridge from an aircraft | |
| US7603737B2 (en) | Passenger boarding bridge alignment system and method therefor | |
| US20070119003A1 (en) | Method for aligning a passenger boarding bridge to a doorway of an aircraft | |
| HK1075030A (en) | Imaging system for a passenger bridge or the like for docking automatically with an aircraft |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: DEW ENGINEERING AND DEVELOPMENT LIMITED, CANADA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HUTTON, NEIL;REEL/FRAME:018483/0507 Effective date: 20061026 |
|
| AS | Assignment |
Owner name: DEW ENGINEERING AND DEVELOPMENT ULC, CANADA Free format text: CHANGE OF NAME;ASSIGNOR:DEW ENGINEERING AND DEVELOPMENT LIMITED;REEL/FRAME:022440/0041 Effective date: 20080613 Owner name: DEW ENGINEERING AND DEVELOPMENT ULC,CANADA Free format text: CHANGE OF NAME;ASSIGNOR:DEW ENGINEERING AND DEVELOPMENT LIMITED;REEL/FRAME:022440/0041 Effective date: 20080613 |
|
| STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |