US12486040B2 - Artificial potential field-based boom guidance for automated air-to-air refueling - Google Patents
Artificial potential field-based boom guidance for automated air-to-air refuelingInfo
- Publication number
- US12486040B2 US12486040B2 US18/172,701 US202318172701A US12486040B2 US 12486040 B2 US12486040 B2 US 12486040B2 US 202318172701 A US202318172701 A US 202318172701A US 12486040 B2 US12486040 B2 US 12486040B2
- Authority
- US
- United States
- Prior art keywords
- boom
- receiver
- trajectory
- model
- refueling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B64—AIRCRAFT; AVIATION; COSMONAUTICS
- B64D—EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- B64D39/00—Refuelling during flight
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Program-controlled manipulators
- B25J9/02—Program-controlled manipulators characterised by movement of the arms, e.g. cartesian coordinate type
- B25J9/023—Cartesian coordinate type
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Program-controlled manipulators
- B25J9/16—Program controls
- B25J9/1656—Program controls characterised by programming, planning systems for manipulators
- B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
- B25J9/1666—Avoiding collision or forbidden zones
Definitions
- Air-to-air refueling is the process by which aviation fuel is transferred from a fuel-supplying aircraft (“tanker”) to a fuel-receiving aircraft (“receiver”) while the tanker and receiver fly together in close formation. Air-to-air refueling allows the receiver to remain airborne for extended periods of time, or to increase its flight range relative to traditional stationary/ground-based refueling options.
- refueling boom During boom refueling in particular, aviation fuel is offloaded via an intervening boom apparatus (“refueling boom”) from a fuel tank located aboard the tanker to a fuel tank of the receiver.
- the motion trajectory of the boom is typically controlled by-wire in response to manual or automated control inputs from one or more boom operators situated at a control station aboard the tanker.
- the boom operators carefully align a distal end (“boom tip”) of the boom with a fuel receptacle on the receiver while respective pilots of the tanker and receiver minimize relative motion between the two aircraft. After alignment, the boom tip securely engages the receptable.
- Aviation fuel is then transferred to the receiver through a fuel conduit in the refueling boom.
- the aforementioned control station is typically located just aft of the tanker's cockpit. In this forward position, the refueling boom and the receiver are well outside of the direct view of the boom operators. The boom operators are therefore assisted in visualizing the boom and the receiver via a live video feed from one or more tanker-mounted cameras. Real-time image data of the boom and receiver, assisted by three-dimensional (“3D”) receiver and boom models, are then projected onto high-resolution display screens of the control station. In this manner, the boom operators are able to monitor and control the aerial refueling process.
- 3D three-dimensional
- the present disclosure relates to systems and methods for performing an aerial refueling process, in particular a boom-type automated air-to-air refueling (“A3R”) operation during which a substantially rigid refueling boom is used to offload aviation fuel from a fuel-supplying aircraft (“tanker”) to a fuel-receiving aircraft (“receiver”).
- A3R automated air-to-air refueling
- the technical solutions described herein allow boom operators located aboard the tanker to guide the boom through free space along a planned motion trajectory.
- the planned trajectory in turn is calculated using predictive artificial potential fields (“APFs”) to avoid collisions between the boom and the receiver.
- the planned trajectory in some embodiments can also prevent ingress of the boom into predefined “keep-out” zones around surfaces of the receiver acting as obstacles in the overall context of boom trajectory planning.
- a trajectory planning system for use with a fuel-supplying aircraft (“tanker”) having a refueling boom configured for delivering aviation fuel to a fuel-receiving aircraft (“receiver”) includes a human-machine interface (“HMI”) device and electronic control unit (“ECU”) in communication therewith.
- the HMI device is configured to output electronic control signals.
- the ECU which is also in communication with a plurality of sensors operable for outputting sensor data, is configured, in response to the electronic control signals and the sensor data, to access a three-dimensional (“3D”) receiver boundary model and a 3D boom model of the receiver and the refueling boom, respectively.
- the 3D receiver boundary model includes one or more receiver features.
- the ECU in this particular embodiment also calculates a boom-to-receiver (“bTr”) relative position using the 3D receiver boundary model, the 3D boom model, and the sensor data. Additionally, the ECU is configured to calculate a planned trajectory between a boom tip of the refueling boom and a receptacle of the receiver using the bTr relative position and predictive artificial potential fields (“APFs”). This action includes finding a point on a baseline trajectory of boom tip that is farthest from a straight line between the receptacle and the boom, recording the point on the baseline trajectory as a temporary goal, and shaping the planned trajectory to avoid contact between the boom and the one or more receiver features. The ECU then moves the boom tip toward the temporary goal.
- APFs predictive artificial potential fields
- a method for determining a planned trajectory of a boom tip of a refueling boom connected to a tanker when delivering aviation fuel to a receiver.
- An implementation of the method includes accessing a 3D receiver boundary model and a 3D boom model of the receiver and the refueling boom, respectively, via an ECU.
- the 3D receiver boundary model includes one or more predetermined receiver features.
- the method also includes calculating a bTr relative position via the ECU using the 3D receiver boundary model, the 3D boom model, and sensor data from a plurality of sensors, as well as calculating the planned trajectory between the boom tip and a receptacle of the receiver using the bTr relative position and predictive AFPs.
- this latter action includes finding a point on a baseline trajectory of the refueling boom that is farthest from a straight line between the receptacle and the boom, recording the point on the baseline trajectory as a temporary goal, and shaping the planned trajectory to avoid contact between the boom and the predetermined receiver features.
- the method includes executing a control action via the ECU using the planned trajectory, including moving the boom tip toward the temporary goal.
- a tanker having a fuselage, a refueling boom connected to the fuselage, a boom resolver, a camera, and a trajectory planning system.
- the fuselage is configured to transport aviation fuel.
- the boom resolver is connected to the fuselage and configured to measure a position of the refueling boom and output boom position signals indicative thereof.
- the camera is mounted to the fuselage and configured to output real-time image data of the refueling boom and a receiver.
- the trajectory planning system for its part includes an HMI device configured to output electronic control signals, and an ECU in communication with the HMI device, the boom resolver, and the camera.
- the ECU in this embodiment is configured, in response to the electronic control signals, the boom position signals, and the real-time image data, to access a 3D receiver boundary model and a 3D boom model of the receiver and the refueling boom, respectively, with the 3D receiver boundary model including one or more predetermined receiver features.
- the ECU calculates a bTr relative position using the 3D receiver boundary model, the 3D boom model, the boom position signals, and the real-time image data, and also calculates a planned trajectory between a boom tip of the refueling boom and a receptacle of the receiver using the bTr relative position and predictive APFs.
- This action includes finding a point on a baseline trajectory of the refueling boom that is farthest from a straight line between the receptacle and the boom, recording the point on the baseline trajectory as a temporary goal, and shaping the planned trajectory to avoid contact between the boom and the one or more predetermined receiver features.
- the ECU then moves the boom tip toward the temporary goal.
- FIG. 1 illustrates a representative automated air-to-air (“A3R”) refueling process during which a fuel-supplying aircraft (“tanker”) delivers aviation fuel to a fuel-receiving aircraft (“receiver”) through a refueling boom, the motion trajectory of which is planned herein using predictive artificial potential fields (“APFs”).
- A3R automated air-to-air
- FIG. 2 is a schematic flow diagram of an exemplary trajectory planning system operable for guiding the refueling boom of FIG. 1 during an A3R process using predictive APFs in accordance with the disclosure.
- FIG. 3 is an illustration of a representative receiver and a boom motion trajectory, the latter being determined using the exemplary trajectory planning system of FIG. 2 .
- FIG. 4 is a flow chart describing an embodiment of a method for planning a boom trajectory using the trajectory planning system of FIG. 2 .
- FIGS. 5 A, 5 B, and 5 C are three-dimensional (“3D”) boundary models for a representative receiver, with the 3D receiver boundary models being configured for use with the representative trajectory planning system of FIG. 2 .
- 3D three-dimensional
- FIGS. 6 and 7 illustrate a planned boom trajectory determined using the predictive APFs of the present disclosure.
- FIG. 1 a fuel-supplying aircraft (“tanker”) 10 and a fuel-receiving aircraft (“receiver”) 12 are shown engaged in a representative automated air-to-air refueling (“A3R”) operation.
- Boom operators 14 (see FIG. 2 ), i.e., personnel trained to perform the disclosed aerial refueling tasks, are situated aboard the tanker 10 , typically in proximity to/aft of a cockpit 16 .
- the boom operators 14 control motion of a refueling boom 18 by-wire during the A3R operation as summarized above. In performing this task, the boom operators 14 of FIG.
- APFs predictive artificial potential fields
- APF-based collision avoidance techniques can be applied during end-effector path planning stages of robot control. Such techniques typically apply artificial attractive or repulsive potential fields around obstacles located in the end-effector's operational space. Trajectory planning occurs in programmed logic of a computational path planner. When planning an optimal trajectory for the end-effector, which in the present disclosure encompasses a distal end or “boom tip” E 1 of the refueling boom 18 diametrically opposite a proximal/tanker-connected end E 2 , the APFs are designated in the airspace surrounding predetermined obstacles in the immediate vicinity of the boom 18 , including its boom tip E 1 and other structure.
- APF techniques are specially modified as set forth herein for application as scalar potential fields to the 3D working airspace of the refueling boom 18 .
- the “predictive APFs” of the present disclosure are thereafter used by the trajectory planning system 11 and/or the boom operators 14 of FIG. 2 to help guide the boom tip E 1 toward and into engagement with a fuel receptacle 13 located on the receiver 12 , while at the same avoiding obstacles in the path of the boom 18 .
- the present trajectory planning approach an example of which is shown in FIG.
- relevant obstacles could include structural features of the receiver 12 that, at times, could be located in a line-of-sight with the boom tip E 1 , for instance a canopy 15 , a vertical stabilizer 17 , or an antenna (not shown) of the receiver 12 .
- Travel of the boom tip E 1 and other structural portions or appendages of the refueling boom 18 near obstacles located in the airspace surrounding the receptacle 13 is effectively penalized in control logic of the trajectory planning system 11 to cause the refueling boom 18 to move away from such obstacles, and to thus deviate from a typical line-of-sight/straight-line path.
- This desired effect occurs via application of repulsive APFs around predefined obstacles.
- movement of the boom tip E 1 toward its intended target or goal, in this case the receptacle 13 is rewarded in logic using an attractive APF around the receptacle 13 and, as needed, one or more temporary goals as described below with reference to FIGS. 4 , 6 , and 7 .
- the various APFs are then mathematically combined to shape the planned trajectory of the refueling boom 18 , with the planned trajectory thereafter used to control flight inputs to flight control surfaces 19 and drive actuators (not shown) of the refueling boom 18 .
- obstacles could include protruding flight control surfaces or other structure of the receiver 12 that may be located in a straight line/line-of-sight path between the boom tip E 1 and the receptacle 13 or approach the longitudinal axis of the refueling boom 18 .
- the trajectory planning system 11 uses the APFs as modified herein to cause the refueling boom 18 to follow the gradient of the potential fields when moving toward the receptacle 13 , while at the same time avoiding the obstacles and thus collisions therewith.
- the trajectory planning system 11 may calculate a net potential at the current 3D position of the refueling boom 18 before moving the boom 18 and its boom tip E 1 in the direction of highest potential. More advanced implementations could consider the kinematics and flight dynamics of the refueling boom 18 as well as additional constraints. As a result, the planned trajectory of the boom tip E 1 as determined by the trajectory planning system 11 would deviate from a straight-line path, with such a deviation depicted in FIGS. 6 and 7 and described in detail hereinbelow.
- the tanker 10 includes a fuselage 24 connected to one or more wings 25 .
- the fuselage 24 defines a cargo bay with one or more fuel tanks (not shown) holding aviation fuel for eventual delivery to the receiver 12 .
- Each of the wings 25 may be connected in some configurations to a refueling pod 26 and one or more engines 28 , e.g., jet turbines, with the engines 28 collectively providing sufficient thrust for propelling the tanker 10 .
- the tanker 10 in one or more configurations could also deploy flexible drogues (not shown), with each drogue fluidly coupled to the refueling pods 26 or to the fuselage 24 .
- the fuselage 24 also defines the cockpit 16 proximate a nose 29 of the tanker 10 .
- an empennage assembly 30 is connected to the fuselage 24 , with an empennage assembly 30 in the representative construction of FIG. 1 including a vertical stabilizer 31 and horizontal stabilizers 32 .
- the tanker 10 of FIG. 1 is equipped to perform A3R operations of the types contemplated herein, e.g., as a structurally-modified commercial passenger or transport aircraft having a reinforced airframe suitable for securely transporting the above-noted aviation fuel and associated fuel tanks, and equipped with mission-suitable avionics and control systems. Such modifications collectively enable the tanker 10 to transport aviation fuel to a predetermined rendezvous site with the receiver 12 . Upon reaching the rendezvous site, the tanker 10 flies in close formation with the receiver 12 , the particular configuration of which may differ from that which is depicted in FIG. 1 .
- the tanker 10 may be used to refuel any suitably-equipped receiver 12 , such as but not limited to cargo planes, other tankers, surveillance and/or reconnaissance aircraft, air traffic control aircraft, weather monitoring aircraft, etc.
- any suitably-equipped receiver 12 such as but not limited to cargo planes, other tankers, surveillance and/or reconnaissance aircraft, air traffic control aircraft, weather monitoring aircraft, etc.
- the depicted construction of the receiver 12 of FIG. 1 is therefore illustrative of just one possible embodiment thereof.
- the refueling boom 18 moves within the slipstream of the tanker 10 with a level of control afforded by the flight control surfaces 19 .
- Control inputs to the flight control surfaces 19 are transmitted by-wire from operator inputs from the boom operators 14 of FIG. 2 , with the end result being the guidance of the boom tip E 1 into mating engagement with the receptacle 13 .
- the receptacle 13 is located on a fuselage 124 of the receiver 12 aft of the canopy 15 .
- FIG. 1 illustrates just one possible embodiment.
- the tanker 10 deploys the refueling boom 18 under the close observation and control of the boom operators 14 .
- the boom operators 14 are assisted in their task by real-time image data 200 of the boom 18 and the receiver 12 collected by the camera(s) 20 .
- Each camera 20 is securely mounted to the tanker 10 of FIG. 1 on or near the proximal end E 2 of the boom 18 and communicated to an electronic control unit (“ECU”) 50 for image processing.
- ECU electronice control unit
- the real-time image data 200 is used by the trajectory planning system 11 to calculate a planned trajectory of the refueling boom 18 , which is ultimately used to control motion of the boom 18 via one or more rotary and/or linear actuators (not shown) and its flight control surfaces 19 .
- the present strategy may be contrasted with typical line-of-sight approaches which attempt to maneuver a robot end-effector in a straight line to a goal (see FIG. 6 ).
- a drawback of such an approach if applied to the A3R process of FIG. 1 is a potential collision of the boom 18 of FIG. 1 with potentially obstructing components of the receiver 12 .
- the trajectory planning system 11 employs automated computer vision/machine learning-based solutions, 3D modeling, and predictive APFs determined in accordance with the method 50 M to help plan an optimal trajectory for the refueling boom 18 as the boom 18 travels toward engagement with the receptacle 13 of FIG. 1 .
- the trajectory planning system 11 includes the above-noted ECU 50 , which in turn is in wired or wireless/remote communication with the camera(s) 20 .
- Each camera 20 in turn may be embodied as rearward-facing monocular cameras of an application-specific spectral capability.
- the camera(s) 20 could be configured to collect the real-time image data 200 in a human-visible/red-green-blue (“RGB”) spectral range, or using near-infrared (“near-IR”), infrared (“IR”), or other portions of the electromagnetic spectrum.
- RGB human-visible/red-green-blue
- IR infrared
- the boom resolver 22 for its part acts as an additional data source.
- the boom resolver 22 may be configured as a rotary position sensor or encoder configured to measure an angular position of the refueling boom 18 of FIG. 1 in free space, and to report the measured position to the ECU 50 as boom position data 220 .
- the boom operators 14 suitably informed by the real-time image data 200 and the boom position data 220 , then impart electronic control signals (CC 50 ) to the ECU 50 using a human-machine interface (“HMI”) device 60 , which likewise is in wired or wireless communication with the ECU 50 .
- HMI human-machine interface
- Constituent two-dimensional (“2D”) image frames of the real-time image data 200 contain therein digital pixel images of at least the receiver 12 and the refueling boom 18 .
- the trajectory planning system 11 of FIG. 2 ultimately outputs a real-time trajectory graphic (CC T ) indicative of the planned trajectory, e.g., a suitable graphical overlay, annotation, and/or text message, to the HMI device 60 to assist the boom operators 14 in visualizing and flying the boom 18 to the receptacle 13 shown in FIG. 1 .
- the ECU 50 ultimately transmits boom control signals (arrow CC 19 ) by-wire to the flight control surfaces 19 of the boom 18 illustrated in FIG. 1 .
- the boom operators 14 could accomplish this task aided refueling by the trajectory graphic (CC T ) in one or more embodiments, for instance as trajectory guidelines forming a recommended directional “roadmap” or virtual runway leading to the receptacle 13 , while fully autonomous control of the boom 18 could use the calculated boom trajectory in a different manner, e.g., as a path planning control input.
- the trajectory graphic CC T
- fully autonomous control of the boom 18 could use the calculated boom trajectory in a different manner, e.g., as a path planning control input.
- the HMI device 60 of FIG. 2 in one or more embodiments could include, by way of example and not of limitation, one or more high-resolution display screens (“Displays”) 600 controllable via display input signals (CC 60 ), e.g., touch-sensitive screens, as well as keyboards, joysticks, dials, etc.
- the electronic control signals (CC 50 ) from the HMI device 60 will cause the ECU 50 , using a processor 52 and memory 54 , to electronically control a corresponding attitude of the flight control surfaces 19 of FIG. 1 via the boom control signals (CC 19 ).
- the refueling boom 18 is controllable in a fly-by-wire manner such that a physical kinematic chain does not exist between the boom operators 14 and the boom 18 of FIG. 1 .
- Other control implementations may be envisioned within the scope of the disclosure, including semi-autonomous or fully-autonomous control implementations, and therefore the present solutions are not limited to crewed aerial refueling operations as described herein.
- the ECU 50 depicted in FIG. 2 operates herein as a trajectory planning controller, and may be optionally embodied as one or more computer systems configured to execute computer-readable instructions embodying the method 50 M, a non-limiting exemplary embodiment of which is described below with reference to FIG. 4 .
- the processor(s) 52 may be implemented as a microcontroller, one or more Application Specific Integrated Circuit(s) (ASICs), Field-Programmable Gate Array (FPGAs), electronic circuits, central processing units (CPUs), etc.
- the memory 54 in turn includes associated transitory and non-transitory memory/storage component(s), e.g., read only memory, programmable read only memory, solid-state memory, random access memory, optical and/or magnetic memory, etc.
- Computer-readable instructions embodying the method 50 M may be recorded in the memory 54 and executed by the processor(s) 52 , e.g., as machine-readable code/instructions, software, and/or firmware programs.
- Execution of the method 50 M as set forth herein also requires, in one or more embodiments, one or more 3D models 55 , including a 3D model 55 -B (hereinafter referred to as a “3D boom model”) of the refueling boom 18 and a 3D model 55 -R (hereinafter a “receiver boundary model”) of one or more designated “keep out” zones around the receiver 12 of FIGS. 1 and 5 C , the alternatively constructed receiver 120 modeled in FIGS. 5 A and 5 B , or a receiver aircraft of a different construction than those exemplified herein, e.g., another tanker 10 or an aircraft lacking the canopy 15 .
- 3D model 55 -B hereinafter referred to as a “3D boom model” of the refueling boom 18
- a 3D model 55 -R hereinafter a “receiver boundary model” of one or more designated “keep out” zones around the receiver 12 of FIGS. 1 and 5 C
- a silhouette 40 of a model of the receiver 12 is shown relative to the boom tip E 1 of the refueling boom 18 shown in FIG. 1 , with the refueling boom 18 depicted in FIG. 3 as a ray 18 R pointing in the direction of the tanker 10 , i.e., along a logitudinal axis of the boom 18 .
- the boom operators 14 of FIG. 2 would ordinarily determine an optimal trajectory. Quite often, this includes resorting to the aforementioned line-of-sight/straight line trajectory planning strategy.
- APFs are artificial forces characterized as being either attractive or repulsive.
- an APF can be expressed as:
- U ⁇ ( x , y ) Ua ⁇ ( x , y ) + Ur ⁇ ( x , y )
- Ua(x,y) and Ur(x,y) are the attractive potential and the repulsive potential, respectively.
- the attractive potential Ua(x,y) in turn is defined as:
- Ua ⁇ ( x , y ) Ka * ln ⁇ ( d ⁇ ( x , y , xg , yg ) )
- (xg, yg) are the corresponding coordinates of the goal point, i.e., the point in space corresponding to the receiver 12
- d(x, y, xg, yg) represents the Euclidean distance between the current position (x, y) of the boom tip E 1 and the goal point.
- Ka is a positive constant that determines the strength of the attractive APF.
- the repulsive potential Ur(x, y) is defined as follows:
- Ur ⁇ ( x , y ) Kr * 1 d ⁇ ( x , y , xi , yi ) 2
- (xi, yi) are the corresponding coordinates of the i th obstacle
- d(x, y, xi, yi) is the Euclidean distance between the current position (x, y) of the boom tip E 1 and the i th obstacle
- Kr is a positive constant analogous to the above-noted constant Ka, which in this case determines the strength of the repulsive force.
- the gradient of the APF can be used to compute the direction of the next move of the refueling boom 18 of FIG. 1 , with the trajectory planning system 11 of FIG. 2 iteratively updating the current position of the boom tip E 1 based on the gradient of the APF until the goal point/receptacle 13 is reached, or until reaching a maximum number of iterations.
- Attactive force field lines 42 for an applied attractive APF are depicted radiating toward the receptacle 13 , or rather a point P 13 coinciding therewith.
- Repulsive forces are shown as repulsive force field lines 41 radiating away from obstacles disposed between the boom tip E 1 (point P 18 ) or other structure of the refueling boom 18 and the receptacle 13 , with the refueling boom 18 represented as the above-noted ray 18 R.
- the obstacle during an A3R operation may include the canopy 15 when present on the receiver 12 , e.g., a transparent bubble canopy enclosing a cockpit of the receiver 12 as appreciated in the art.
- the dome-like shape of the canopy 15 could project into a line-of-sight trajectory between the boom tip E 1 and the receptacle 13 , and thus increases the risk of a collision with the refueling boom 18 .
- the various APFs are summed together and thereafter used to shape the planned trajectory 45 .
- an embodiment of the method 50 M is illustrated as a series of flow steps or logic blocks for illustrative simplicity.
- the term “block” refers to algorithms or computer-executable code segments, sequences, or other constituent portions of the method 50 M.
- the individual logic blocks are implemented by the ECU 50 of FIG. 2 to perform the method 50 M.
- the method 50 M includes receiving the real-time image data 200 from the camera(s) 20 of FIGS. 1 and 2 via the ECU 50 , as well as receiving the boom position data 220 from the boom resolver 22 .
- the real-time image data 200 as contemplated herein includes the constituent digital image frames of a video feed from the camera(s) 20 .
- the camera(s) 20 may include one or more rearward-facing monocular cameras as noted above.
- the camera(s) 20 may be part of an aircraft-mounted vision system providing the boom operators 14 of FIG. 2 with a high-definition real-time view of the receiver 12 and the refueling boom 18 , and allowing the boom operators 14 to control motion of the boom 18 using the live/real-time image feed.
- the ECU 50 uses programmed vision and image processing capabilities to determine respective positions P 12 and P 18 of the receiver 12 and the refueling boom 18 in free space aft of the tanker 10 , i.e., as corresponding 3D coordinate sets on the 3D receiver boundary model 55 -R and the 3D boom model 55 -B.
- the method 50 M proceeds to block B 54 once the ECU 50 has ascertained the positions P 12 and P 18 .
- Block B 54 (“Closest Point”) entails processing the real-time image data 200 and the boom position data 220 from block B 52 and accessing the 3D models 55 , i.e., the 3D reciever boundary model 55 -R and the 3D boom model 55 -B, with the ECU 50 identifying the closest point of the 3D models 55 using this information.
- block B 54 includes calculating a boom-to-receiver (“bTr”) relative position using the 3D receiver boundary model 55 -R and the 3D boom model 55 -B to determine the closest point on the 3D receiver boundary model 55 -R to the 3D boom model 55 -B, e.g., using a geometric calculation based on the predetermined geometries and a common coordinate system of the respective 3D receiver boundary and boom models 55 -R and 55 -B.
- Block B 54 would also entail calculating the closest point on the boom model 55 -B to the receiver boundary model 55 -R.
- the 3D receiver boundary model 55 -R may be embodied as a 3D solid or digital wire model, e.g., a computer-aided design (“CAD”) model, that the ECU 50 references when identifying the closest points in block B 54 .
- the 3D receiver boundary model 55 -R envelops external structural features of the receiver 12 or airspace around the receiver 12 to ensure collision-free guidance of the refeling boom 18 during the A3R process. Such points are potential contact locations to be avoided.
- the 3D receiver boundary model 55 -R may be a precise representation of all external surfaces of the receiver 12 , e.g., as shown in FIG. 5 A for a representative receiver 120 having different configuration than the receiver 12 of FIG. 1 .
- a simpler/coarser but faster-executing 3D receiver boundary model 155 -R of the receiver 120 could be used.
- Such a 3D receiver boundary model 155 -R may be embodied as a simplified mesh representation that encloses the outer dimensions or full envelope of the actual receiver 12 and additional surrounding space, i.e., designated “keep out zones” 56 around the receiver 12 .
- the 3D receiver boundary model 55 -R of FIG. 5 A may be a complex or multi-faceted model having upwards of 40,000 facets, while the simplified mesh embodiment of FIG. 5 B could be implemented with as fewer than about 500 facets, e.g., 200-300 facets.
- a receiver boundary model 255 -R defines boundaries around the receiver 12 where the refueling boom 18 of FIG. 1 is not permitted to go.
- the receiver boundary model 255 -R could be implemented as a triangulated mesh with normal vectors that point away from designated keep-out zones, which in FIG. 5 C are implemented as keep-out zones 560 fully surrounding the receiver 12 . As depicted in FIG.
- a plane 59 separates the keep-out zones 560 from other airspace (“Free Space”) 58 in which the refueling boom 18 of FIG. 1 is permitted to freely maneuver.
- Penetrating the plane 59 is a hollow pyramid 158 having a terminal point 158 P at the receptacle 13 .
- the pyramid 158 clears a space that coincides with the volume in which the boom 18 can maneuver while coupled to the receiver 12 in the defined disconnect envelope.
- a highly detailed 3D model of the receiver 12 shown in FIG. 5 C solely for context, is not required to be part of the receiver boundary model 255 -R. This same concept can be applied to an enclosed volume (not shown).
- the receiver boundary model 255 -R may be described by very few facets or triangles, e.g., 10-20, which is computationally advantageous for the present guidance law.
- FIGS. 5 A, 5 B, and 5 C represent different levels of detail possible within the scope of the disclosure.
- the 3D receiver boundary models 55 -R, 155 -R, or 255 -R thus could be augmented to define the keep-out zones 56 or 560 to as great of a degree, and to thus shape the eventual boom trajectory as desired.
- the 3D receiver models 55 -R, 155 -R, or 255 -R could encompass a bubble or bump over exposed canopy 150 , antenna locations (not shown), or between vertical stabilizers 170 as shown in FIG. 5 A .
- the 3D receiver boundary models 155 -R and 155 -R may be constructed as a virtual fence around the receiver 120 that, due to implementation of the method 50 M, will not be penetrated by the refueling boom 18 of FIG. 1 during the A3R process.
- the method 50 M thereafter proceeds to block B 56 .
- the ECU 50 next begins to calculate a planned trajectory between the boom tip E 1 and the receptacle 13 of the receiver 12 using the aforementioned bTr relative position and predictive artificial potential fields (“APFs”) as noted above. In doing so, the ECU 50 estimates the size and direction of net forces from such APFs.
- APFs can be used for path planning of a robot end-effector by defining a scalar potential function that describes the attraction or repulsion of the end-effector, or other links in the kinematic chain, to different locations or objects in the robot's environment.
- the typical approach is modified in the present control scheme in which the robot in question is the refueling boom 18 , the end-effector being controlled is the boom tip E 1 , and the motion target or goal is the receptacle 13 .
- Obstacles in this use scenario include prior-identified external surfaces of the receiver 12 that could at times be present in a line-of-sight between the boom tip E 1 and the receptacle 13 .
- the goal in this particular case is represented by a local minimum of the above-noted potential function, while the obstacles are represented by local maxima. Motion of the refueling boom 18 is thus guided via operation of the ECU 50 by the gradient of the potential function, with the boom 18 ultimately moving in the direction of steepest descent towards the goal while avoiding any predefined obstacles.
- the method 50 M then proceeds to block B 58 .
- the ECU 50 of FIG. 2 next uses the 3D boom model 55 -B to estimate the particular direction and distance that the refueling boom 18 would move, assuming that the APFs were to be applied as an actual net force.
- the ECU 50 then updates the orientation or pose of the 3D boom model 55 -B by moving the boom tip E 1 over the same estimated distance in the direction of the estimated net forces.
- the method 50 M then proceeds to block B 60 .
- the ECU 50 repeats blocks B 54 , B 56 , and B 58 until this condition has been satisfied.
- the method 50 M then proceeds to block B 62 .
- the ECU 50 records the preceeding movement sequence as a baseline trajectory in memory 54 .
- the method 50 M thereafter proceeds to block B 64 .
- Block B 64 (“Set G TEMP ”) includes finding the point on the baseline tajectory from block B 60 that is farthest from the line-of-sight or straight line between the receptacle 13 and the boom tip E 1 at the latter's current position. This point is then recorded in memory 54 of the ECU as a temporary goal (G TEMP ). The method 50 M then returns to block B 54 and proceeds at the same time to block B 66 .
- Block B 66 (“Motion Control of ( 18 )”) entails inputting the temporary goal from block B 62 to existing control laws used to govern motion of the refueling boom 18 , i.e., the speed, pitch, yaw, roll, and X, Y, and Z coordinate motion commands ultimately regulating the flight control surfaces 19 as appreciated in the art.
- block B 66 includes moving the boom tip E 1 to the temporary goal of block B 64 as a control action.
- FIGS. 6 and 7 exemplary trajectory planning results of the method 50 M described above are shown relative to those provided using existing APF techniques.
- typical APF approaches would result in an initial trajectory ⁇ right arrow over (Fw) ⁇ * between a starting position and a goal, or points P 18 and P 13 , respectively.
- Line-of-sight path planning would follow a straight line (LL) through multiple waypoints 70 , with each waypoint 70 having a correspondance distance to the line-of-sight or straight line (LL) between points P 18 and P 13 .
- execution of instructions embodying the method 50 M as described results in a new temporary goal (P 13, TEMP ) as the farthest point from the straght line (LL).
- P 13, TEMP a new temporary goal
- the farthest point from the striaght line (LL) is found at distance da.
- This waypoint 70 becomes the new temporary target (P 13, TEMP ), which in turn changes the trajectory from a straightline trajectory, i.e., initial trajectory ⁇ right arrow over (F W ) ⁇ *, to a revised trajectory ⁇ right arrow over (F W ) ⁇ .
- the resulting trajectory path from typical APF teachiniques i.e., trace T 1 (APF)
- trace T 1 APF
- APF P resulting trajectory path
- the trajectory path T 2 (APF P ) from a qualitative perspective is more efficient in its motion relative to trace T 1 (APF).
- the practical result of the present teachings is a reduced risk of collision between the refeuling boom 18 and the receiver 12 during the A3R process shown in FIG. 1 .
- Performance of the method 50 M by the trajectory planning system 11 of FIGS. 1 and 2 is computationally efficient relative to competing strategies.
- the application of APFs in the course of trajectory planning can result in certain undesirable outcomes. Such outcomes are avoided by aspects of the method 50 M as described above.
- the refueling boom 18 could become stuck in local minima, or may oscillate in narrow passages around defined boundaries of the obstacles.
- the ECU 50 may avoid becoming stuck in local minima by applying a rotating APF. This action could entail adding a tangential component to the repulsive forces to perturb the boom tip E 1 out of the local minima, e.g., whenever an obstacle is present directly between the refueling boom 18 and the receptacle 13 .
- the ECU 50 finds the closest point on the refueling boom 18 to the straight line (LL) and calculating the applied forces at that point.
- the problem of oscillation near an obstacle boundary may be resolved by adding a predictive feature to the calculations.
- the ECU 50 may find the traditional tajectory via APF and thereafter set the farthest deviation from the line-of-sight to the goal as the temporary target. This action is repeated until the boom tip E 1 reaches the receptacle 13 , as described above with reference to FIG. 4 .
- Use of the method 50 M thus produces a shorter path to the receptacle 13 than does traditional APF, which can be seen by comparing traces T 1 (APF) and T 2 (APF P ) of FIG. 7 .
- the method 50 M applied herein considers the fact that the refueling boom 18 is both an extended surface and constrained by virtue of being attached to the tanker 10 , and thus modeled as the ray 18 R in FIG. 3 .
- a trajectory planning system for use with a fuel-supplying aircraft (“tanker”) having a refueling boom configured for delivering aviation fuel to a fuel-receiving aircraft (“receiver”) comprising: a human-machine interface (“HMI”) device configured to output electronic control signals; and an electronic control unit (“ECU”) in communication with the HMI device and a plurality of sensors operable for outputting sensor data, wherein the ECU is configured, in response to the electronic control signals and the sensor data, to: access a three-dimensional (“3D”) receiver boundary model and a 3D boom model of the receiver and the refueling boom, respectively, wherein the 3D receiver boundary model includes one or more receiver features; calculate a boom-to-receiver (“bTr”) relative position using the 3D receiver boundary model, the 3D boom model, and the sensor data; calculate a planned trajectory between a boom tip of the refueling boom and a receptacle of the receiver using the bTr relative position and predictive artificial potential fields (“
- Clause 2 The trajectory planning system of clause 1, further comprising the plurality of sensors, wherein the sensors include a rearward-facing camera and a boom resolver respectively configured to output real-time image data and boom position data as the sensor data.
- the sensors include a rearward-facing camera and a boom resolver respectively configured to output real-time image data and boom position data as the sensor data.
- Clause 3 The trajectory planning system of clauses 1 or 2, wherein the 3D receiver boundary model is a multi-faceted model of the receiver providing a simplified representation thereof in fewer than about 500 facets.
- Clause 4 The trajectory planning system of any of clauses 1-3, wherein the ECU is configured to output a real-time trajectory graphic to the HMI device, the real-time trajectory graphic being indicative of the planned trajectory.
- Clause 5 The trajectory planning system of any of clauses 1-4, wherein the ECU is configured to calculate the bTr relative position as corresponding coordinate sets on the 3D receiver boundary model and the 3D boom model.
- Clause 6 The trajectory planning system of any of clauses 1-5, wherein the receiver features include one or more designated keep out zones surrounding the receiver.
- Clause 7 The trajectory planning system of any of clauses 1-6, wherein the receiver includes a canopy, and wherein the receiver features include the canopy.
- Clause 8 The trajectory planning system of any of clauses 1-7, wherein the ECU is configured to calculate the planned trajectory of the boom tip at least in part by assuming the APFs are applied to the refueling boom as a net force, and then determining a predicted distance and a predicted direction of movement of the refueling boom resulting from the net force, and wherein the ECU is configured to update a pose of the refueling boom in the 3D boom model by moving the boom tip through the predicted distance in the predicted direction.
- a method for determining a planned trajectory of a boom tip of a refueling boom connected to a fuel-supplying aircraft (“tanker”) when delivering aviation fuel to a fuel-receiving aircraft (“receiver”) comprising: accessing a three-dimensional (“3D”) receiver boundary model and a 3D boom model of the receiver and the refueling boom, respectively, via an electronic control unit (“ECU”), wherein the 3D receiver boundary model includes one or more predetermined receiver features; calculating a boom-to-receiver (“bTr”) relative position via the ECU using the 3D receiver boundary model, the 3D boom model, and sensor data from a plurality of sensors; calculating the planned trajectory between the boom tip and a receptacle of the receiver using the bTr relative position and predictive artificial potential fields (“APFs”), including: finding a point on a baseline trajectory of the refueling boom that is farthest from a straight line between the receptacle and the boom; recording the
- Clause 10 The method of clause 9, wherein the sensors include at least one rearward-facing camera and a boom resolver, the method further comprising: receiving real-time image data from the at least one rearward-facing camera; and receiving boom position data from the boom resolver.
- Clause 11 The method of clauses 9 or 10, wherein accessing the 3D receiver boundary model includes accessing a multi-faceted model providing a simplified representation of the receiver, the simplified representation of the receiver including fewer than about 500 facets.
- Clause 12 The method of any of clauses 9-11, further comprising: outputting a real-time trajectory graphic to a human-machine interface (“HMI”) device via the ECU, the real-time trajectory graphic being indicative of the planned trajectory.
- HMI human-machine interface
- Clause 13 The method of any of clauses 9-12, wherein calculating the bTr relative position includes calculating corresponding coordinate sets on the 3D receiver boundary model and the 3D boom model.
- Clause 14 The method of any of clauses 9-13, further comprising: recording the one or more predetermined receiver features in the 3D receiver boundary model in memory of the ECU as one or more keep out zones surrounding the receiver.
- Clause 15 The method of any of clauses 9-14, wherein recording the one or more predetermined receiver features in the 3D receiver boundary model includes recording an area of a canopy of the receiver as at least one of the predetermined receiver features.
- Clause 16 The method of any of clauses 9-15, further comprising: assuming via the ECU that the APFs are applied to the refueling boom as a net force, calculating the planned trajectory at least in part by determining a predicted distance and a predicted direction of movement of the refueling boom; and updating a pose of the refueling boom in the 3D boom model by moving the boom tip over the predicted distance in the predicted direction.
- a tanker comprising: a fuselage configured to transport aviation fuel; a refueling boom connected to the fuselage; a boom resolver connected to the fuselage, wherein the boom resolver is configured to measure a position of the refueling boom and output boom position signals indicative thereof; a camera mounted to the fuselage and configured to output real-time image data of the refueling boom and a fuel-receiving aircraft (“receiver”); and a trajectory planning system aboard the fuselage, comprising: a human-machine interface (“HMI”) device configured to output electronic control signals; and an electronic control unit (“ECU”) in communication with the HMI device, the boom resolver, and the camera, wherein the ECU is configured, in response to the electronic control signals, the boom position signals, and the real-time image data, to: access a three-dimensional (“3D”) receiver boundary model a 3D boom model of the receiver and the refueling boom, respectively, wherein the 3D receiver boundary model includes one or more predetermined receiver features; calculate a
- Clause 18 The tanker of clause 17, wherein the 3D receiver model is a multi-faceted model providing a simplified representation of the receiver, the simplified representation including fewer than about 500 facets.
- Clause 19 The tanker of clause 17 or 18, wherein the ECU is configured to output a real-time trajectory graphic to the HMI device that is indicative of the planned trajectory.
- Clause 20 The tanker of any of clauses 17-19, wherein the ECU is configured to assume the APFs were to be applied to the refueling boom as a net force, and then calculate the planned trajectory at least in part by determining a predicted distance and a predicted direction of movement of the refueling boom caused by the net force, and wherein the ECU is configured to update a pose of the refueling boom in the 3D boom model by moving the boom over the predicted distance in the predicted direction.
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Aviation & Aerospace Engineering (AREA)
- Jib Cranes (AREA)
Abstract
Description
where Ua(x,y) and Ur(x,y) are the attractive potential and the repulsive potential, respectively. The attractive potential Ua(x,y) in turn is defined as:
where (xg, yg) are the corresponding coordinates of the goal point, i.e., the point in space corresponding to the receiver 12, and d(x, y, xg, yg) represents the Euclidean distance between the current position (x, y) of the boom tip E1 and the goal point. In the above equation, Ka is a positive constant that determines the strength of the attractive APF. Similarly, the repulsive potential Ur(x, y) is defined as follows:
Claims (20)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US18/172,701 US12486040B2 (en) | 2023-02-22 | 2023-02-22 | Artificial potential field-based boom guidance for automated air-to-air refueling |
| US19/380,598 US20260062139A1 (en) | 2023-02-22 | 2025-11-05 | Artificial potential field - based boom guidance for automated air-to-air refueling |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US18/172,701 US12486040B2 (en) | 2023-02-22 | 2023-02-22 | Artificial potential field-based boom guidance for automated air-to-air refueling |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US19/380,598 Continuation US20260062139A1 (en) | 2023-02-22 | 2025-11-05 | Artificial potential field - based boom guidance for automated air-to-air refueling |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20240278926A1 US20240278926A1 (en) | 2024-08-22 |
| US12486040B2 true US12486040B2 (en) | 2025-12-02 |
Family
ID=92304820
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US18/172,701 Active 2044-03-20 US12486040B2 (en) | 2023-02-22 | 2023-02-22 | Artificial potential field-based boom guidance for automated air-to-air refueling |
| US19/380,598 Pending US20260062139A1 (en) | 2023-02-22 | 2025-11-05 | Artificial potential field - based boom guidance for automated air-to-air refueling |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US19/380,598 Pending US20260062139A1 (en) | 2023-02-22 | 2025-11-05 | Artificial potential field - based boom guidance for automated air-to-air refueling |
Country Status (1)
| Country | Link |
|---|---|
| US (2) | US12486040B2 (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019172409A (en) * | 2018-03-27 | 2019-10-10 | 株式会社タダノ | Work vehicle |
| US20200039076A1 (en) * | 2016-03-04 | 2020-02-06 | Ge Global Sourcing Llc | Robotic system and method for control and manipulation |
| US20200086486A1 (en) * | 2018-09-13 | 2020-03-19 | Pilz Gmbh & Co. Kg | Method and apparatus for collision-free motion planning of a manipulator |
| US20210041243A1 (en) * | 2019-08-06 | 2021-02-11 | Boston Dynamics, Inc. | Intermediate Waypoint Generator |
| US20210039804A1 (en) * | 2019-08-07 | 2021-02-11 | Rockwell Collins, Inc. | Relative Navigation and Machine Vision for Automated Aerial Refueling System and Method |
| US20250018238A1 (en) * | 2021-11-22 | 2025-01-16 | Hale Products, Inc. | Fire-fighting system |
-
2023
- 2023-02-22 US US18/172,701 patent/US12486040B2/en active Active
-
2025
- 2025-11-05 US US19/380,598 patent/US20260062139A1/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200039076A1 (en) * | 2016-03-04 | 2020-02-06 | Ge Global Sourcing Llc | Robotic system and method for control and manipulation |
| JP2019172409A (en) * | 2018-03-27 | 2019-10-10 | 株式会社タダノ | Work vehicle |
| US20200086486A1 (en) * | 2018-09-13 | 2020-03-19 | Pilz Gmbh & Co. Kg | Method and apparatus for collision-free motion planning of a manipulator |
| US20210041243A1 (en) * | 2019-08-06 | 2021-02-11 | Boston Dynamics, Inc. | Intermediate Waypoint Generator |
| US20210039804A1 (en) * | 2019-08-07 | 2021-02-11 | Rockwell Collins, Inc. | Relative Navigation and Machine Vision for Automated Aerial Refueling System and Method |
| US20250018238A1 (en) * | 2021-11-22 | 2025-01-16 | Hale Products, Inc. | Fire-fighting system |
Non-Patent Citations (6)
Also Published As
| Publication number | Publication date |
|---|---|
| US20260062139A1 (en) | 2026-03-05 |
| US20240278926A1 (en) | 2024-08-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10218893B2 (en) | Image capturing system for shape measurement of structure, method of capturing image of structure for shape measurement of structure, on-board control device, remote control device, program, and storage medium | |
| Mammarella et al. | Machine vision/GPS integration using EKF for the UAV aerial refueling problem | |
| WO2018053861A1 (en) | Methods and system for vision-based landing | |
| KR101740312B1 (en) | Induction control method using camera control information of unmanned air vehicle | |
| US20240076066A1 (en) | Fixed-wing unmanned aerial vehicle capable of high angle-of-attack maneuvering | |
| US12292742B2 (en) | Information processing method and information processor | |
| Nieuwenhuisen et al. | Search-based 3d planning and trajectory optimization for safe micro aerial vehicle flight under sensor visibility constraints | |
| US10223922B2 (en) | Mission parameterization system | |
| US11900633B2 (en) | Vehicle pose determination systems and methods | |
| Lee et al. | Obstacle avoidance using image-based visual servoing integrated with nonlinear model predictive control | |
| Mebarki et al. | Autonomous landing of rotary-wing aerial vehicles by image-based visual servoing in GPS-denied environments | |
| US12486040B2 (en) | Artificial potential field-based boom guidance for automated air-to-air refueling | |
| US11320268B2 (en) | Construction method of a join flight path to a mobile point, associated join method to a mobile point, computer program product and module | |
| Kucherov et al. | Model of UAV as agent of multiagent system | |
| US20250319987A1 (en) | Boom guidance system for automated air-to-air refueling | |
| US20250346360A1 (en) | System and method for breakaway during air-to-air refueling | |
| Edwards et al. | A vision system for precision MAV targeted landing | |
| Kong et al. | Real-time stereo-vision localization system for safe landing of unmanned aerial vehicles | |
| US20250147515A1 (en) | Systems and methods for controlling aircraft during in-flight refueling | |
| US12234028B2 (en) | Automated air-to-air refueling using keypoint-based perception of aircraft flight control surfaces | |
| Hwangbo et al. | Maneuver-based autonomous navigation of a small fixed-wing UAV | |
| US12087017B2 (en) | Camera calibration optimization using image segmentation and boom resolvers for automated air-to-air refueling | |
| KR102013358B1 (en) | Unmanned aerial vehicles flight control system of BVLOS | |
| Martín et al. | Tanker uav for autonomous aerial refueling | |
| Olivares-Mendez et al. | Vision based fuzzy control approaches for unmanned aerial vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: THE BOEING COMPANY, VIRGINIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:O'SHEA, DANIEL MICHAEL;CHAMBERLAIN, SHAWN M.;REEL/FRAME:062769/0952 Effective date: 20230222 |
|
| FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS Free format text: ALLOWED -- NOTICE OF ALLOWANCE NOT YET MAILED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
| STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |