EP3365866A1 - System adapted for providing an operator with augmented visibility and associated method - Google Patents
System adapted for providing an operator with augmented visibility and associated methodInfo
- Publication number
- EP3365866A1 EP3365866A1 EP16787785.1A EP16787785A EP3365866A1 EP 3365866 A1 EP3365866 A1 EP 3365866A1 EP 16787785 A EP16787785 A EP 16787785A EP 3365866 A1 EP3365866 A1 EP 3365866A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- digital image
- resolution
- sensor
- spectral band
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 24
- 230000003190 augmentative effect Effects 0.000 title abstract description 7
- 230000003595 spectral effect Effects 0.000 claims abstract description 49
- 238000012545 processing Methods 0.000 claims abstract description 41
- 238000001914 filtration Methods 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000006073 displacement reaction Methods 0.000 claims description 4
- 210000000056 organ Anatomy 0.000 claims 1
- 238000004364 calculation method Methods 0.000 abstract description 5
- 238000013459 approach Methods 0.000 description 7
- 230000000717 retained effect Effects 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 3
- 238000010521 absorption reaction Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- RZVHIXYEVGDQDX-UHFFFAOYSA-N 9,10-anthraquinone Chemical compound C1=CC=C2C(=O)C3=CC=CC=C3C(=O)C2=C1 RZVHIXYEVGDQDX-UHFFFAOYSA-N 0.000 description 1
- 101100494773 Caenorhabditis elegans ctl-2 gene Proteins 0.000 description 1
- 101100112369 Fasciola hepatica Cat-1 gene Proteins 0.000 description 1
- 101100005271 Neurospora crassa (strain ATCC 24698 / 74-OR23-1A / CBS 708.71 / DSM 1257 / FGSC 987) cat-1 gene Proteins 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005670 electromagnetic radiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 229910044991 metal oxide Inorganic materials 0.000 description 1
- 150000004706 metal oxides Chemical class 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000011282 treatment Methods 0.000 description 1
- 238000007794 visualization technique Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
- G06T3/4061—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution by injecting details from different spectral ranges
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0017—Arrangements for implementing traffic-related aircraft activities, e.g. arrangements for generating, displaying, acquiring or managing traffic information
- G08G5/0021—Arrangements for implementing traffic-related aircraft activities, e.g. arrangements for generating, displaying, acquiring or managing traffic information located in the aircraft
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/0047—Navigation or guidance aids for a single aircraft
- G08G5/0065—Navigation or guidance aids for a single aircraft for taking-off
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/02—Automatic approach or landing aids, i.e. systems in which flight data of incoming planes are processed to provide landing data
- G08G5/025—Navigation or guidance aids
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G5/00—Traffic control systems for aircraft, e.g. air-traffic control [ATC]
- G08G5/06—Traffic control systems for aircraft, e.g. air-traffic control [ATC] for control when on the ground
- G08G5/065—Navigation or guidance aids, e.g. for taxiing or rolling
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0147—Head-up displays characterised by optical features comprising a device modifying the resolution of the displayed image
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
Definitions
- the present invention relates to a system adapted to provide an operator with increased visibility, which can be used in particular for assisting aircraft control, as well as an associated method.
- the invention is in the field of augmented vision systems (EVS), which are imaging systems intended to provide an operator with an image of the environment improved by compared to a human perception, this image can be presented through a display head up (or HUD for "head-up display” in English) or on a display screen (display called “head down”).
- EVS augmented vision systems
- EVS systems have applications in particular in the field of aircraft piloting assistance, especially in the approach and landing phase, but also taxiing and takeoff, in case of low visibility due to environmental conditions and / or degraded weather.
- Runaway Visual Range is defined as the distance up to which an aircraft pilot, placed in the centreline of the runway, can see by his natural vision the marks or the lights which delimit the track or which mark the axis of this one.
- the RVR is generally evaluated by an automatic calculation integrating the instrumental measurements relating to the transmission coefficient of the atmosphere and the background luminance and information on the intensity of the lighting.
- the regulation imposes a minimum RVR value of 550 meters to initiate an approach, other requirements have to be met otherwise, and the regulation allows to descend to a decision height (DH) at least 200ft (200ft), the height at which the pilot must discern visual references to descend below the DH decision height.
- DH decision height
- 200ft 200ft
- Such a visual range is difficult to obtain in certain degraded meteorological conditions which can make the lighting markings undetectable by the pilot at the decision height DH.
- EVS systems have been designed, in particular to remedy this problem and improve the natural vision of flight crews and extend landing capabilities in conditions of degraded visibility.
- the regulation allows, in the example cited above, to go down to 10Oft if the necessary visual references could be discerned by the pilot at 200ft with the help of an EVS and even if this is not the case. not discernible by the human eye.
- EVS systems comprising image sensors in the infrared spectral band using the spectral bands of 3 to 5 ⁇ or 8 to 14 ⁇ and in the spectral band SWIR for "Short Wave Infra Red" of signals electromagnetic wave which extends from 1 ⁇ to 2.5 ⁇ .
- the purpose of using sensors in the SWIR spectral band is to optimize the detection capabilities of incandescent lamps commonly used to mark tracks.
- LED light emitting diode
- the patent application WO 2009/128065 A1 describes an EVS system comprising a plurality of sensors able to operate in various spectral bands, comprising the NIR spectral band for "Near Infra-Red" of electromagnetic signals of wavelength extending from 0.7 ⁇ at 1 .0 ⁇ , and the spectral band of visible light which extends from 0.4 ⁇ to 0.7 ⁇ .
- This system mergers the image data acquired by the different sensors.
- the spectral bands to be fused are selected according to the previously identified meteorological conditions and the nature of the light markings to be detected.
- the angular resolution is defined as being the elementary field of view of a pixel of an image sensor detector. It is generally considered that the angular resolution of the human eye is about 0.8 arc minutes, or 0.0135 ° (or 0.00029 radians). Likewise in what follows, a high spatial resolution will designate a fine angular resolution, thus a small resolution angle / field of view.
- the object of the invention is to overcome the disadvantages of the prior art mentioned above.
- the invention proposes, according to a first aspect, a system adapted to provide an operator with increased visibility, intended for the purpose of piloting an aircraft, comprising at least one sensor capable of acquiring image data. in a given spectral band and a central processing unit able to process the acquired image data and to transmit the processed image data for displaying a digital image on a display unit.
- This system comprises:
- At least one high-resolution sensor capable of acquiring image data forming a digital image in a spectral band including at least all or part of the spectral band corresponding to the electromagnetic signals visible to the human eye, of first spatial resolution, and angular resolution of capture strictly finer than the angular resolution of the human eye, and
- a non-linear processing module adapted to achieve a spatial resolution change while preserving bright spots of said acquired digital image to obtain a digital image to display second spatial resolution less than the first spatial resolution.
- the system of the invention is an EVS system using at least one sensor (orientable or fixed) of very fine angular resolution and significantly better than that of the eye in the visible spectral band, which allows:
- the proposed system is less expensive in hardware and requires less computational resources than a system based on sensors adapted to operate in several different spectral bands.
- the system according to the invention may have one or more of the following characteristics, taken independently or in all their technically acceptable combinations.
- Each digital image is defined by a matrix of pixels, each pixel having an associated value, said value being even higher when a pixel is glossy, and the nonlinear processing module is able to apply a nonlinear filtering has a block of pixels of the digital image acquired to determine a corresponding pixel value in the digital image to be displayed, said non-linear filtering taking into account, for a block of pixels of the acquired digital image, at least the maximum value said block of pixels.
- Non-linear filtering involves associating with a pixel of the digital image to display a value calculated from values above a predetermined threshold of the corresponding block of pixels of the acquired digital image.
- the nonlinear filtering involves associating with a pixel of the digital image to display a value calculated from a given number of the highest values of the block of pixels.
- the system includes a plurality of high resolution juxtaposed sensors.
- the system comprises a high-definition sensor, adapted to be positioned in a position for acquiring image data along a line of sight, and displacement members of said sensor, making it possible to move the angle of sight of this sensor.
- the high-resolution sensor capable of acquiring digital image data is a first sensor in a first spectral band corresponding to the signals visible to the human eye, the system further comprising a second sensor capable of acquiring second digital image data. in a second spectral band different from the first spectral band.
- the second spectral band belongs to the field of infrared electromagnetic waves of wavelength between 3 and 14 micrometers.
- the system further comprises an image processing module adapted to effect a merger between said digital image to be displayed and the second digital image data acquired by the second sensor.
- the angular capture resolution is strictly finer than the angular resolution of the human eye, by a factor greater than or equal to 3.
- the invention proposes a method adapted to provide an operator with increased visibility, intended for use in aircraft piloting, implemented by a system comprising at least one sensor capable of acquiring data from aircraft. image in a given spectral band and a central processing unit adapted to process the acquired image data and to transmit the processed image data for displaying a digital image on a display unit.
- the method comprises the following steps:
- the process according to the invention may have one or more of the following characteristics, taken independently or in all their technically acceptable combinations.
- Each digital image is defined by a matrix of pixels, each pixel having an associated value, said value being even higher when a pixel is shiny, and the nonlinear processing comprises the application of a nonlinear filtering a block of pixels of the acquired digital image for determining a corresponding pixel value in the digital image to be displayed, said non-linear filtering taking into account at least the maximum value of said block of pixels.
- Non-linear filtering involves associating with a pixel of the digital image to display a value calculated from values above a predetermined threshold of the corresponding block of pixels of the acquired digital image.
- the value associated with the pixel of the digital image to be displayed is calculated from a given number of the highest values of the block of pixels.
- the method includes another step of acquiring second digital image data in a second spectral band different from the first spectral band.
- the method includes a step of merging the second resolution digital image obtained by nonlinear processing with the second digital image data.
- FIG. 1 schematically represents an aircraft approaching a track marked by light markings
- FIG. 2 diagrammatically illustrates an augmented vision system according to a first embodiment
- FIG. 3 schematically illustrates two images of different spatial resolutions
- FIG. 4 schematically illustrates an augmented vision system according to a second embodiment.
- the invention finds applications more generally in any context in which an increased vision compared to the human vision of an operator is useful, for example for controlling other types of devices.
- Figure 1 schematically illustrates an application context of the invention, which is the landing of an aircraft.
- an aircraft 2 is on landing approach on a landing field 4, comprising a landing runway 6.
- the landing strip is marked by various markers 8, 10, 12, 16, 18.
- the markers 8 are markers of the center of the track
- the markers 10, 12 are light markers of the edge of the track, arranged Regularly along its entire length
- the markers 16 are runway threshold markers
- the markers 18 are approach ramp markers.
- the markers 8, 10, 12, 16, 18 emit at least in the spectral band visible to the eye by the operator and some can be made by LED lamps and others by incandescent lamps .
- the aircraft 2 is equipped with a system 14 adapted to provide the pilot with increased vision.
- system 14 is shown schematically in Figure 1, and that it is in practice made of several elements that are positioned at different locations or grouped together, as explained in more detail below.
- a system 14 comprises an image sensor 20 in the spectral band of the electromagnetic radiation or signals visible to the human eye, of wavelength between 0.4 ⁇ to 0.7 ⁇ but can in a variant go up to 1 ⁇ .
- the image sensor 20 operates in a spectral band comprising only a part of the spectral band of the electromagnetic signals visible to the human eye.
- the sensor 20 is a high resolution sensor, making it possible to obtain a level of angular resolution of shooting that is finer than the human eye.
- the digital image acquired by the sensor 20 has an associated spatial resolution, the spatial resolution being defined as the number of image data or pixels per unit length. Each pixel has an associated radiometry value, also called intensity value.
- the senor 20 is such that the ratio K between the angular resolution of the human eye and the angular image acquisition resolution is greater than or equal to 3.
- the digital image acquired by such a sensor is said on -resolved because it has angular resolution finer than the angular resolution achievable by the human eye.
- the sensor 20 makes it possible to capture, along a line of sight, a field of view of maximum angle ⁇ preferably of the order of 35 ° to 40 °.
- the senor 20 is a CMOS sensor ("Complementarity Metal-Oxide Semiconductor”), composed of photodiodes, whose manufacturing cost is moderate.
- CMOS sensor Complementarity Metal-Oxide Semiconductor
- the senor 20 is a CCD (Charge Coupled Device) or uses any other sensor technology.
- CCD Charge Coupled Device
- the sensor 20 in order to acquire image data corresponding to the angle of view field ⁇ , the sensor 20 is replaced by a plurality of angle-of-view sensors smaller than ⁇ , juxtaposed and adapted capturing image data corresponding to adjacent fields of view or having an overlapping portion.
- the high resolution sensor 20 has a field of view of angle smaller than the desired angle, thus a narrower field of view, but this sensor 20 is made movable by displacement members, allowing it to be rotated so as to cover a wide field of view of the order of ⁇ .
- displacement members are formed by an articulated or fixed connection associated with a motor.
- the sensor 20 is steerable.
- the senor 20 is placed for example at the front of the fuselage of the aircraft.
- the capture of an overbooked image makes it possible to improve the visibility of the light markings with a quantifiable performance, even in conditions of reduced visibility.
- a fog-reducing weather condition is schematically illustrated by a cloud 21 in FIG.
- a cloud 21 for example, a fog-reducing weather condition is schematically illustrated by a cloud 21 in FIG.
- an over-resolved digital image 10 of first spatial resolution R 0 is obtained, the image consisting of K * L pixels, for example 5120 * 4096.
- the digital image is defined by a matrix of pixel values.
- non-linear processing module 22 The data of the over-resolved digital image 10 , of first spatial resolution R 0 , are transmitted to a non-linear processing module 22, for example via a data bus connected to the output of the sensor 20.
- non-linear processing 22 is implemented by a not shown programmable device, for example an on-board computer, comprising one or more processors able to execute calculations and computer program code instructions when they are powered up.
- the programmable device implementing the non-linear processing module 22, as well as any other calculation module is implemented by an integrated circuit of the FPGA type or by a dedicated integrated circuit of ASIC type.
- the processing carried out by the nonlinear processing module 22 makes it possible to go from the first image 10 of the first spatial resolution R 0 to a digital image of second spatial resolution R 1; less than the first spatial resolution R 0 , while preserving contrast points of the image, in particular bright spots (or points of positive contrast) of the image.
- Points of contrast are called points or pixels whose associated value is clearly greater or substantially less than the average value of the pixels of the neighborhood, for example greater than 3 times the standard deviation of the pixels of this neighborhood.
- the points whose associated value is much higher than the surrounding values are bright points, the contrast is positive.
- the points whose associated value is significantly lower than the surrounding values are dark points, the contrast is negative.
- the processing performed by the nonlinear processing module 22 makes it possible to maintain positive contrast points in the digital image of second spatial resolution less than the first resolution.
- FIG. 3 illustrates two such images 10 and 1 with a resolution factor equal to 3 between the first resolution R 0 and the second resolution.
- a block B of 3 ⁇ 3 pixels of the digital image 10 corresponds to a pixel P of the image .
- the correspondence is a spatial correspondence in the respective matrices, as shown in Figure 3.
- a block of MxN pixels of the image 10 corresponds to a pixel of the image.
- the gain provided by the over-resolution of the image 10 acquired by the sensor 20 is preserved.
- the maximum intensity emitted by the luminous markings, captured by the over-resolved image acquisition, is preserved in the second resolution digital image.
- the non-linear processing module is adapted to retain the detected light spots, in other words to preserve the brightest points of the block, because in fact the more a point is shiny, the higher the associated value in the image digital is high.
- the applied nonlinear processing preserves the bright spots but does not preserve the dark spots, as only the brightest points are of interest for the intended piloting application.
- the nonlinear processing module 22 applies other nonlinear treatments of filtering type that retain the maximum values or that process the pixels according to their rank. Filtering on overlapping windows can also be considered.
- the values of the block pixels greater than a threshold S are retained.
- the threshold S can be set or dynamically calculated, for example, being the average increased by 2 or 3 times the standard deviation of pixel values of the block. All the values retained, which are the values of the brightest pixels of the block according to the chosen criterion, are then used to obtain the final value of the pixel of the second resolution digital image. For example, an average of the values retained, therefore greater than threshold S, of the block considered, is calculated and assigned as the final value of the corresponding pixel in the second resolution digital image
- the values of the pixels of a block considered are ordered in descending order of the values and the 2 (or 3) most important values are retained.
- the values retained which are the values of the brightest pixels of the block according to the chosen criterion, are then used to obtain the final value of the pixel of the second resolution digital image. For example, an average of the values retained. of the block considered, is calculated and assigned as the final value of the corresponding pixel in the second resolution digital image.
- the digital image of second spatial resolution Ri less than the resolution R 0 is transmitted to an image processing module 24, able to apply conventional processing, for example correction of radiometry, geometric alignment in view of the display of the image on a display unit 26, for example a screen.
- processing modules 22 and 24 are applied by the same onboard computer.
- the display is performed by incrustation on a display screen 26, located at the sight of a flight operator, called head-up display.
- the second resolution is preferably chosen according to the display resolution of the display screen 26, for example a "head-up” display.
- a “head-down” display screen for example located on the dashboard, is used.
- such a head-up display makes it possible to present to the operator an augmented vision of the reality that he can perceive naturally, and thus to help him for the piloting operations.
- a visualization method adapted to provide an operator with an augmented vision comprises a first step of acquiring a first digital image of first spatial resolution by a high resolution sensor able to acquire image data in a spectral band corresponding to the signals visible by the human eye, with a level of angular resolution of capture strictly greater than the level of angular resolution of the human eye.
- This first step is followed by a non-linear processing step making it possible to obtain a second image of second spatial resolution R 2 , less than the first spatial resolution and of second angular resolution adapted to the resolution of the display device.
- the factor between the angular resolution of capture and the angular resolution of the human eye is greater than or equal to 3, that is to say that the angular resolution of capture is at least 3 times finer than the resolution angular of the human eye.
- the performance of the proposed system is computable, regardless of weather conditions.
- a meteorological condition is characterized by an absorption coefficient ⁇ in a given spectral band.
- X 0 is a reference distance to which the intensity l 0 of a beacon is associated.
- the system of the invention is an EVS system using at least one sensor (orientable or fixed) of very fine angular resolution in a spectral band different from that of the eye, which allows to have significantly improved lighting range detection range performance compared to an EVS system having a resolution of the class of that of the eye in this same spectral band.
- FIG. 4 A second embodiment of a system 30 adapted to provide increased vision according to the invention is illustrated in FIG. 4.
- the system 30 comprises, in addition to the first sensor 20 capable of acquiring images of very high spatial resolution in the visible spectral band, and the processing unit 22 described above, constituting a first imaging channel, a second sensor 32 capable of acquire the images 2 in a different spectral band than the visible spectral range, preferably in the infrared spectral range.
- the spatial resolution of the images 1 2 acquired by the sensor 32 is substantially equal to the second spatial resolution R 2 of the images obtained at the output of the nonlinear processing module 22.
- the acquisition by this second sensor 32 forms a second imaging path.
- the system 30 also comprises a processing module 34 adapted to perform the fusion of the images and 1 2 , corresponding to the same field of view, by applying in particular, with techniques known in the field of image processing, a radiometry control on each of the channels, a geometric alignment for rendering the images and 2 superimposable and a pixel-to-pixel addition.
- the processing module 34 is also adapted to perform any image correction for display.
- the processing module 34 is implemented by a not shown programmable device, for example an on-board computer, comprising one or more processors capable of executing calculations and instructions of computer program code when they are put under control. voltage.
- the processing module 34 implements a step of merging between the second resolution digital image obtained by non-linear processing performed during the non-linear processing step by the module 22, and the second digital image data. 2 acquired by the sensor 32.
- the resulting merged image is then transmitted to a display unit 26, analogous to the display unit 26 described above with reference to FIG. 2, for example a display for overlay and display.
- the proposed system makes it possible to acquire an image of first spatial resolution which is over-resolved, which makes it possible to capture positive contrast points, that is to say points which are shiny with respect to their neighborhood, and of preserve these positive contrast points in the digital image of second spatial resolution, lower than the first resolution.
- a second resolution digital image space for display and operation is obtained, but this image includes gloss information that could not have been captured with an image acquisition at said second spatial resolution.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Aviation & Aerospace Engineering (AREA)
- Theoretical Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Optics & Photonics (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Image Processing (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR1502237A FR3042882B1 (en) | 2015-10-22 | 2015-10-22 | SYSTEM PROVIDED TO PROVIDE OPERATOR WITH INCREASED VISIBILITY AND ASSOCIATED METHOD |
PCT/EP2016/075412 WO2017068141A1 (en) | 2015-10-22 | 2016-10-21 | System adapted for providing an operator with augmented visibility and associated method |
Publications (1)
Publication Number | Publication Date |
---|---|
EP3365866A1 true EP3365866A1 (en) | 2018-08-29 |
Family
ID=55646639
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP16787785.1A Withdrawn EP3365866A1 (en) | 2015-10-22 | 2016-10-21 | System adapted for providing an operator with augmented visibility and associated method |
Country Status (7)
Country | Link |
---|---|
US (1) | US20180300856A1 (en) |
EP (1) | EP3365866A1 (en) |
CN (1) | CN108369727A (en) |
CA (1) | CA3001668A1 (en) |
FR (1) | FR3042882B1 (en) |
IL (1) | IL258758A (en) |
WO (1) | WO2017068141A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
IT201800005817A1 (en) * | 2018-05-29 | 2019-11-29 | DEVICE AND METHOD FOR VISIBILITY MEASUREMENTS | |
US10777013B1 (en) * | 2018-12-21 | 2020-09-15 | Rockwell Collins, Inc. | System and method for enhancing approach light display |
CN111746816B (en) * | 2020-05-18 | 2022-03-08 | 陈穗 | Airplane landing auxiliary device and landing method |
CN116129249B (en) * | 2023-04-04 | 2023-07-07 | 上海燧原科技有限公司 | Image processing method, device, electronic equipment and storage medium |
Family Cites Families (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4028725A (en) * | 1976-04-21 | 1977-06-07 | Grumman Aerospace Corporation | High-resolution vision system |
US4513317A (en) * | 1982-09-28 | 1985-04-23 | The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration | Retinally stabilized differential resolution television display |
US5719567A (en) * | 1995-05-30 | 1998-02-17 | Victor J. Norris, Jr. | System for enhancing navigation and surveillance in low visibility conditions |
US6119055A (en) * | 1998-01-23 | 2000-09-12 | Mcdonnell Douglas Corporation | Real time imaging system and method for use in aiding a landing operation of an aircraft in obscured weather conditions |
US20050232512A1 (en) * | 2004-04-20 | 2005-10-20 | Max-Viz, Inc. | Neural net based processor for synthetic vision fusion |
US20060126959A1 (en) * | 2004-12-13 | 2006-06-15 | Digitalglobe, Inc. | Method and apparatus for enhancing a digital image |
US7355179B1 (en) * | 2005-07-30 | 2008-04-08 | Rockwell Collins, Inc. | Scene imaging system integrity monitor and method thereof |
US7605369B2 (en) * | 2007-01-08 | 2009-10-20 | Max-Viz, Inc. | Assessing runway visibility to airborne infrared vision devices |
US10168179B2 (en) * | 2007-01-26 | 2019-01-01 | Honeywell International Inc. | Vehicle display system and method with enhanced vision system and synthetic vision system image display |
EP2167920B1 (en) * | 2007-07-18 | 2013-09-18 | Elbit Systems Ltd. | Aircraft landing assistance |
WO2009128065A1 (en) * | 2008-04-16 | 2009-10-22 | Elbit Systems Ltd. | Multispectral enhanced vision system and method for aircraft landing in inclement weather conditions |
US7850306B2 (en) * | 2008-08-28 | 2010-12-14 | Nokia Corporation | Visual cognition aware display and visual data transmission architecture |
US8749635B2 (en) * | 2009-06-03 | 2014-06-10 | Flir Systems, Inc. | Infrared camera systems and methods for dual sensor applications |
US9105115B2 (en) * | 2010-03-16 | 2015-08-11 | Honeywell International Inc. | Display systems and methods for displaying enhanced vision and synthetic images |
US9494447B2 (en) * | 2010-07-28 | 2016-11-15 | Honeywell International Inc. | Methods and systems for attitude differentiation in enhanced vision images |
IL221863A (en) * | 2012-09-10 | 2014-01-30 | Elbit Systems Ltd | Digital system for surgical video capturing and display |
US9000350B1 (en) * | 2012-09-11 | 2015-04-07 | Rockwell Collins, Inc. | Time-domain overlap imagery detecting system and method of using same |
US9390559B2 (en) * | 2013-03-12 | 2016-07-12 | Honeywell International Inc. | Aircraft flight deck displays and systems and methods for enhanced display of obstacles in a combined vision display |
CN104766337B (en) * | 2015-04-27 | 2017-10-20 | 西北工业大学 | One kind is based on the enhanced aircraft landing vision enhancement method in runway boundary |
-
2015
- 2015-10-22 FR FR1502237A patent/FR3042882B1/en not_active Expired - Fee Related
-
2016
- 2016-10-21 WO PCT/EP2016/075412 patent/WO2017068141A1/en active Application Filing
- 2016-10-21 US US15/768,350 patent/US20180300856A1/en not_active Abandoned
- 2016-10-21 CN CN201680061634.5A patent/CN108369727A/en active Pending
- 2016-10-21 EP EP16787785.1A patent/EP3365866A1/en not_active Withdrawn
- 2016-10-21 CA CA3001668A patent/CA3001668A1/en not_active Abandoned
-
2018
- 2018-04-17 IL IL258758A patent/IL258758A/en unknown
Also Published As
Publication number | Publication date |
---|---|
FR3042882A1 (en) | 2017-04-28 |
FR3042882B1 (en) | 2018-09-21 |
CA3001668A1 (en) | 2017-04-27 |
WO2017068141A1 (en) | 2017-04-27 |
US20180300856A1 (en) | 2018-10-18 |
CN108369727A (en) | 2018-08-03 |
IL258758A (en) | 2018-06-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3072812B1 (en) | A method and a device for marking the ground for an aircraft in flight, and an aircraft including the device | |
EP3365866A1 (en) | System adapted for providing an operator with augmented visibility and associated method | |
EP2272027B1 (en) | Multispectral enhanced vision system and method for aircraft landing in inclement weather conditions | |
CA2418243C (en) | Aircraft guidance procedure for the final landing phase and the corresponding device | |
CA2989154A1 (en) | System and method for automatically inspecting surfaces | |
WO2016113983A1 (en) | Image processing device, image processing method, program, and system | |
FR3062720A1 (en) | SYSTEM AND METHOD FOR AIDING THE LANDING OF AN AIRCRAFT, AND THE AIRCRAFT CORRESPONDING | |
JP2005537545A (en) | Image fusion system and method | |
CN102314601A (en) | Use nonlinear optical to remove by the shade in the image of catching based on the camera of vehicle according to constant nuclear | |
EP3689755B1 (en) | Directional lighting system provided in an aircraft and associated lighting method | |
FR3089038A1 (en) | METHOD FOR LEARNING A NETWORK OF NEURONES ON BOARD AN AIRCRAFT FOR AID IN LANDING SAID AIRCRAFT AND SERVER FOR THE IMPLEMENTATION OF SUCH A PROCESS | |
JP6555569B2 (en) | Image processing apparatus, mobile device control system, and image processing program | |
CN114729804A (en) | Multispectral imaging system and method for navigation | |
EP1936330B1 (en) | Method and system for processing and visualising images of the surroundings of an aircraft | |
EP2193477B1 (en) | Method and system for aircraft taxiing assistance | |
FR3088908A1 (en) | DEVICE AND METHOD FOR ASSISTING THE LANDING OF AN AIRCRAFT UNDER CONDITIONS OF REDUCED VISIBILITY | |
US20190141262A1 (en) | Systems and methods for detecting light sources | |
CA2503891C (en) | Image processing device with recognition and selection of light sources | |
EP2508870A1 (en) | Device for inspecting travelling belts | |
CN117037007B (en) | Aerial photographing type road illumination uniformity checking method and device | |
EP4086819A1 (en) | Method for learning a supervised artificial intelligence intended for identifying a predetermined object in the surroundings of an aircraft | |
EP3602395B1 (en) | Method based on a pathway article with calibration information | |
FR3058690B1 (en) | ASSISTED LIGHTING SYSTEM FOR VEHICLE AND METHOD FOR FORMING HYBRID IMAGE | |
EP4386719A1 (en) | Method and system for assisting a pilot of an aircraft during taxiing the aircraft on a taxiway of an airfield | |
CA3233479A1 (en) | Method for detecting obstacles |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20180420 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G08G 5/02 20060101ALI20211015BHEP Ipc: G08G 5/06 20060101ALI20211015BHEP Ipc: G08G 5/00 20060101ALI20211015BHEP Ipc: G06T 3/00 20060101ALI20211015BHEP Ipc: G06T 19/00 20110101ALI20211015BHEP Ipc: G06T 5/50 20060101ALI20211015BHEP Ipc: G02B 27/01 20060101AFI20211015BHEP |
|
INTG | Intention to grant announced |
Effective date: 20211102 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20220315 |