CN107076614A - Drafting and cognitive method and system based on polarization - Google Patents
Drafting and cognitive method and system based on polarization Download PDFInfo
- Publication number
- CN107076614A CN107076614A CN201580055967.2A CN201580055967A CN107076614A CN 107076614 A CN107076614 A CN 107076614A CN 201580055967 A CN201580055967 A CN 201580055967A CN 107076614 A CN107076614 A CN 107076614A
- Authority
- CN
- China
- Prior art keywords
- image
- road
- polarization
- data
- addition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000010287 polarization Effects 0.000 title claims abstract description 87
- 238000000034 method Methods 0.000 title claims abstract description 63
- 230000001149 cognitive effect Effects 0.000 title description 2
- 238000012937 correction Methods 0.000 claims abstract description 8
- 238000004566 IR spectroscopy Methods 0.000 claims abstract description 6
- 230000009471 action Effects 0.000 claims abstract description 5
- 238000001514 detection method Methods 0.000 claims description 12
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 11
- 238000000711 polarimetry Methods 0.000 claims description 9
- 230000004888 barrier function Effects 0.000 abstract description 21
- 230000011664 signaling Effects 0.000 abstract description 10
- 230000003287 optical effect Effects 0.000 abstract description 7
- 230000006872 improvement Effects 0.000 abstract description 4
- 230000008447 perception Effects 0.000 abstract description 4
- 238000003331 infrared imaging Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 32
- 230000005855 radiation Effects 0.000 description 14
- 230000000007 visual effect Effects 0.000 description 14
- 230000000740 bleeding effect Effects 0.000 description 13
- 238000003384 imaging method Methods 0.000 description 7
- 239000013598 vector Substances 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000006399 behavior Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000004438 eyesight Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 241001465754 Metazoa Species 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000002310 reflectometry Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 206010003830 Automatism Diseases 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000739 chaotic effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005538 encapsulation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 239000004575 stone Substances 0.000 description 1
- 230000009182 swimming Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
- G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/10—Image acquisition
- G06V10/12—Details of acquisition arrangements; Constructional details thereof
- G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V10/147—Details of sensors, e.g. sensor lenses
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Multimedia (AREA)
- Automation & Control Theory (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Vascular Medicine (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
Abstract
A kind of use LONG WAVE INFRARED imaging polarization determines the drafting and perception and perception that come road improvement or path or the method for detecting barrier, records raw image data to obtain the polarization image of road or region including the use of polarimeter.Then for inhomogeneities, optical distortion and registration correction map picture.IR data results and polarization data result are calculated, and the data result of generation is converted into cube for utilization.Contrast enhancement algorithms are applied to multidimensional image to form enhanced object images.Then enhanced object images can be shown to user and/or signalling means can notice the presence of object.In addition, the vehicles can the presence based on object in road and use hide action.
Description
The reference of related application
This application claims the Serial No. 62/041,778 submitted for 26th in August in 2014, entitled
" Polarization-based Mapping and Perception System and Method " US provisional patent Shen
Rights and interests and priority please, the U.S. Provisional Patent Application are entirely incorporated into herein by quoting.The application is still 2015
The Serial No. 14/602,823 submitted on January 22, entitled " Polarization Imaging for Facial
The part continuation application of Recognition Enhancement System and Method " U.S.'s non-provisional application, the U.S.
The full content of state's non-provisional application is entirely incorporated into herein by quoting.
Governmental approval right
The present invention obtains governmental support under the contract number N00014-13-C-0290 authorized by USN and entered
OK.Government has specific rights in the present invention.
Background technology and the content of the invention
As used in this article, LONG WAVE INFRARED is referred to as " LWIR " or " thermal infrared ".As used in this article, " paint
System " refers to placing the object in scene relative to other objects or element in the scene.As an example, " that is small
Stone is close to that boulder just beside the road in the road ".As used in this article, " road " is referred to
People, animal or the vehicles can along its across any path.
Disclosed herein the drafting for determining road improvement or path using LONG WAVE INFRARED imaging polarization and perceive with
And the method for perceiving or detecting object.Described method is not tied to any specific polarimeter sensor architecture,
Therefore described method is suitable for that all LWIR sensors of the polarimetry mark of key can be detected.This method includes making
With the raw image data of polarimeter posting field to obtain the polarization image in the region.Then according to the framework institute by sensor
The process needed, for inhomogeneities, optical distortion and registration correction map picture.IR data results and polarization data result are calculated, with
And the data result of generation is converted into cube for utilization.Contrast enhancement algorithms are applied to multidimensional image with shape
Into enhanced object images.Then enhanced object images can be shown to user and/or signalling means can notice object
In the presence of.In addition, the vehicles can the presence based on object in road and use hide action.
The IR cameras of standard provide the information (that is, how bright target seem) indicated on IR, spatial information (that is, mesh
The where of mark in the scene) and temporal information (i.e., between frames, the change of target in the scene).As disclosed herein
All this information of polarimetry system and method offer and also offer polarimetry mark.
Brief description of the drawings
This patent or application documents include at least one accompanying drawing performed by color.Filing a request and paying required expense
Afterwards, Patent Office will provide the copy with coloured picture of this patent or patent application publication.
Fig. 1 is the block diagram for showing system according to an illustrative embodiment of the invention.
Fig. 2 shows exemplary polarimeter and signal processing unit as shown in Figure 1.
Fig. 3 is the flow of the exemplary architecture and function that show system logic according to an illustrative embodiment of the invention
Figure.
Fig. 4 a show visual picture of the road at night.
Fig. 4 b are thermal imagery of the same road at night.
Fig. 4 c show the polarization of the road obtained using system and method according to an illustrative embodiment of the invention
Image.
Fig. 5 a are visual picture of the road on daytime.
Fig. 5 b are the thermal imagery of Fig. 5 a road.
The road for Fig. 5 a that system and method for Fig. 5 c for utilization according to an illustrative embodiment of the invention are obtained
Polarization image.
Fig. 6 a are the visual picture in the road with median strip and pavement shown by night.
Fig. 6 b are Fig. 6 a road also in the thermal imagery at night.
The road for Fig. 6 a that system and method for Fig. 6 c for utilization according to an illustrative embodiment of the invention are obtained
Polarization image.
Fig. 7 a are the visual picture of muddy and ruthed lane, and it illustrates the exemplary barrier on the road.
Fig. 7 b are the thermal imagery of Fig. 7 a road.
Fig. 7 c are the enhanced thermal imagery of contrast of Fig. 7 a road.
Fig. 7 d are the polarization image of Fig. 7 a road.
Fig. 7 e are the color bleeding image of Fig. 7 d road.
Fig. 8 a are the visual picture of muddy and ruthed lane, and it illustrates exemplary barrier of the night on the road.
Fig. 8 b are the thermal imagery of Fig. 8 a road.
Fig. 8 c are the enhanced thermal imagery of contrast of Fig. 8 a road.
Fig. 8 d are the polarization image of Fig. 8 a road.
Fig. 8 e are the color bleeding image of Fig. 8 d road.
Fig. 9 a show the visual picture of track for a train.
Fig. 9 b are the thermal imagery of Fig. 9 a track for a train.
Fig. 9 c are the polarization image of Fig. 9 c track for a train.
Figure 10 a are Stokes (Stokes) the vector image S for showing the maritime affairs scene of fisherman on ship0。
Figure 10 b are the color bleeding image of Figure 10 a scene.
Figure 10 c are the Stokes vector image S of Figure 10 a maritime affairs1。
Figure 10 d are the Stokes vector image S of Figure 10 a maritime affairs2。
Figure 10 e are the DoLP images of Figure 10 a maritime affairs scene.
Embodiment
Fig. 1 shows system 100 according to an illustrative embodiment of the invention.The system 100 includes being arranged on traffic work
Polarimeter 101 and signal processing unit 107 on tool 103, the polarimeter 101 and signal processing unit 107 are collected and analyzed
The image on surface 111, detection and notice for object 102.Example object 102 shown in Fig. 1 include barrier 104,
Puddle or mire 105 and road edge 106.As used in this article, term " object " can refer to any object, road
Defect or region interested, in some embodiments including people or other animals.In Fig. 1, barrier 104 and mire
105 be the object that the vehicles are wanted to avoid.Road edge 106 is that the vehicles are wanted to know its position to rest on road
On object.Therefore in some embodiments, object 102 is to wait to avoid or to be positioned so as to the safety navigation vehicles 103
Object.In other embodiments, object 102 is to need the project of position, such as people during search and rescue operation,
As further discussed herein.
Polarimeter 101 includes being used to record the polarization imaging equipment of polarization image, such as collect image digital camera or
Thermal imagers.The vehicles 103 can be automobile, ship, aircraft or any vehicles navigated, or walking people.Partially
The meter 101 that shakes collects the raw image data of road environment, and the road environment includes surface 111 (such as road) and object 102, all
Such as barrier 104, puddle or mire 105 and road edge 106.
Raw image data is sent to signal processing unit 107 by polarimeter 101, the signal processing unit 107 processing number
According to as further discussed herein.Then on a display device 108 by the data display of processing to operator, or in signal
Detection is noticed on device 110, as further discussed herein.Although Fig. 1 by polarimeter 101, signal processing unit 107, display
Device 109 and signalling means 110 are shown as independent entry, but polarimeter 101 and signal processing unit 107 in some embodiments by
It is assembled into an equipment and is placed on the vehicles 103, so that polarimeter has the visual field of road, and display 109
It is assembled together with signalling means 110 and is placed on vehicle interior.
In the embodiment shown, polarimeter 101 (is not shown raw image data by network or communication channel 108
Go out) signal processing unit 107 is sent to, and the data handled are sent to display 109 and signalling means 110.Signal transacting
Unit 107 can be any suitable computer of exploitation be known in the art or following.Signal processing unit 107 is received
Raw image data, filter the data and analyze the data, as further discussed herein, with provide enhanced imaging with
And detection and notice.Network 108 can be any kind of one or more nets of exploitation be known in the art or following
Network, simple communication cable, the Internet backbone net, Ethernet, Wifi, WiMax, broadband over power line, coaxial cable etc..Net
Network 108 can be hardware, software or any combinations of the two.In addition, network 108 may reside in encapsulation polarimeter 101 and letter
In the sensor (not shown) of number processing unit 107.
In another exemplary embodiment (not shown), in agricultural environment, the vehicles 103 include someone or nobody
(autonomous) agricultural equipment, and object 102 are included along farm road or the barrier in field.In another embodiment
In, the vehicles 103 are included in someone for being run on water route or ocean or nobody (autonomous) ship, and object 102 is swum in
In water.In another exemplary embodiment (not shown), the vehicles 103 include scanning for the people with rescue operation or ship
Oceangoing ship, and the victim that object 102 is the accident for being related to water body.In another exemplary embodiment (not shown), traffic work
Tool 103 includes someone or nobody (autonomous) aircraft, and object 102 is those objects found in airport environment, the airport
Environment includes the greenery patches in runway and runway and around runway.In another exemplary embodiment (not shown), traffic work
Those objects that tool 103 includes railway equipment, and object 102 to find in the environment around railroad track and railway switch.
Fig. 2 shows exemplary polarimeter 101 and signal processing unit 107 according to the embodiment of the present invention.This is inclined
The meter 101 that shakes includes image-forming objective lens 1201, filter array 1203 and focal plane arrays (FPA) 1202.Image-forming objective lens 1201 include pointing to
The lens at surface 111 (Fig. 1) place.Filter array 1203 filters the image received from image-forming objective lens system 1201.Focal plane array
Row 1202 include the array of light sensing pixel.
Signal processing unit 107 includes image processing logic 120 and system data 121.In exemplary signal processing unit
In 107, image processing logic 120 and system data 121 are shown to be stored in memory 1123.Image processing logic 120
It can be realized with system data 121 with the combination of hardware, software or hardware and software.
Signal processing unit 107 also includes processor 130, and the processor 130 includes digital processing unit or is configured to pass through
Image processing logic 120 is handled to run the other types of circuit (if applicable) of the image processing logic 120.Processing
Device 130 is by other element communications in local interface 1124 and signal processing unit 107 and drives above-mentioned other elements, should
Local interface 1124 can include one or more buses.When being stored in memory 1123, image processing logic 120 and it is
System data 121 can be stored and conveyed on any computer-readable medium, and the computer-readable medium is made by following item
It is connected with or with following item：Logic circuit, processor, instruction execution system, instruction executing device or instruction perform equipment (such as
Computer based system, the system comprising processor), or can be held from the instruction execution system, instruction executing device or instruction
Row equipment obtains instruction and performs other systems of these instructions.In the context of this article, " computer-readable medium " can be with
For can include, store, transmit, propagate or conveying program be commanded execution system, instruction executing device or instruction perform equipment
Any part that equipment is connected is performed using or with instruction execution system, instruction executing device or instruction.Computer-readable medium
Electronic media, magnetizing mediums, optical medium, electromagnetic medium, infrared medium or semiconductor system can be such as, but not limited to, partly led
Body device, semiconductor equipment or propagation medium.Note, computer-readable medium even can print for paper or thereon journey
Other suitable medias of sequence, because this can electronically be obtained to the optical scanner of paper or other media by for example
If desired program, then then compile, explain or handle the program in other side in an appropriate manner, and then by the program
It is stored in computer storage.
The example system data 121 of diagram include：
A. the raw image data from polarimeter 101 (Fig. 2) obtained from the step 1001 (Fig. 3) of method 100 (is not drawn
Go out).
B. the view data (not shown) corrected, it corrects inequality for the step 1002 (Fig. 3) according to method 1000
Even property, optical distortion and the data of registration.
C. the heat (IR) obtained from the step 1003 (Fig. 3) of method 1000 as and polarization image.
That is d. applied in the step 1004 (Fig. 3) of method 1000 is converted to multidimensional image number by polarization data and dsc data
According to.
E. contrast enhancement algorithms applied to view data in the step 1005 (Fig. 3) of method 1000.
F. the object detection applied to the enhanced view data of contrast in the step 1006 (Fig. 3) of method 1000 is calculated
Method.
G. the view data for being applied to display 109 and signalling means 110 in the step 1007 (Fig. 3) of method 1000.
H. such as thermal-image data described herein.
I. such as heat/polarization image of mixing described herein.
Image processing logic 120 is performed herein in connection with the process described by Fig. 3.
Reference picture 2, external interface device 126 be connected to display 109 and signalling means 110 and with display 109 and letter
Number device 110 communicates.External interface device 126 can also communicate with input equipment or including input equipment, the input equipment is for example
For keyboard, switch, mouse, touch-screen, and/or other types of interface available for the user input data by system 100.Outside
Portion's interface equipment 126 can with or communicate or including following item with following item as an alternative：Personal digital assistant (PDA), computer
Tablet device, notebook computer, portable or non-portable computer, honeycomb or mobile phone etc..External interface device 126 is also
Can with or communicate or including following item with following item as an alternative：It is non-personal computer (such as server), embedded computer, existing
Field programmable gate array (FPGA), microprocessor etc..
In Fig. 2 illustrative embodiments, external interface device 126 is shown as one of signal processing unit 107
Point.In other embodiments, external interface device 126 can be in the outside of signal processing unit 107.
Display device 109 can include：TV, LCD screen, monitor, or pass on from method 1000 view data or
It is attached to personal digital assistant (PDA), computer tablet device, notebook computer, portable or non-portable computer, honeycomb
Or any electronic equipment of mobile phone etc..Signalling means equipment 110 can include warning buzzer, bell, flash lamp or warning behaviour
Author detects object or any other sense of hearing of barrier or vision or haptics member.
In some embodiments, independent behaviour can be taken based on the object 102 (Fig. 1) detected.For example, traffic work
103 (Fig. 1) of tool can automatically be directed to avoid object 102.On this point, external interface device 126 can be with traffic
Instrument 103 is connected, so that processor 130 can instruct the vehicles in the neighbouring steering of object 102.Taking automatism
Under certain situation, it may not be necessary to signalling means 110.
In other embodiments, global positioning system (" GPS ") equipment (not shown) can be with external interface device 126
Connect the position of the object 102 detected to provide.
In the illustrated embodiment, display 109 and signalling means 110 are shown as separation, but can be by signal
Device 110 is combined with display 109, and in other embodiments, the part that noticing can show as view data is adopted
With the form in prominent frame or region or other means for protruding object.For example, indicating that frame (such as (do not show by red block
Go out)) visually indicating for the object 102 that detects can be provided.
Fig. 3 is the exemplary architecture and the flow of function for showing the image processing logic 120 (Fig. 2) according to method 1000
Figure.In the step 1001 of method 1000, polarimeter 101 obtains the image of road scene from the vehicles of road 111 (Fig. 1)
And raw image data is sent to signal processing unit 107 (Fig. 1).
In step 1002, signal processing unit 107 (Fig. 1) corrects the imager of the image received from polarimeter 101 not
Uniformity.The example of imager inhomogeneities includes pattern lines, noise pixel, bad pixel, bright spot fixed in image etc..Can be with
Imager inhomogeneities is corrected using the algorithm being known in the art.In some embodiments, step 1002 is not performed,
Because imager inhomogeneities is without amendment.
In addition in step 1002, signal processing unit 107 removes image fault from view data.Image fault is shown
Example is the distortion caused by image-forming objective lens system in image border.The algorithm being known in the art can be used to correct
Image fault.Using the method being known in the art, registration amendment is can also carry out in step 1002.
In step 1003, IR data results and polarization data result are calculated.In this step, using in step 1002
The weighting subtraction of the polarization image of acquisition calculates Stokes' parameter (S0、S1、S2).LWIR imaging polarizations meter measures radiation diagram
As and polarization image both.Radiation image is standard picture, and thus each pixel in the image is the corresponding picture from scene
The measured value of radiation that plain area reflects or launched, is generally expressed with watt/cm2-sr.Standard photographs and thermal imagery are radiation diagram
Picture, is only the drafting for the radiation profiles launched or reflected from scene.Polarization image is that polarization state is distributed on image
Draw.Polarization state distribution is expressed generally according to Stokes image.
In Stokes' parameter, S0Represent traditional LWIR thermal imagerys without polarization information.S1And S2Display is orthogonal inclined
Measure of shaking information.Therefore, polarized by the G.G.Stokes Stokes vectors being firstly introduced in 1852 available for description section
Light and be defined as
Wherein, I0For in the radiation of the direction linear polarization with the horizontal 0 degree of angle, I90For with the horizontal 90 degree
The radiation of the direction linear polarization at angle.Similarly, I45And I135For with respect to the horizontal plane angle at 45 ° and 135 ° of angles it is linear partially
Shake the radiation value of light.Finally, IRAnd ILFor the radiation value for right-hand circularly polarized light and left light.For the present invention, right circle is inclined
Shake light and left light is not required, and imaging polarization meter need not measure these polarization states.For this reason, Wo Menkao
The Stokes vector of worry will be limited by only expressing first three element of linearly polarized photon.
Another useful form of formula (2) is by the normalized form for the formula being given below
The polarization launched or reflected to the object in side and road or surface from the surface of road surface, road
State depends on a large amount of factors, including the angle of departure, the surface temperature on surface, the microroughness on surface (texture), the multiple folding on surface
Penetrate the ambient temperature of rate and surrounding environment.The present invention is main using following true herein：Launch and reflect from surface and object
Light polarization state for the angle of departure and different surfaces texture function.
The emissivity of object is determined from the radiation law of kirchhoff (Kirchoff).Kirchhoff's law most familiar with shape
Formula provides the emissivity ε on surface according to reflectivity r, and it is given by：
ε (θ, φ)=1-r (θ) (4)
Wherein, θ is the angle between surface normal and the sight of camera.More typically change for Kirchhoff's law is public
Formula is given by
εp(θ)=1-rp(θ) (5)
And
εs(θ)=1-rs(θ) (6)
Wherein, subscript p and s represent the emissivity and reflectivity of specific polarization state.P- state instructions are used to include table
The plane of departure of the light of linear polarization in the plane of the sight of face normal and camera.If for example, camera looks down level
Surface, then p- polarization states will the vertical polarization of presentation.S- polarization orthogonals are in p- states.Note, we are in formula 4 to formula
Temperature and wavelength dependency are inhibited in 6.
Formula (5) and formula (6) are substituted into and provided in formula (3)
Wherein, φ is the plane of incidence and horizontal plane angulation, and
Formula 8 can be more clearly written to
Wherein, rpAnd rsProvided by Fresnel (Fresnel) formula for reflection
Note, P (θ) ambiguously depends on the plane of incidence and horizontal plane angulation φ.Angle φ is incident for determining
The azimuth of the orientation in face and final surface normal is vital.Angle φ can be determined from such as lower angle
Angle, θ can be determined using many modes.Being known in the art is used for from normalized Stokes image
The method that (formula 3) determines θ and φ.
And in step 1003, Linear polarization (DoLP) image is calculated from Stokes image.DoLP images can use
In provide for the object in road surface and road contrast and can be such as calculating of getting off：
Or
Note, DoLP is linear polarization.As those skilled in the art will be appreciated by, in some cases, it may be desirable to
Nonlinear polarization (such as circular polarization).Therefore in other embodiments, step 1004 can be used from S0、S1、S2Or S3
Any combination derived from polarization image and be not limited to DoLP.
DoLP images are a kind of for the usable image for the polarization contrast watched in image.For watching polarization content
Another alternative image is to generate " color bleeding " image by the way that radiation, DoLP and orientation image are plotted into color table.Ability
The technical staff in domain carries out the following drafting that polarization data is represented to colourity-saturation degree-value for color：
S0=value
DoLP=saturation degrees
It is orientated φ=colourity
This expression, which can show all optical information (radiation and polarize) and provide in single image, shows field
The means of the understanding enhanced with polarization contrast of the actinometry of scape.Strong in many cases, this table in polarization contrast
The surface or object for being shown as polarization provide scene environment.Those skilled in the art is envisioned that the other manner for doing this point.
Color bleeding is an embodiment of the producible multi-C representation in step 1004.Those skilled in the art
It is contemplated that similar draw.For an example, when radiation value is low, DoLP information can be strengthened.
As described above, the polarization state launched or reflected from the surface in the surface of object or image scene is depending on more
Individual factor, it includes the complex refractivity index on the angle of departure, the surface temperature on surface, the microroughness on surface or texture and surface.Then
In general, the geometry for the object that surface in scene and the contrast of object are depended in scene due to polarization and
Material or surface characteristic.Although the temperature difference for the object that surface temperature is contributed in polarization mark contrast, scene differs
Surely it is to there is polarization contrast.This is important, because many objects in image scene can be continually in identical
Or very similar temperature and therefore the contrast of very little is shown.
Because the optical radiation on basis depends on transmitting, therefore for polarization imaging, it is not necessary to additional light source, illumination,
Or ambient light.This is key point and this method is different from all methods of prior art.In addition, this method night also as
It equally well works on daytime.
In step 1005, the contrast enhancement algorithms being known in the art are applied to the multidimensional from step 1004
Image.Multidimensional data develops polarization data to significantly increase the information content in scene.Non-limiting example include it is global average,
Square analysis, principal component analysis or linear judgment analysis, the statistics meter as overall multidimensional data of variance and greater degree
Calculate and be then based on carrying out convolution and then by the sheet of the normalized core of the global statistics of scene as overall image
The calculating of ground value.
In step 1006, the object detection algorithm being known in the art is applied to the contrast from step 1005
Enhanced data.The non-limiting example of object detection algorithm includes manually or automatically setting threshold value based on image statisticses, being based on
Contrast strengthens some, rim detection and the morphological character of segmentation figure picture.
In step 1007, it may then pass through vision or audio portion and the object detected noticed to user.It is unrestricted
Property example include guiding to the notice of operator into the bell, buzzer or lamp of display, or instruction over the display, such as
Distinguished color or square frame in the region on barrier or surface.Additionally or alternatively, can in step 1007
So that the enhanced image of contrast is shown into user's (not shown).
In other embodiments, step 1003, step 1004, step 1005 and step 1006 can be applied in combination, this
Omit one or more steps.In other embodiments, polarization image data or multidimensional (color bleeding) data can be by people
Check for object detection, and do not apply algorithm.
The algorithm for developing the combination of the characteristics of image extracted from LWIR imaging polarizations meter can be used for detecting potential obstacle
Thing or road edge.In the case of track for a train, algorithm can be used automatically to confirm the continuity of track.Once detection
To potential noticeable feature, then automatically these features can be highlighted to operator and can be by certain notice machine
Structure (buzzer or lamp) provides warning.Potentially it can also develop orientation information using algorithm to help improve the reason to image
Solution, such as segmentation or shape recognition.
For the purpose of the operation vehicles, enhanced contrast realizes the drafting of the feature in image scene, passes through behaviour
Author perceives or automatic detection and warning, and the drafting improves the security of operator, or in equipment (such as agriculture of autonomous operation
Industry equipment) in the case of provide autonomous Obstacle avoidance to steering or navigation system.Specifically, the improvement inspection of barrier
Surveying and recognizing will allow operator to dispatch the vehicles (or ship) avoiding obstacles.The improvement detection and perception of road edge will
Reduction not inadvertently departs from the probability of road.This is particularly true at the night that the eyesight of operator is limited by dark.
As discussed in this article, system and method for the invention by polarimetry mark (that is, time tag, distinctive mark,
Indicate with IR) it is added to the information that can be previously obtained by IR cameras.Can be simultaneously using this four category information come to detecting
Object sort out/classify.In addition, classification/classification of the object detected can be influenceed hiding for being taken by the vehicles
Action.For example, the object detected in the road can be classified as the barrier for needing to avoid, and non-vehicle can lead to
Capable hollow.In addition, according to given standard, can based on its time tag, space indicate, IR marks and/or polarimetry mark
So that multiple objects independently and to be synchronously categorized as to independent group or subgroup.
Fig. 4 a show visual picture of the road 400 at night.Fig. 4 b are thermal imagery of the same road 400 at night.Note,
In Fig. 4 b, road 400 and surrounding terrain have almost identical temperature, therefore in thermal imagery, on the road of road 400 and the road
There is the contrast of very little between shoulder 401.Fig. 4 c show to utilize system and method according to an illustrative embodiment of the invention
The polarization image of the road 400 obtained.Polarization image in Fig. 4 c shows that the strong contrast of road 400 and curb 401 is very
Easily differentiate.It is pavement in the left-hand side white stripes 402 parallel with road 400.In the case of without external light source
Obtain the polarization image in Fig. 4 c.
Fig. 5 a are visual picture of the road 500 on daytime.Fig. 5 b are the thermal imagery of Fig. 5 a road 500.In Fig. 5 b thermal imagery
In, road 500 and other situation elements show chaotic contrast.Fig. 5 c are to utilize the exemplary embodiment party according to the present invention
The polarization image of the road 500 for Fig. 5 a that the system and method for formula are obtained.Fig. 5 c polarization image shows only road 500
Strong contrast.In Fig. 5 c polarization image, driveway 502 on the left-hand side pavement 501 parallel with road and on the right
It is easy to differentiate.In Fig. 5 b thermal imagery, pavement 501 and driveway 502 are not easily perceived.
Fig. 6 a are the visual picture in the road 600 with median strip 601 and pavement 602 shown by night.
Fig. 6 b are Fig. 6 a road 600 also in the thermal imagery at night.Road has similar temperature with surrounding terrain, therefore in thermal imagery,
There is very weak contrast between road and the median strip of the road.
Fig. 6 c are Fig. 6 a obtained using system and method according to an illustrative embodiment of the invention road 600
Polarization image.Fig. 6 c polarization image shows the strong contrast of road 600.In Fig. 6 c polarization image, in right-hand side
The pavement 602 and median strip 601 parallel with road is easy to differentiate.In Fig. 6 b thermal imagery, pavement 602 and center
Dividing strip 601 is not easily perceived.
Fig. 7 a are the visual picture of muddy and ruthed lane 700, and it illustrates the exemplary barrier 701 on the road 700.In figure
As in, barrier 701 includes plank.Fig. 7 b are the thermal imagery of Fig. 7 a road 700.Compared in Fig. 7 a visual picture,
In Fig. 7 b image, barrier 701 is easier to differentiate.Fig. 7 c are the enhanced thermal imagery of contrast of Fig. 7 a road 700.
Fig. 7 d are the polarization image of Fig. 7 a road 700.Although polarization image is not provided relative to road 700 to barrier
Many contexts, but the barrier 701 in the image is easy to be resolved.Fig. 7 e are the color bleeding figure of Fig. 7 d road
Picture.The color bleeding image shows both dsc data and polarimetry data in single image and provides maximum-contrast.
Fig. 8 a are the visual picture of muddy and ruthed lane 800, and it illustrates exemplary barrier 801 of the night on road 800.
In the image, barrier 801 includes wet soil and mud.These are the potential hazards that possible hinder some land crafts to move.
Fig. 8 b are the thermal imagery of Fig. 8 a road 800.Fig. 8 c are the enhanced thermal imagery of contrast of Fig. 8 a road 800.
Fig. 8 d are the polarization image of Fig. 8 a road 800.Although polarization image is not provided relative to road 800 to barrier
Many contexts, but the barrier 801 in the image is easy to be resolved.Fig. 8 e are the color bleeding figure of Fig. 8 d road
Picture.The color bleeding image shows both dsc data and polarimetry data in single image and provides maximum-contrast.
Fig. 8 e color bleeding image, which is shown, how to provide link table using the combination of dsc data result and polarization data result
The good expression in face.
Fig. 9 a (insertion photo) are the visual picture of track for a train 900, and it illustrates showing for the rail in Railway Environment
Example property segmentation.Fig. 9 b are thermal imagery, wherein, the identification of track is made due to the different temperatures of the object in the region of adjacent tracks
It is highly difficult.Fig. 9 c are the polarimetric images of Fig. 9 a track for a train 900 and show the good profile of track.
Similarly, for the ship for the navigation water body for needing to avoid underwater obstacles, LWIR polarimeters can be used to increase
Contrast between the barrier and water background on thwack one's water breaks surface.This can be particularly effective, because swimming in water
Therefore object is intended to have is likely difficult in water identical temperature therein and by actinometry (heat) with their just floatings
Image is detected.
Figure 10 a, Figure 10 c and Figure 10 d are respectively the Stokes vector image S for showing the maritime affairs scene of fisherman on ship0、S1
And S2.Figure 10 e are the DoLP images of Same Scene.Figure 10 b are the color bleeding image of the scene.The color bleeding image is shown
It is used for the contrast enhancing of the Obstacle avoidance of ship or advantage for searching for and succouring.
Claims (23)
1. a kind of from mobile traffic along the method for path detection object, methods described includes：
Using the raw image data of polarimeter posting field to obtain the polarization image in the region；
The polarization image is performed amendment to form correction map picture；
IR data results and polarization data result are calculated from the correction map picture；
Data result is converted to cube to form multidimensional image；
Contrast enhancement algorithms are applied to multidimensional image to form enhanced object images；
Object images described in automatic identification.
2. the method as described in claim 1, in addition to generate the drafting of the object detected.
3. the method as described in claim 1, in addition to the enhanced object images are shown to user.
4. the method as described in claim 1, in addition to the object detected is advertised to user.
5. the method as described in claim 1, in addition to take automatically and hide action to avoid the object detected.
6. the method for claim 1, wherein to the polarization image perform amendment with formed correction map as the step of wrap
Include for polarization image described in inhomogeneities amendment.
7. method as claimed in claim 6, wherein, the step of correcting is performed to the polarization image also to be included performing optics mistake
True amendment and registration amendment.
8. the method for claim 1, wherein from the correction map as calculating IR data results and polarization data result
Step includes：By the weighting subtraction of the polarization image, Stokes' parameter S is calculated from the polarization image0、S1And S2With
Create Stokes image.
9. method as claimed in claim 8, in addition to calculate the polarization image derived from the Stokes image.
10. method as claimed in claim 9, wherein, calculate derived from the Stokes image the step of polarization image
Including calculating DoLP images from the Stokes image.
11. the method for claim 1, wherein the path includes road, and methods described includes identification and is used for
The object of the safe on-road navigation vehicles.
12. the method for claim 1, wherein the path includes railway, and methods described includes identification and is used for
The object of the safety navigation vehicles on the railway.
13. the method for claim 1, wherein the path includes water route, and methods described includes recognizing the water
Object on road.
14. the method for claim 1, wherein the path includes course line, and methods described is included from aircraft identification
Object.
15. a kind of method for drawing road, including：
Record the raw image data of the road to obtain the polarization of the road by the mobile traffic with polarimeter
Image；
The polarization image is performed amendment to form correction map picture；
IR data results and polarization data result are calculated from the correction map picture；
Data result is converted to cube to form multidimensional image；
Contrast enhancement algorithms are applied to multidimensional image to form enhanced object images；
By the position record of the object detected in the road in memory.
16. method as claimed in claim 15, wherein, the step of recording the raw image data of the road is independent of certainly
The brightness of right light.
17. method as claimed in claim 15, wherein, light is not needed the step of the raw image data for recording the road
Source.
18. method as claimed in claim 15, in addition to take automatically and hide action to avoid the object in the road.
19. method as claimed in claim 15, includes the edge of the identification road.
20. water or mud in method as claimed in claim 15, in addition to the identification road.
21. method as claimed in claim 15, in addition to the time tag based on the object detected, space indicate, IR mark
Will and/or polarimetry mark are classified to the object detected.
22. method as claimed in claim 21, wherein, time tag, space indicate based on multiple objects, IR marks and/
Or polarimetry mark, the multiple object independently can be categorized as to independent group or subgroup.
23. method as claimed in claim 22, determines and initiates also including the use of the classification of object suitably to hide
Act to avoid the object detected.
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201462041778P | 2014-08-26 | 2014-08-26 | |
| US62/041,778 | 2014-08-26 | ||
| US14/602,823 | 2015-01-22 | ||
| US14/602,823 US10311285B2 (en) | 2014-01-22 | 2015-01-22 | Polarization imaging for facial recognition enhancement system and method |
| PCT/US2015/047008 WO2016076936A2 (en) | 2014-08-26 | 2015-08-26 | Polarization-based mapping and perception method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107076614A true CN107076614A (en) | 2017-08-18 |
Family
ID=58397916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201580055967.2A Pending CN107076614A (en) | 2014-08-26 | 2015-08-26 | Drafting and cognitive method and system based on polarization |
Country Status (3)
| Country | Link |
|---|---|
| EP (1) | EP3186606A4 (en) |
| CN (1) | CN107076614A (en) |
| AU (1) | AU2015347259A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109490867A (en) * | 2018-12-25 | 2019-03-19 | 北京理工大学 | Waterborne target polarization remote sensing detectivity evaluation method |
| CN111059454A (en) * | 2018-10-17 | 2020-04-24 | 西克股份公司 | Method for protecting a human being in the environment of a mobile machine |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111310607B (en) * | 2020-01-22 | 2021-01-01 | 交通运输部公路科学研究所 | Highway safety risk identification method and system based on computer vision and artificial intelligence |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030103214A1 (en) * | 2001-11-02 | 2003-06-05 | Vandelden Jay S. | Interferometric polarization interrogating filter assembly and method |
| US20080035834A1 (en) * | 2000-04-26 | 2008-02-14 | Gleckler Anthony D | Very fast time resolved imaging in multiparameter measurement space |
| US20080129541A1 (en) * | 2006-12-01 | 2008-06-05 | Magna Electronics | Black ice detection and warning system |
| US20090147238A1 (en) * | 2007-03-27 | 2009-06-11 | Markov Vladimir B | Integrated multi-sensor survailance and tracking system |
| US20100026555A1 (en) * | 2006-06-09 | 2010-02-04 | Whittaker William L | Obstacle detection arrangements in and for autonomous vehicles |
| US20100271475A1 (en) * | 2007-04-12 | 2010-10-28 | Schwiegerling James T | Compact snapshot polarimetry camera |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7426437B2 (en) * | 1997-10-22 | 2008-09-16 | Intelligent Technologies International, Inc. | Accident avoidance systems and methods |
| JP4645714B2 (en) * | 2008-09-19 | 2011-03-09 | 株式会社デンソー | Image processing device |
-
2015
- 2015-08-26 EP EP15859072.9A patent/EP3186606A4/en not_active Withdrawn
- 2015-08-26 CN CN201580055967.2A patent/CN107076614A/en active Pending
- 2015-08-26 AU AU2015347259A patent/AU2015347259A1/en not_active Abandoned
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080035834A1 (en) * | 2000-04-26 | 2008-02-14 | Gleckler Anthony D | Very fast time resolved imaging in multiparameter measurement space |
| US20030103214A1 (en) * | 2001-11-02 | 2003-06-05 | Vandelden Jay S. | Interferometric polarization interrogating filter assembly and method |
| US20100026555A1 (en) * | 2006-06-09 | 2010-02-04 | Whittaker William L | Obstacle detection arrangements in and for autonomous vehicles |
| US20080129541A1 (en) * | 2006-12-01 | 2008-06-05 | Magna Electronics | Black ice detection and warning system |
| US20090147238A1 (en) * | 2007-03-27 | 2009-06-11 | Markov Vladimir B | Integrated multi-sensor survailance and tracking system |
| US20100271475A1 (en) * | 2007-04-12 | 2010-10-28 | Schwiegerling James T | Compact snapshot polarimetry camera |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111059454A (en) * | 2018-10-17 | 2020-04-24 | 西克股份公司 | Method for protecting a human being in the environment of a mobile machine |
| CN111059454B (en) * | 2018-10-17 | 2023-03-21 | 西克股份公司 | Method for protecting a human being in the environment of a mobile machine |
| CN109490867A (en) * | 2018-12-25 | 2019-03-19 | 北京理工大学 | Waterborne target polarization remote sensing detectivity evaluation method |
| CN109490867B (en) * | 2018-12-25 | 2020-05-22 | 北京理工大学 | Water surface target polarization remote sensing detection capability evaluation method |
Also Published As
| Publication number | Publication date |
|---|---|
| EP3186606A4 (en) | 2018-05-09 |
| AU2015347259A1 (en) | 2017-03-30 |
| EP3186606A2 (en) | 2017-07-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11386648B2 (en) | Polarization-based mapping and perception method and system | |
| US11022541B2 (en) | Polarimetric detection of foreign fluids on surfaces | |
| CA3157194C (en) | Systems and methods for augmentation of sensor systems and imaging systems with polarization | |
| US9589195B2 (en) | Polarization-based mapping and perception method and system | |
| US7961906B2 (en) | Human detection with imaging sensors | |
| CN110998596B (en) | Night sensing | |
| US9832394B2 (en) | Adaptive low-light view modes | |
| Brooks et al. | Evaluating the use of unmanned aerial vehicles for transportation purposes:[parts AD]. | |
| US10708557B1 (en) | Multispectrum, multi-polarization (MSMP) filtering for improved perception of difficult to perceive colors | |
| CN107004272A (en) | Two-way community information manager | |
| US9064406B1 (en) | Portable and persistent vehicle surveillance system | |
| US10803590B2 (en) | Multiple-wavelength images analysis electro optical system for detection of accident ship and submerged person and analysis method thereof | |
| CN107076614A (en) | Drafting and cognitive method and system based on polarization | |
| Bar et al. | Target detection and verification via airborne hyperspectral and high-resolution imagery processing and fusion | |
| EP3631391A1 (en) | Polarization-based detection and mapping method and system | |
| KR101538488B1 (en) | Parking lot management system and method using omnidirectional camera | |
| RU164432U1 (en) | DEVICE FOR AUTOMATIC PHOTOVIDEO FIXATION OF VIOLATIONS DO NOT GIVE ADVANTAGES TO THE PEDESTRIAN AT THE UNRESOLVED PEDESTRIAN TRANSITION | |
| WO2016076936A2 (en) | Polarization-based mapping and perception method and system | |
| TWI499999B (en) | The 3D ring car image system based on probability calculation and its obtaining method | |
| Podobna et al. | Optical detection of marine mammals | |
| Gurton | Polarimetric longwave infrared (LWIR) thermal vision for autonomous vehicle operation | |
| TWM486116U (en) | 3D vehicle surrounding image system based on probability calculation | |
| US20210150885A1 (en) | Intelligent traffic signal control system and method | |
| Jang et al. | A multiple feature-based image-switching strategy in visual sensor networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170818 |
|
| WD01 | Invention patent application deemed withdrawn after publication |