JP2021533633A - Vehicle support system - Google Patents

Abstract

The present disclosure describes an exemplary vehicle assist system including an optical sensor, a pixelated filter array adjacent to the optical sensor, and a full-field optical selection element adjacent to the pixelated filter array. The optical selection element is configured to selectively guide the optical component of light incident on the optical selection element to the photosensor across the pixelated filter array.

Description

The present disclosure describes vehicle assistance systems, in particular optical vehicle assistance systems.

Autonomous driving technology uses an optical sensor system to detect objects on the road. Objects can include infrastructure, other vehicles, or pedestrians. Increasing the range of detection performance, improving the signal-to-noise ratio, and improving object recognition continue to be areas of development. Although virtually unrecognizable, a system capable of remotely providing attractiveness, distinctiveness, and data via an optical sensor system can be advantageous. For example, a sign can serve two purposes. That is, the sign can be visually read by conventional methods, and at the same time, the invisible code that supports the automatic driving of the on-board driving system can be sensed by the optical system.

Another industry challenge for optical sensors is detection under adverse conditions that can affect the light path and light quality, which can cause signal-to-noise ratio problems in detecting infrastructure, vehicles, or pedestrians. There is a need to improve.

The present disclosure describes an exemplary vehicle assist system including an optical sensor, a pixelated filter array adjacent to the optical sensor, and a full-field optical selection element adjacent to the pixelated filter array. The optical selection element is configured to selectively guide the optical component of light incident on the optical selection element to the photosensor across the pixelated filter array. In some examples, the vehicle includes a land vehicle, a marine vehicle, or an aerial vehicle.

The present disclosure describes an exemplary technique involving receiving an optical signal from an object by means of a vehicle assist system full-field optical selection element. This exemplary technique involves selectively guiding the optical components of an optical signal to an optical sensor through a pixelated filter array by means of a full-field optical selection element. The computing device may receive an image data signal from an image sensor in response to an optical signal, may compare the image data signal to multiple reference images in a lookup table, and output in response to the comparison. A signal may be generated.

Details of one or more embodiments of the invention are described in the accompanying drawings and the following description. Other features, objects, and advantages of the invention will become apparent from the specification and drawings, as well as the claims.

The above and other aspects of the invention will be further clarified by reading the following "forms for carrying out the invention" together with the accompanying drawings.

It is a conceptual diagram of an exemplary vehicle assist system including an optical sensor, a pixelated filter array, and a full-field optical selection element.

FIG. 3 is a conceptual diagram of an exemplary system including the vehicle support system of FIG. 1 for detecting light deflected by an object.

It is a conceptual diagram of an exemplary system including a vehicle assist system including a cascaded optical selection element for detecting light deflected by an object.

FIG. 3 is a conceptual diagram of an exemplary optical selection element including a cross-type dichroic splitter.

It is a conceptual diagram of an exemplary optical selection element including a trichromatic prism.

It is a conceptual diagram of an exemplary optical selection element including a trichromatic prism.

It is a conceptual diagram of a Bayer color filter array.

It is a conceptual diagram of a red / clear / clear / clear (RCCC) color filter array.

It is a conceptual diagram of a monochrome filter array.

It is a conceptual diagram of a red / clear / clear / blue (RCCB) color filter array.

It is a conceptual diagram of a red / green / clear / blue (RGCB) color filter array.

It is a conceptual diagram of a full-field optical selection element configured to reflect an infrared wavelength band and transmit visible light.

FIG. 3 is a conceptual diagram of a full-field optical selection element configured to reflect a first infrared wavelength band and transmit a second infrared wavelength band and visible light.

FIG. 3 is a conceptual diagram of a full-field optical selection element configured to reflect a first and second infrared wavelength band and transmit a third infrared wavelength band and visible light.

FIG. 3 is a conceptual diagram of a full-field optical selection element configured to reflect an infrared wavelength band and transmit visible light and an ultraviolet wavelength band.

FIG. 3 is a conceptual diagram of a full-field optical selection element configured to reflect the ultraviolet wavelength band and transmit the visible and infrared wavelength bands.

FIG. 3 is a conceptual diagram of a full-field optical selection element configured to reflect an infrared wavelength band and an ultraviolet wavelength band and transmit visible light.

FIG. 3 is a conceptual diagram of a full-field optical selection element configured to reflect a first red and green wavelength band and transmit a second and green wavelength band as well as a blue wavelength band.

FIG. 6 is a conceptual diagram of a full-field optical selection element configured to reflect an s-polarized red wavelength band, transmit a p-polarized red wavelength band, and transmit a green and blue wavelength band.

FIG. 6 is a conceptual diagram of an exemplary optical selection element including a lens.

FIG. 6 is a conceptual diagram of an exemplary field coupled-selective element, including a curved reflective interface.

FIG. 3 is a conceptual diagram of an exemplary optical selection element including a tilted reflector.

It is a conceptual diagram of a vehicle including an automated driver assistance system (ADAS).

FIG. 11A is a conceptual partial front view of the vehicle of FIG. 11A.

It is a flowchart of an exemplary method for sensing an optical signal by a vehicle support system.

It is a conceptual diagram of a coding pattern that can be read by a vehicle support system.

It is a chart which shows the spectrum of an exemplary narrow band cutoff multilayer optical film (MOF).

FIG. 14 is a chart showing the relationship between the wavelength, the polar angle, and the reflectance in the air of the MOF of FIG.

FIG. 14 is a chart showing the relationship between the wavelength, the polar angle, and the transmittance of the MOF in FIG. 14 in the air.

FIG. 14 is a chart showing the relationship between the wavelength, the polar angle, and the reflectance in the glass of the MOF of FIG.

FIG. 14 is a chart showing the relationship between the wavelength, the polar angle, and the transmittance of the MOF in FIG. 14 in the glass.

FIG. 14 is a chart showing the relationship between the wavelength, the polar angle, and the p-polarized light transmittance of the MOF in FIG. 14 in the glass.

FIG. 14 is a chart showing the relationship between the wavelength, the polar angle, and the s polarization transmittance of the MOF in FIG. 14 in the glass.

It is a chart which shows the spectrum of an exemplary dual band cutoff multilayer optical film (MOF).

FIG. 17 is a chart showing the relationship between the wavelength, the polar angle, and the reflectance in the air of the MOF of FIG.

FIG. 17 is a chart showing the relationship between the wavelength, the polar angle, and the transmittance of the MOF in FIG. 17 in the air.

It is understood that the features of the particular figures in this disclosure are not necessarily drawn in proportion to their actual size and that the figures represent non-exclusive examples of the techniques disclosed herein. I want to be.

The present disclosure describes a vehicle navigation system. In some examples, the vehicle navigation system according to the present disclosure may be used to decode an optically coded pattern of goods or an optical signature, such as a navigation aid or a road sign pattern or object.

The vehicle assistance system may include an autonomous driving assistance system (ADAS). Object sensing and detection in ADAS systems, for example with ADAS cameras or optical sensors, can pose challenges in terms of spectral resolution and polarization. In some examples, the systems and techniques according to the present disclosure may provide a method of increasing the signal-to-noise ratio in a compact and practical manner that is compatible with current imager systems. Optical filters may be combined with imager pixel arrays. In some examples, beam splitters may be used to allow for highly efficient and compact designs. In some examples, the beam splitter may allow high spatial resolution of the wavelength to be sensed or analyzed. For example, the entire imager is subject to a particular wavelength or band (eg, centered around 840 nm), as opposed to the imager, where only a few pixels can be associated with the wavelength or band of interest, to sense different bands or wavelengths. This can provide high resolution of variations in its wavelength or band (eg, 840 nm) throughout the image.

In some examples, the system acts as a transceiver, an optical filter that changes the wavelength of light incident on the imaging system and allows the decoding of optically coded article patterns or optical signatures. Includes components. The system may include an optical selection filter (eg, wavelength selection, polarization selection, or both), where the optical selection filter is a visible or invisible (UV and / or IR) wavelength or linearly polarized state or circle. Articles such as IR-coded signs or objects that selectively block polarization conditions, such as the unique spectral features of objects that land vehicles, aerial vehicles, or marine vehicles encounter or are in the vicinity of. Enhance detection of. The filter can be used as a self-supporting element or as a beam splitter component. The filter may be used in combination with one or more filters in the imager pixel array to analyze images with invisible spectral features. Unique signatures can be compared to known signatures and meaning lookup tables.

In some examples, the angular wavelength shift property of the multilayer optical film (MOF) may be used to convert the beam splitter imager to a hyperspectral camera in a vehicle assist system. The MOF may include a birefringence MOF. Such MOFs can exhibit good off-angle performance and relatively high angle shifts. For example, an angle-shift optical selection filter may be embedded in a beam splitter that optically communicates with the imager. In some examples, the pixel array adjacent to the imager contains at least one clear pixel. The pixel array may be in contact with the imager or may be spaced apart from the imager, but may be optically coupled to the imager. The system further includes an angle limiting factor for introducing light with various angles of incidence onto the filter surface. The system may include two imagers, one primarily for spectroscopy and the other for imaging. This makes it possible to enable a highly efficient imaging spectroscope or spectropolarizer for ADAS or vehicle assistance systems. Therefore, the problem of detection of the ADAS camera in terms of spectral resolution and polarization can be addressed. For example, both image information of the scene and spectral / polarization analysis can be performed.

In the present disclosure, "visible" refers to wavelengths in the range of about 400 nm to about 700 nm, and "infrared" (IR) refers to wavelengths in the range of about 700 nm to about 2000 nm, eg, about 800 nm to about 1200 nm. Refers to wavelengths in the range, including infrared and near infrared. Ultraviolet (UV) refers to wavelengths of about 400 nm or less.

FIG. 1 is a conceptual diagram of an exemplary vehicle support system 10 including an optical sensor 12a, a pixelated filter array 14a, and a full-field optical selection element 16 (also referred to as a “wavelength selection element”). The term "whole field" refers to the optical selection element 16 as having the optical sensor 12a and the pixelated filter array 14a so that all light incident on the optical sensor 12a or the pixelated filter array 14a passes through the optical selection element 16. It is shown that the whole of is optically covered. For example, the light from the optical selection element 16 may be output in parallel, at an angle, converged or divergent, or guided by other means, such as an optical sensor 12a or a pixelated filter. The array 14a can be substantially optically covered. In some examples, the system 10 is such that the light from the optical selection element 16 is optically diffused over the optical sensor 12a or the pixelated filter array 14a or covers the optical sensor 12a or the pixelated filter array 14a. It may include one or more optical elements for guiding to. The pixelated filter array 14a is adjacent to the optical sensor 12a (eg, in contact with the optical sensor 12a or spaced apart from the optical sensor 12a and optically coupled to it). The optical selection element 16 is adjacent to the pixelated filter array 14a (eg, in contact with the pixelated filter array 14a or spaced apart from the pixelated filter array 14a and optically coupled to it). Has been).

The optical selection element 16 may include an optical filter, a multilayer optical film, a microreplicated article, a dichroic filter, a retarder or wave plate, at least one beam splitter, or a combination thereof. The optical selection element 16 may include glass, one or more polymers, or any suitable optical material, or a combination thereof. In the example shown in FIG. 1, the full-field optical selection element 16 includes a beam splitter. In some examples, the beam splitter includes a polarizing beam splitter, a wavelength beam splitter, a dichroic prism, a trichromic prism, or a combination thereof. The beam splitter includes two triangular prisms, which are joined at their bottom surfaces (eg, by an adhesive) to form an interface 18. A dichroic coating or layer may be provided at the interface 18 for splitting the arriving optical signal into two or more spectral components, eg, components having different wavelengths or polarization states. The optical selection element 16 may have wavelength selectivity, polarization selectivity, or both. An optical coating or filter may be placed on one or more surfaces of the optical selection element 16 or the optical selection element 16 to filter a given wavelength or polarization state, eg, selectively absorb, transmit, or change. It may be provided adjacent to (eg, in contact with) one or more faces of. In some examples, the optical coating may include a waveplate or retarder, such as a half-wave retarder or a quarter-wave retarder, to change the direction of polarization or to convert linearly polarized light to circularly polarized light. .. In some examples, the optical coating comprises a spatially mutating wavelength selection filter. In the present disclosure, the term "polarized state" includes a linearly polarized state and a circularly polarized state. In some examples, the system 10 includes at least one polarizing filter across an optical path that reaches the optical sensor 12a. In some examples, the optical selection element 16 is an ultraviolet (UV) transmissive visible light reflective multilayer film filter, an ultraviolet (UV) transmissive visible light transmissive multilayer film filter, an edge filter, a transmission notch filter. ), Reflective notch filter, or at least one of a multi-band filter.

As shown in FIG. 1, the optical selection element 16 divides the optical signal L incident on the optical selection element 16 into two optical components C1 and C2, and the optical component C1 of the optical L is passed through the pixelated filter array 14a. It is selectively guided to the optical sensor 12a. In some examples, the second optical component C2 is discarded. In another example, the second optical component C2 is transmitted to another optical sensor. For example, the optical sensor 12a may include a first optical sensor and the system 10 may include a second optical sensor 12b. Similarly, the pixel filter array 14a may include a first pixel filter array and the system 10 may include a second pixel filter array 14b. Therefore, the optical selection element 16 can selectively guide the second optical component C2 to the second optical sensor 12b through the second pixelated filter array 14b. The pixelated filter arrays 14a, 14b can impose a predetermined light component onto a discrete region of the photosensor 12a and the photosensor 12b, i.e., a pixel. Thus, each pixel may include subpixels for one or more predetermined channels or components of light. For example, each pixel of the pixelated filter arrays 14a, 14b may contain one or more of the red, green, blue, or clear subpixels. Some examples of subpixel configurations of the pixelated filter array are described with reference to FIGS. 6A-6E.

In some examples, the pixelated filter arrays 14a, 14b may be integrated with, for example, an optical sensor 12a and an optical sensor 12b made in the same integrated chip, respectively. Therefore, the pixelated filter arrays 14a, 14b may be grown on the optical sensor 12a and the optical sensor 12b, or may be in direct contact with the optical sensor 12a and the optical sensor 12b otherwise.

The first optical component C1 and the second optical component C2 may differ in at least one wavelength band or polarization state or a combination thereof, where C2 is typically the optical complement of C1. be. In some examples, the first optical component C1 comprises at least the first ultraviolet wavelength band, the visible wavelength band, or the infrared wavelength band ( _{centered on λ 1} ), and the second optical component C2 is It includes at least a second ultraviolet band, a visible band, or an infrared band (centered on _{λ 2) different from the first band.} In some examples, the first wavelength band has a bandwidth of less than 200 nm and the second wavelength band comprises spectral complement of the first wavelength band. In some examples, the first wavelength band has a bandwidth of less than 100 nm or less than 50 nm. In some examples, the first wavelength band comprises at least one visible wavelength band and the second wavelength band comprises at least one near infrared band. In some examples, the first wavelength band includes at least one visible wavelength band and at least the first near-infrared band, and the second wavelength band includes at least the second near-infrared band. In some examples, the first wavelength band comprises at least one visible wavelength band and the second wavelength band comprises at least one UV band. In some examples, the first wavelength band comprises at least one visible wavelength band and the second wavelength band comprises at least a second visible wavelength band. In some examples, the first optical component C1 comprises a first polarization state and the second optical component C2 comprises at least a second polarization state different from the first polarization state. In some examples, the first optical sensor 12a functions as an imaging sensor and the second optical sensor 12b functions as a hyperspectral sensor.

In some examples, the optical selection element 16 includes an angle limiting optical element. In some examples, in addition to or instead of the angle limiting optics, the optical selection element 16 includes an angle diffusing optic. The angle limiting or diffusing element may include a refracting element, a diffractive element, a lens, a prism, a microreplicated surface or article, or a combination thereof. In some examples, the optical selection element 16 including the angular diffusion optical element can function as a spectroscope and emit different wavelengths at different angles.

The system 10 may include a computing device 20. The optical sensors 12a and 12b may electronically communicate with the computing device 20. The computing device 20 may include a processor 22 and a memory 24. The processor 22 may be configured to implement functions and / or processing instructions for execution within the computing device 20. For example, the processor 22 can process the instructions stored by the storage device, eg, the memory 24 in the computing device 20. Examples of processor 22 include microprocessors, controllers, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or equivalent discrete or integrated logic circuits. Or one or more can be mentioned. The memory 24 may include a look-up table containing a plurality of reference images.

The computing device 20 may receive at least one image data signal from the optical sensors 12a, 12b, and the processor 22 may be configured to compare the image data signal with a plurality of reference images. Processor 22 may be configured to generate an output signal based on comparison. The computing device 20 can transmit an output signal to the controller of the vehicle and cause the controller to take an action based on the output signal. This operation may include physical operation, communication operation, optical transmission, or control or operation of a sensor. In some examples, the computing device 20 itself may be the controller of the vehicle. For example, the computing device 20 may direct navigation or control the movement of the vehicle. The output signal is one of adjustment of navigation operation, search of response signal via communication network, search of vehicle environment information via communication network, or transmission of communication signal to target vehicle via communication network. It may be configured to do the above. Sensing and / or communication may be with another vehicle, but may also be with a portion of the infrastructure (such as a sign) or with a person. In some examples, the computing device 20 may communicate with a transceiver that may be mounted on a different vehicle, infrastructure component, or person.

FIG. 2 is a conceptual diagram of an exemplary system 30 including the vehicle support system 10 of FIG. 1 for detecting light deflected by an object 31. The object 31 may include any object that a land vehicle, an air vehicle, or a marine vehicle encounters or is in the vicinity of. For example, the object 31 includes road signs, construction equipment or signs, pedestrian coats or clothes, retroreflective signs, billboards, advertisements, navigation markers, mileage markings, bridges, pavements, road markings, and the like. obtain. In some examples, the system 30 includes an optical transmitter 32 configured to transmit light towards an object 31. Therefore, the photosensors 12a, 12b may be configured to sense the light from the phototransmitter 32 reflected or retroreflected by the object 31. In some examples, the system 10 may include a particular light transmitter, where the object 31 may reflect ambient light, such as sunlight, or light directed at the object 31 from multiple light sources. In some examples, the system 30 includes an optical selection element 16 that includes an optical filter instead of a beam splitter as shown in FIG.

As shown in FIG. 2, in some examples, the system 30 includes a housing 34. The housing 34 may include a rigid housing or a semi-rigid or soft housing that includes the optical sensors 12a, 12b, the pixelated filter arrays 14a, 14b, and the optical selection element 16. The housing 34 may be substantially opaque so that the optical components can be protected from stray light and the photosensors 12a, 12b are protected from inadvertent exposure to light. In some examples, the housing 34 defines an optical window 36 for selectively receiving light in the optical selection element 16, and finally in the optical sensors 12a, 12b. The optical window 36 may include a lens (eg, a fisheye lens), a refracting element, an optical filter, or may be substantially optically transparent. The housing 34 may be fixed or attached to a vehicle, such as an aerial vehicle, a marine vehicle, or a land vehicle at an appropriate location, area, or component. In some examples, the housing 34 may be fixed such that the optical window 36 faces a predetermined orientation with respect to the traveling direction of the vehicle, eg, forward, backward, or laterally. In some examples, the plurality of enclosures 34 may include a plurality of systems 10 or 30 located at different positions or oriented in different directions around or on the vehicle. The system 10 or system 30 may include a single optical selection element 16, but in another example, the exemplary system comprises two or more optical selection elements as described, for example, with reference to FIG. But it may be.

FIG. 3 is a conceptual diagram of an exemplary system 40 including a vehicle support system including cascaded optical selection elements 16a, 16b for detecting light deflected by an object 31. The system 40 is substantially similar to the exemplary system 30, but two optical selection elements 16a and 16b that are substantially similar to the single optical selection element 16 described with reference to FIGS. 1 and 2. including. The system 40 includes three optical sensors 12a, 12b, 12c and three pixelated filter arrays 14a, 14b, 14c. The first optical selection element 16a divides the incident light into two components, the first component being guided to the first photosensor 12a through the first pixelated filter array 14a. The second component is guided by a second optical selection element 16b that divides the second component into two further components (third component and fourth component), the third component being the second component. The fourth component is selectively guided to the third optical sensor 12c through the third pixelated filter array 14c, with the fourth component being selectively guided to the second optical sensor 12b through the pixelated filter array 14b. The system 40 may include three or more optical selection elements that similarly divide the set of optical components and selectively guide them to each sensor, as well as four or more optical sensors and a pixelated filter array. Different components may differ in at least one wavelength band or polarization state.

Instead of or in addition to the beam splitter, the system 10, 30, or 40 may include other optical selection elements, such as those described with reference to FIGS. 4 and 5.

FIG. 4 is a conceptual diagram of an exemplary optical selection element 16c including a cross-type dichroic splitter. A cross-type dichroic splitter (eg, also known as an "X-cube" or "RGB prism" available from WTS Photonics Technology Co., Ltd (Fuzhou, China)) is embedded in glass or another suitable optical medium. It can be defined by two dichroic interfaces 18b and 18c. In some examples, interface 18b can define red and green filters, and interface 18c can define a cyan filter across interface 18b. As can be seen in FIG. 4, the optical selection element 16c can substantially guide the three components C1, C2, and C3 of the incident light L along three distinct directions. In some examples, each of the three optical sensors can detect the three components separately. C1, C2, and C3 may correspond to any suitable predetermined combination of UV wavelength, visible wavelength, or IR wavelength and polarization state. In some examples, C1, C2, and C3 may correspond to red, green, and blue channels.

FIG. 5A is a conceptual diagram of an exemplary optical selection element 16d including a trichromatic prism. The trichroic prism can be defined by glass or any suitable optical medium and can include two dichroic interfaces 18d and 18e at a given angle. The dichroic interfaces 18d and 18e can function as a dichroic filter to guide different components of the incident light L, such as C1, C2, and C3, along three distinct directions. In some examples, each of the three optical sensors can detect the three components separately. C1, C2, and C3 are any suitable predetermined combination of UV wavelength, visible wavelength, or IR wavelength (eg, _{a band centered on the predetermined wavelengths λ 1} , λ ₂ , and λ ₃ ) and the polarization state. Can correspond to. In some examples, C1, C2, and C3 may correspond to red, green, and blue channels.

FIG. 5B is a conceptual diagram of an exemplary optical selection element 16e including a trichromatic prism. The trichromic prism can be defined by glass or any suitable optical medium and can include a dichroic interface between the prism element or the refracting element. The dichroic interface acts as a dichroic filter and can guide different components of the incident light L, such as C1, C2, and C3, along three distinct directions. In some examples, each of the three optical sensors can detect the three components separately. C1, C2, and C3 are any suitable predetermined combination of UV wavelength, visible wavelength, or IR wavelength (eg, _{a band centered on the predetermined wavelengths λ 1} , λ ₂ , and λ ₃ ) and the polarization state. Can correspond to. In some examples, C1, C2, and C3 may correspond to red, green, and blue channels.

Instead of or in addition to the pixelated filter arrays 14a, 14b, the system 10, 30, or 40 may include other pixelated filter arrays, eg, those described with reference to FIGS. 6A-6E.

FIG. 6A is a conceptual diagram of a Bayer color filter array. The Bayer color filter array contains a red pixel, a blue pixel, and two green pixels (RGGB) within each block. Although FIG. 6A shows specific relative arrangements of red, green, and blue pixels, other geometric arrangements may be used. The Bayer color filter array provides information about the intensity of light in the red, green, and blue wavelength regions by sending these wavelengths to the discrete regions of adjacent image sensors. The raw image data captured by the image sensor is then converted into a full-color image by a demosaic processing algorithm with the intensities of all three primary colors (red, green, blue) represented by each pixel or block. The Bayer color filter array has 25% R pixels, 25% B pixels, and 50% G pixels.

FIG. 6B is a conceptual diagram of a red / clear / clear / clear (RCCC) color filter array. In a vehicle assistance system or an advanced driver assistance system (ADAS), a plurality of cameras may capture a scene around the vehicle for assistance during transportation. Typical machine vision algorithms can use or analyze only the intensity of light. However, in ADAS, special color filter arrays may be created to provide color information. One of the useful color information channels is in the red channel, which helps identify areas of interest in the image, such as traffic lights, taillights of automobiles, and so on. For vehicle support applications, red / clear (RCCC) color filter arrays may be used. Unlike the Bayer sensor, the RCCC sensor uses a clear filter instead of the blue and two green filters of the 2x2 pixel pattern and has 75% of clear pixels that give light intensity information but not color information. Twenty-five percent of the pixels have red color information. The red filter remains the same. "Clear filter" is the same concept as a monochrome sensor. The advantage of this format is that it can function better in dark conditions, as it can be more sensitive to light. Thus, in some examples, the pixelated filter array according to the present disclosure may include at least one clear pixel, eg, a plurality of clear pixels.

FIG. 6C is a conceptual diagram of a monochrome filter array. Monochrome arrays have 100% "clear" pixels that provide light intensity information and no color information. This is acceptable for either monochrome viewing or analytical applications where no color information is required (eg, driver monitoring). The advantage of this format is that it is more sensitive to light and can work better in dark conditions.

FIG. 6D is a conceptual diagram of a red / clear / clear / blue (RCCB) color filter array. The RCCB is similar to the Bayer (RGGB), except that half of the pixels are clear rather than green. The advantage of this format is that clear pixels provide lower light sensitivity, which leads to lower noise. This format may allow the same camera to be used for visual and analytical applications.

FIG. 6E is a conceptual diagram of a red / clear / clear / blue (RCCB) color filter array. The RGCB is similar to the Bayer (RGGB), except that half of the green pixels are clear rather than green. The advantage of this format is that clear pixels provide lower light sensitivity, which leads to lower noise. This format may allow the same camera to be used for visual and analytical applications.

Clear pixels can be transparent at visible wavelengths, infrared wavelengths, or ultraviolet wavelengths, or one or more of these combinations. In some examples, clear pixels are substantially transparent only to visible wavelengths. In some examples, clear pixels are substantially transparent only to infrared wavelengths. In some examples, clear pixels are substantially transparent only to UV wavelengths. In some examples, clear pixels are substantially transparent only to visible and infrared wavelengths.

Different color filter arrays are available, but systems that do not include optical selection elements can present problems. For example, in the absence of optical selection elements, vehicle assistance systems or ADAS will limit spectral resolution in IR and UV, lack polarization information, signal loss when a spectrometer is used, signal loss due to filtering, and between channels. May exhibit poor contrast.

In an exemplary system according to the present disclosure, one or more optical selection elements, for example, separating channels to provide better contrast, eliminating or attenuating interfering wavelengths, spectral resolution in IR and UV. One or more of these problems can be addressed by improving and obtaining polarization information. Some examples of dividing light into different components by exemplary wavelength selection elements are described with reference to FIGS. 7A-7H.

FIG. 7A is a conceptual diagram of a full-field optical selection element 16f configured to reflect an infrared wavelength band and transmit visible light. FIG. 7B is a conceptual diagram of the full-field optical selection element 16f configured to reflect the first infrared wavelength band and transmit the second infrared wavelength band and visible light. FIG. 7C is a conceptual diagram of the full-field optical selection element 16h configured to reflect the first and second infrared wavelength bands and transmit the third infrared wavelength band and visible light. FIG. 7D is a conceptual diagram of the full-field optical selection element 16i configured to reflect the infrared wavelength band and transmit the visible light and the ultraviolet wavelength band. FIG. 7E is a conceptual diagram of the full-field optical selection element 16j configured to reflect the ultraviolet wavelength band and transmit the visible light and infrared wavelength bands. FIG. 7F is a conceptual diagram of a full-field optical selection element 16k configured to reflect an infrared wavelength band and an ultraviolet wavelength band and transmit visible light. FIG. 7G is a conceptual diagram of a full-field optical selection element 16l configured to reflect the first red and green wavelength bands and transmit the second and green wavelength bands as well as the blue wavelength band. FIG. 7H is a conceptual diagram of a full-field optical selection element 16m configured to reflect the s-polarized red wavelength band, transmit the p-polarized red wavelength band, and transmit the green and blue wavelength bands. .. By using a narrow band s-polarized reflector, it is possible to detect polarized light with reduced or minimized signal loss. Two image analyzes between the s-polarized image and the p-polarized image can be used for the polarization analysis of the scene. One example is the detection of road surface conditions, such as determining whether the pavement is wet. Another example is to eliminate surface glare so that the spectrum of the pavement display can be analyzed.

In the example of FIGS. 7A-7H, the filter is in the cube (diagonal interface), but in other examples, in addition to or instead of the filter on the diagonal of the cube, the filter is placed on the surface of the cube. May be done. In some examples, the diagonal film may include a half mirror used in combination with a wavelength selective cubic surface filter. The filter may include a narrowband reflection filter and a narrowband transmission filter.

FIG. 8 is a conceptual diagram of an exemplary optical selection element 16m including a lens 42. The lens 42 creates various angles of incidence on the filter within the optical selection element 16n (eg, a multilayer optical film), which results in an upward wavelength shift. The wavelength shift can be detected by an image sensor adjacent to the top surface. The second lens 44 can converge the light on the second image sensor adjacent to the right side of the optical selection element 16m.

FIG. 9 is a conceptual diagram of an exemplary visual field optical selection element 16o including a curved reflective interface 46. In some examples, the curved reflective interface 46 may include a curved multilayer optical film (MOF). This curvature gives rise to various angles of incidence that are later mapped to pixel positions. Each pixel position senses the effect of different reflection spectra. One particular curve is shown in FIG. 9, where the interface 46 is a straight line, arc, elliptical arc, parabolic or bicurve arc, face, spherical, elliptical, parabolic, bicurved, free plane. Alternatively, it may be arranged along any suitable geometric curve, composite curve, surface, or composite surface, including an arc, or a combination thereof. In some examples, as shown in FIG. 9, the interface 46 comprises an IR reflective visible light transmissive film. In IR, an angular wavelength shift occurs, providing light rays that diffuse to the top surface, while visible light passes through the right surface. Adjacent imagers may be placed on each surface to capture the separated components.

FIG. 10 is a conceptual diagram of an exemplary optical selection element 16p including a tilted reflector 48. The slanted reflector 48 can be used to generate two angles of incidence. In some examples, the slanted reflector 48 may be curved as well as the curved interface 46. The slanted or curved reflector 48 can separate the light into two components, as shown in FIG. The optical selection element 16o may include a filter 18f at the diagonal interface.

FIG. 11A is a conceptual diagram of a vehicle 50 including an automatic driving support system (ADAS). 11B is a conceptual partial front view of the vehicle 50 of FIG. 11A. ADAS may include system 10, or system 30 or system 40, described with reference to FIG. For example, the system 10, 30, or 40 may be mounted or included in a housing (eg, housing 34, or similar housing) fixed to the body or frame 52 of the vehicle 50. .. The system 10 can detect the light 54 deflected by the object 56. In some examples, the vehicle 50 may include a light source 58 that directs light 60 deflected (eg, reflected or retroreflected) to the system 10 by the object 56 toward the object 56. The light source 58 may include a headlight or vehicle light 62 or a dedicated light source 64 (or a combination thereof) separate from the headlight or vehicle light 62. Although the vehicle is shown in FIG. 11A, the vehicle 50 may include any land vehicle, marine vehicle, or aerial vehicle.

FIG. 12 is a flowchart of an exemplary method for sensing an optical signal by a vehicle assist system. An exemplary method of FIG. 12 will be described with reference to system 10 of FIG. 1 and system 30 of FIG. However, exemplary techniques can be performed using any suitable system according to the present disclosure. In some examples, the exemplary approach comprises receiving an optical signal L from an object 31 (70) by the full-field optical selection element 16 of the vehicle assist system 10. An exemplary technique comprises selectively guiding the optical component C1 of the optical signal L to the optical sensor 12a through the pixelated filter array 14a by means of an optical selection element 16 (72).

An exemplary method comprises receiving an image data signal from the image sensor 12a corresponding to the optical signal L by the computing device 20 (74). In some examples, the image data signal can correspond to a single image captured at a given moment. In another example, the image data signal may include a series of images captured in real time, near real time, or intermittently. In some examples, the light source 32 can irradiate an object 31 with an optical signal having a predetermined frequency or a predetermined time pattern, and the object 31 deflects a response signal having a response frequency or response time pattern. Can be done. In some such examples, the received optical signal L (74) may be synchronized with or asynchronous with the optical signal transmitted to the object 31.

An exemplary technique comprises comparing an image data signal with a plurality of reference images in a look-up table by a computing device 20 (76). The comparison may be for a single image captured at one moment, or may include a series of comparisons for a series of images captured in real time, near real time, or intermittently. .. In some examples, the lookup table may be implemented or replaced by a machine learning module, such as a deep learning model, or a convolutional neural network, or a pattern recognition module. Therefore, in some examples, the lookup table entry may correspond to the output of the machine learning module or pattern recognition module associated with the image. In some examples, the optical signal L may be generated by the object 31 in response to the optical signal having the spectrum S (λ) generated by the light source 32. The image sensor 12a and the pixelated filter array 14a may have a first wavelength transmission function T1 (λ). The optical selection element 16 may have a second transmission function T2 (λ). The object 31 may have a reflection spectrum R (λ). In such an example, the component of the signal L received by the image sensor 12a can correspond to S (λ) * T1 (λ) * T2 (λ) * R (λ), and is the computing device 20. Can compare S (λ) * T1 (λ) * T2 (λ) * R (λ) with the elements of the look-up table.

An exemplary approach comprises generating an output signal (78), depending on the comparison, by the computing device 20. In some examples, the output signal is the coordination of navigation behavior, the search for response signals over the communication network, the search for vehicle environment information over the communication network, or the transmission of communication signals to the target vehicle over the communication network. , Are configured to do one or more of the above.

The exemplary system or method according to the present disclosure may be implemented in a non-vehicle system, such as a handheld device, a wearable device, a computing device, etc., instead of the vehicle.

The techniques described in the present disclosure may be at least partially implemented in hardware, software, firmware, or any combination thereof. For example, various aspects of the techniques described may be one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integration. It can be implemented in one or more processors, including circuits or individual logic circuits, as well as any combination of such components. The term "processor" or "processing circuit" may generally refer to either the aforementioned logic circuit alone or in combination with other logic circuits, or other equivalent circuits. Control units, including hardware, can also perform one or more techniques of the present disclosure.

Such hardware, software, and firmware may be implemented within the same device or within separate devices to support the various techniques described in this disclosure. Moreover, any of the described units, modules or components can be implemented together or separately as separate but interoperable logical devices. The depiction of different features as a module or unit is intended to emphasize different functional aspects, and such module or unit must necessarily be realized by separate hardware, firmware, or software components. It doesn't mean that. Rather, the functions associated with one or more modules or units may be performed by separate hardware, firmware, or software components, or of common or separate hardware, firmware, or software. It may be integrated into a component.

The techniques described in the present disclosure may also be embodied or encoded in a computer system readable medium, such as a computer system readable storage medium containing instructions. When an instruction contained in or coded in a computer system readable medium is executed by one or more processors by an instruction embedded in or coded in a computer system readable medium, including a computer system readable storage medium. As such, it is possible to perform one or more of the techniques described herein by one or more programmable processors or other processors. Computer system readable storage media include random access memory (RAM), read-only memory (ROM), programmable read-only memory (ROM), erasable programmable read-only memory (EPROM), and electronically erasable programmable read-only memory (ROM). EEPROM), flash memory, hard disk, compact disk ROM (CD-ROM), floppy disk, cassette, magnetic medium, optical medium, or other computer system readable medium. In some examples, the manufactured article may include one or more computer system readable storage media.

Example 1
A virtual example of the coding pattern is described. FIG. 13 is a conceptual diagram of a coding pattern that can be read by the vehicle support system. The pattern contains two constructs that together define a two-dimensional (2D) QR barcode. The first construct comprises a first dye having a transition end at the first _{wavelength λ 1.} The second construct comprises a second dye having a transition end at a _second wavelength λ _{2 higher than λ 1.} When the pattern is illuminated or observed in a narrow wavelength range, a computing device that receives image data from an image sensor that captures the pattern in _{λ 1} and λ _{2 has a synthetic code as shown in FIG.} It can detect that it is made up of two separate codes. A computing device can combine two separate codes to generate a combined pattern and detect information from the combined pattern.

Example 2
A virtual embodiment of the optical selection element will be described. A narrowband cutoff multilayer optical film (MOF) having a primary reflection centered at 1000 nm and having a regulated secondary reflection is used. This bandwidth is adjusted from 50 nm to 200 nm. FIG. 14 is a chart showing the spectrum of an exemplary narrowband cutoff multilayer optical film (MOF). FIG. 15A is a chart showing the relationship between the wavelength, the polar angle, and the reflectance in the air of the MOF of FIG. FIG. 15B is a chart showing the relationship between the wavelength, the polar angle, and the transmittance of the MOF of FIG. 14 in the air. The light receiving angle is ± 40 °.

FIG. 16A is a chart showing the relationship between the wavelength, the pole angle, and the reflectance in the glass (glass beam splitter cube) of the MOF of FIG. FIG. 16B is a chart showing the relationship between the wavelength, the polar angle, and the transmittance of the MOF of FIG. 14 in the glass. FIG. 16C is a chart showing the relationship between the wavelength, the polar angle, and the p-polarized light transmittance of the MOF of FIG. 14 in the glass. FIG. 16D is a chart showing the relationship between the wavelength, the polar angle, and the s polarization transmittance of the MOF of FIG. 14 in the glass. Light enters the cube in the shape of a 45 ° ± 15 ° cone. This indicates a high angle shift and thus indicates the need for collimation optics to limit the angle of incidence of light on the film.

Example 3
A virtual example of a dual band optical selection element will be described. This element includes a filter made by laminating two multilayer optical films (MOFs) with single bands of 800 nm and 1000 nm, respectively. Multi-bands of 350 nm to 1000 nm or more can be used. FIG. 17 is a chart showing the spectrum of an exemplary dual band cutoff multilayer optical film (MOF). FIG. 18A is a chart showing the relationship between the wavelength, the polar angle, and the reflectance in the air of the MOF of FIG. FIG. 18B is a chart showing the relationship between the wavelength, the polar angle, and the transmittance of the MOF of FIG. 17 in the air. The two bands are individually adjustable.

Various embodiments of the present invention have been described. These and other examples are within the scope of the following claims.

Claims (34)

Optical sensor and
A pixelated filter array adjacent to the optical sensor and
A full-field optical selection element adjacent to the pixelated filter array, wherein the optical selection element selectively guides optical components of light incident on the optical selection element to the optical sensor through the pixelated filter array. With a full-field optical selection element, which is configured to
A vehicle support system equipped with. The system of claim 1, wherein the pixelated filter array comprises at least one clear pixel. The system according to claim 1, wherein the pixelated filter array is composed of a plurality of clear pixels. The system of claim 1, wherein the pixelated filter array comprises a Bayer color filter array (BCFA), a red / clear color filter array (RCCC), a red / clear blue color filter array (RCCB), or a monochrome array. The system according to any one of claims 1 to 4, wherein the full-field optical selection element includes an angle-restricting optical element. The system according to any one of claims 1 to 5, wherein the full-field optical selection element includes an angular diffusion optical element. The system according to any one of claims 1 to 6, wherein the full-field optical selection element comprises a curved multilayer optical film. The full-field optical selection element may be an ultraviolet (UV) transmissive visible light reflective multilayer film filter, an ultraviolet (UV) reflective visible light transmissive multilayer film filter, an edge filter, a transmissive notch filter, a reflective notch filter, or a multiband. The system of any one of claims 1-7, comprising at least one of the filters. The system according to any one of claims 1 to 8, wherein the full-field optical selection element includes a beam splitter. 9. The system of claim 9, wherein the full-field optical selection element further comprises at least one lens adjacent to the beam splitter. The system of claim 9 or 10, wherein the full-field optical selection element further comprises at least one tilt mirror adjacent to the beam splitter. 11. The system of claim 11, wherein the beam splitter includes a polarizing beam splitter, a wavelength beam splitter, a dichroic prism, a trichromic prism, or a combination thereof. The system of any one of claims 1-12, further comprising at least one lenticular element adjacent to the photosensor, configured to transmit substantially parallel light rays through the photosensor. .. Further comprising an optical transmitter configured to transmit light towards an object, the optical sensor is configured to sense light from the optical transmitter reflected or retroreflected by the object. , The system according to any one of claims 1 to 13. The system according to any one of claims 1 to 14, further comprising at least one optical element configured to direct light from the full-field optical selection element to the photosensor. The system according to any one of claims 1 to 15, comprising at least one polarizing filter across an optical path reaching the optical sensor. The optical sensor includes a first optical sensor, the pixelated filter array includes a first pixelated filter array, the system further comprises a second optical sensor, and the optical component is a first optical component. The system further comprises a second pixelated filter array, the full-field optical selection element comprising a second optical component of light incident on the optical selection element, said second pixelated filter array. The system according to any one of claims 1 to 16, which is configured to selectively guide across the second optical sensor. The first optical component includes at least a first ultraviolet wavelength band, a visible wavelength band, or an infrared wavelength band, and the second optical component is at least a second ultraviolet band different from the first band. 17. The system of claim 17, comprising the visible band, or the infrared band. 18. The system of claim 18, wherein the first wavelength band has a bandwidth of less than 200 nm and the second wavelength band comprises spectral complementation of the first wavelength band. The system according to any one of claims 17 to 19, wherein the first wavelength band includes at least one visible wavelength band, and the second wavelength band includes at least one near-infrared band. The first wavelength band includes at least one visible wavelength band and at least the first near-infrared band, and the second wavelength band includes at least a second near-infrared band, claims 17-19. The system described in any one of the above. The system according to any one of claims 17 to 19, wherein the first wavelength band includes at least one visible wavelength band, and the second wavelength band includes at least one UV band. The first aspect band includes at least one visible wavelength band, and the second wavelength band includes at least a second visible wavelength band, according to any one of claims 17 to 19. System. Any of claims 17-23, wherein the first optical component comprises a first polarized state, and the second optical component comprises at least a second polarized state different from the first polarized state. The system described in paragraph 1. The system according to any one of claims 17 to 24, wherein the first optical sensor includes an imaging sensor and the second optical sensor includes a hyperspectral sensor. The system according to any one of claims 1 to 25, further comprising a retarder adjacent to the full-field optical selection element. Further comprising a housing, the optical sensor, the pixelated filter array, and the full-field optical selection element are fixed adjacent to each other within the housing, the housing being at least one for receiving light. The system according to any one of claims 1 to 26, which defines two optical windows. Further comprising a computing device configured to receive an image data signal from an image sensor, said computing device.
Memory with a look-up table with multiple reference images,
A processor configured to compare the image data signal with the plurality of reference images and generate an output signal according to the comparison.
The system according to any one of claims 1 to 27. The output signal includes adjustment of navigation operation, search of response signal via communication network, search of vehicle environment information via communication network, or transmission of communication signal to target vehicle via communication network. 28. The system of claim 28, which is configured to perform one or more of the above. The system according to any one of claims 1 to 29, comprising an advanced driver assistance system (ADAS). A land, water, or air vehicle comprising the system according to any one of claims 1 to 30. Receiving an optical signal from an object by the full-field optical selection element of the vehicle assistance system,
The full-field optical selection element selectively guides the optical component of the optical signal to the optical sensor through a pixelated filter array.
Including, how. Receiving an image data signal from the image sensor in response to the optical signal by the computing device.
Comparing the image data signal with a plurality of reference images in a look-up table by the computing device.
Producing an output signal by the computing device in response to the comparison.
32. The method of claim 32. The output signal includes adjustment of navigation operation, search of response signal via communication network, search of vehicle environment information via communication network, or transmission of communication signal to target vehicle via communication network. 33. The method of claim 33, which is configured to cause one or more of the above.

Images