CN113596360A - Infrared sub-pixel polarized light imaging device and imaging method - Google Patents

Abstract

The application discloses infrared sub-pixel polarized light imaging device and imaging method, on the basis of using dislocation rhombus array mode, through the pixel to containing different polarization angle polarization information rebuild, can reduce the pixel interval by a wide margin, because the size of pixel interval has decided the resolving power, consequently can accomplish under the unchangeable prerequisite of resolution ratio, reduce optical area by a wide margin, reduced the sensor volume, reduce the size and the cost of camera lens simultaneously, the complete machine cost reduces by a wide margin.

Description

Infrared sub-pixel polarized light imaging device and imaging method

Technical Field

The present disclosure relates to infrared imaging devices, and particularly to an infrared sub-pixel polarized light imaging device and an imaging method.

Background

In order to acquire an infrared polarized light image, a layer of polarizing plate is usually added above an acquisition unit of a conventional infrared polarized light imaging sensor, for example, N polarizing plates with different angles are respectively placed on a single pixel, and each group of N pixels is used as a calculation unit, which results in that the resolution is reduced to 1/N.

Therefore, in order to improve the resolution, it is necessary to increase the number of pixels in a unit optical area, i.e. to reduce the size of the pixels, but when the pixels are reduced to a certain size, crosstalk occurs in signals of adjacent pixels due to the constraint effect of the diffraction limit, when a beam of light is focused on a single pixel, it will rapidly defocus on the surface of the pixel due to the diffraction effect to form a light cone section, i.e. airy spot, and the size of the pixel should not be smaller than the radius of the airy spot, so that it is extremely difficult to maintain the resolution by increasing the number of pixels in a unit optical area.

Disclosure of Invention

In view of the foregoing, the present application provides an infrared sub-pixel polarized light imaging method.

In order to solve the technical problem, the following technical scheme is adopted in the application:

in a first aspect, the present application provides an infrared sub-pixel polarized light imaging device, comprising:

an array of polarizing devices configured to convert incident light into polarized light;

a pixel array configured to produce a varying resistance in response to polarized light; the pixel arrays are arranged in a staggered diamond array; the pixel array comprises a plurality of pixel groups, each pixel group comprises four pixels, and the polarization angles of the four pixels corresponding to the polarization devices are different;

a readout circuit configured to read out an electrical signal according to the changed resistance;

an ADC configured to convert the electrical signal into infrared image raw data;

an interface circuit configured to transmit infrared image raw data to an image processor;

an image processor configured to create, at each pel center position, a first pixel from nine pixels of a four pixel group comprising the pel; establishing a second pixel at the center position of each pixel group according to four pixels of the pixel group where the position is located;

preferably, the polarizing means is a metal wire grid.

Preferably, the polarization angles of the four pixels in the pixel group corresponding to the polarization means are 0 °, 45 °, 90 ° and 135 °, respectively.

In a second aspect, the present application provides a method of imaging, the method comprising:

acquiring original data of an infrared polarization image;

at the center position of each pixel element, establishing a first pixel according to nine pixels of four pixel groups including the pixel element;

and at the central position of each pixel group, establishing a second pixel according to four pixels of the pixel group at which the position is located.

Preferably, the first pixel is established at the center of each pixel element according to nine pixels of four pixel groups including the pixel element, specifically:

at each pixel center position, calculating a first arithmetic mean value of four pixels having horizontal angles of 0 °, 90 °, 180 ° and 270 ° with respect to the position among nine pixels including four pixel groups of the pixel, calculating a second arithmetic mean value of two pixels having horizontal angles of 45 ° and 225 ° with respect to the position, calculating a third arithmetic mean value of two pixels having horizontal angles of 135 ° and 315 ° with respect to the position, obtaining a Stokes vector, a linear polarized light component in the 0 degree direction and a linear polarized light component in the 45 degree direction according to the pixel value, the first arithmetic mean value, the second arithmetic mean value and the third arithmetic mean value at the pixel position, and acquiring a polarization degree and a polarization angle according to the Stokes vector, and establishing a first pixel according to the 0-degree direction linear polarized light component, the 45-degree direction linear polarized light component, the polarization degree and the polarization angle.

Preferably, the establishing of the second pixel at the center position of each pixel group according to four pixels of the pixel group at which the position is located specifically is:

at the central position of each pixel group, according to four pixel values of four different polarization angles in the pixel group, a Stokes vector, a linear polarized light component in the 0-degree direction and a linear polarized light component in the 45-degree direction are obtained, according to the Stokes vector, a polarization degree and a polarization angle are obtained, and a second pixel is established according to the linear polarized light component in the 0-degree direction, the linear polarized light component in the 45-degree direction, the polarization degree and the polarization angle.

Compared with the prior art, the method has the following beneficial effects:

based on the technical scheme, the infrared sub-pixel polarized light imaging device and the imaging method can greatly reduce the pixel spacing by reconstructing the pixels containing different polarization angle polarization information on the basis of using the staggered diamond array mode, and greatly reduce the optical area, the sensor volume, the size and the cost of a lens and the cost of the whole machine on the premise of keeping unchanged resolution ratio due to the fact that the resolution capability is determined by the size of the pixel spacing.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic diagram of a pixel group according to an embodiment of the present application.

Fig. 2 is a flowchart of an imaging method according to an embodiment of the present disclosure.

Fig. 3 is a schematic diagram of a method for creating a first pixel according to an embodiment of the present disclosure.

Fig. 4 is a schematic diagram of a method for creating a second pixel according to an embodiment of the present disclosure.

Fig. 5 is a pixel distribution of an infrared sub-pixel polarized light image according to an embodiment of the present disclosure.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or device that comprises a list of elements does not include only those elements but may include other elements not expressly listed. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of additional like elements in the article or device comprising the element.

Before describing the infrared sub-pixel polarized light imaging device and the imaging method provided by the present application, polarized light imaging is first described.

After the natural light is irradiated, the amplitudes of the electric vector vertical component and the parallel component in the reflected light are changed, so that the reflected light is not isotropic natural light any more and is partially polarized light or linearly polarized light, polarization characteristics determined by the properties of the reflected light and the electromagnetic radiation process of an object can be generated, different states (such as roughness, water content, material physical and chemical characteristics and the like) of different objects or the same object often have different polarization states in a thermal infrared band, and by utilizing an infrared polarization imaging technology that a target shows radiation or reflected polarization information, multi-dimensional characteristic information such as the intensity, polarization, images and the like of the target can be comprehensively obtained, the contrast ratio of the target and a background is effectively improved, the detailed characteristics of the target are highlighted, the target identification effect is enhanced, and the attributes and behaviors of the target are more comprehensively and deeply known.

The infrared sub-pixel polarized light imaging device comprises a polarizing device array, a pixel array, a reading circuit, an ADC and an image processor.

An array of polarizing devices configured to convert incident light into polarized light.

A pixel array configured to produce a varying resistance in response to polarized light; the pixel arrays are arranged in a staggered diamond array; the pixel array comprises a plurality of pixel groups, each pixel group comprises four pixels, and the polarization angles of the four pixels corresponding to the polarization devices are different.

The pixel containing the polarization device can receive polarization characteristic information reflected by an external object, the polarization device can be an adjustable polarization element additionally arranged at the front section of the camera, and a metal wire grating array is manufactured on the pixel by adopting an MEMS (micro electro mechanical system) process, and the polarization device is integrated on the pixel.

In one embodiment, referring to fig. 1, fig. 1 is a schematic diagram of a pixel group provided in an embodiment of the present application, and the polarization angles of the polarization devices corresponding to four pixels in the pixel group are 0 °, 45 °, 90 °, and 135 °, respectively.

A readout circuit configured to read out the electrical signal according to the changed resistance.

The reading circuit is composed of a plurality of columns, each column of the reading circuit is electrically connected with all the pixels of the column, the pixels receive infrared waves radiated by an object, the resistance changes, the row selection switches of two adjacent rows of the pixels are switched on each time, the resistance changed by the pixels of two adjacent rows is read, and the read signals are in an analog signal format.

An ADC configured to convert the electrical signal into an image digital signal.

The interface circuit is used for transmitting the infrared polarization image digital signals output by the ADC to the image processor, so that subsequent image analysis processing is facilitated.

The interface circuit may include one of an AER interface, a Mobile Industrial Processor Interface (MIPI), and a parallel interface.

An image processor configured to create, at each pel center position, a first pixel from nine pixels of a four pixel group comprising the pel; and at the central position of each pixel group, establishing a second pixel according to four pixels of the pixel group at which the position is located.

The image processor may specifically include, but is not limited to, a Graphics Processor (GPU), a Central Processing Unit (CPU), an Arithmetic Logic Unit (ALU), a Digital Signal Processor (DSP), a microcomputer, a Field Programmable Gate Array (FPGA), a system on chip (SoC), a programmable logic unit, a microprocessor, an Application Specific Integrated Circuit (ASIC), and the like.

Fig. 2 is a flow chart of an imaging method provided herein.

The imaging method provided in the present application with reference to fig. 2 includes: s201, acquiring original data of an infrared polarization image; s202, establishing a first pixel at the center of each pixel element according to nine pixels of four pixel groups including the pixel element; s203, at the center position of each pixel group, establishing a second pixel according to four pixels of the pixel group where the position is located.

In the process of S201, raw data of the infrared polarization image is acquired.

When the thermosensitive film layer in the pixel is irradiated by infrared rays, the resistance of the thermosensitive film layer changes, so that an electric signal can be generated, the electric signal is converted into original data of an infrared polarization image through the reading circuit, the ADC and the interface circuit, and the original data is sent to the image processor.

In S202, at each pixel center position, a first pixel is established from nine pixels of a four-pixel group including the pixel.

Fig. 3 is a schematic diagram of a method for creating a first pixel provided in the present application.

Referring to fig. 3, at each pixel center position, a first arithmetic average of four pixels horizontally included at 0 °, 90 °, 180 °, and 270 ° from among nine pixels of four pixel groups including the pixel is calculated, a second arithmetic average of two pixels horizontally included at 45 ° and 225 ° from the position is calculated, and a third arithmetic average of two pixels horizontally included at 135 ° and 315 ° from the position is calculated.

And acquiring a Stokes vector, a 0-degree direction linearly polarized light component and a 45-degree direction linearly polarized light component according to the pixel value, the first arithmetic mean value, the second arithmetic mean value and the third arithmetic mean value at the pixel position, wherein the 0-degree direction linearly polarized light component is the difference between the 0-degree direction polarized image and the 90-degree direction polarized image, and the 45-degree direction linearly polarized light component is the difference between the 45-degree direction polarized image and the 135-degree direction polarized image.

And acquiring a polarization degree and a polarization angle according to the Stokes vector, wherein the polarization degree and the polarization angle are important parameters for representing the surface state and the attribute characteristics of the target scenery. The degree of polarization is a dimensionless number from 0 to 1 and can be generally calculated from the stokes vector. The included angle of the polarization direction of the incident light relative to the 0 degree direction is shown, and for the partially polarized light, the polarization angle is the included angle of the polarization direction with the largest energy and the 0 degree direction. The polarization angle is also expressed as a function of the stokes parameter.

The first pixel is created from the 0 ° direction linearly polarized light component, the 45 ° direction linearly polarized light component, the degree of polarization, and the polarization angle. For example, the first pixel is obtained by directly synthesizing the fused pixels by an arithmetic weighting method.

In the process of S203, at the center of each pixel group, a second pixel is established according to four pixels of the pixel group where the position is located.

Fig. 4 is a schematic diagram of a method for creating a second pixel provided in the present application.

Referring to fig. 4, at the central position of each pixel group, according to four pixel values of four different polarization angles in the pixel group, a stokes vector, a 0 ° direction linearly polarized light component, and a 45 ° direction linearly polarized light component are obtained, according to the stokes vector, a polarization degree and a polarization angle are obtained, and a second pixel is established according to the 0 ° direction linearly polarized light component, the 45 ° direction linearly polarized light component, the polarization degree, and the polarization angle. The calculation process for creating the second pixel using the four pixel values around the position is similar to the calculation process for creating the first pixel, and will not be described in detail here.

After obtaining a plurality of first pixels and a plurality of second pixels, the pixel distribution of the infrared sub-pixel polarized light image is as shown in fig. 5, on the basis of using the staggered diamond array mode, the imaging method provided by the application can reduce the pixel pitch from L to 1.414L/2, reduce the pixel pitch, and the resolution capability is determined by the size of the pixel pitch, so that the effect of substantially reducing the number of pixels but increasing the pixel density is achieved, and the spatial resolution of the imaging device is improved.

The infrared sub-pixel polarized light imaging device provided by the application can be installed in an electronic device with an image sensing function and/or a light sensing function. For example, the infrared sub-pixel polarized light imaging device may be installed in electronic devices such as smartphones, cameras, internet of things (IoT) devices, tablet Personal Computers (PCs), Personal Digital Assistants (PDAs), Portable Multimedia Players (PMPs), navigation devices, drones, and Advanced Driver Assistance Systems (ADAS). In addition, the infrared sub-pixel polarized light imaging device may be provided as an element in vehicles, electronic devices, smart furniture, manufacturing devices, various measurement devices, and the like.

The foregoing is only a preferred embodiment of the present invention, and although the present invention has been disclosed in the preferred embodiments, it is not intended to limit the present invention. Those skilled in the art can make numerous possible variations and modifications to the present teachings, or modify equivalent embodiments to equivalent variations, without departing from the scope of the present teachings, using the methods and techniques disclosed above. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.

Claims (6)

1. An infrared sub-pixel polarized light imaging device, comprising:

an array of polarizing devices configured to convert incident light into polarized light;

a pixel array configured to produce a varying resistance in response to polarized light; the pixel arrays are arranged in a staggered diamond array; the pixel array comprises a plurality of pixel groups, each pixel group comprises four pixels, and the polarization angles of the four pixels corresponding to the polarization devices are different;

a readout circuit configured to read out an electrical signal according to the changed resistance;

an ADC configured to convert the electrical signal into infrared image raw data;

an interface circuit configured to transmit infrared image raw data to an image processor;

an image processor configured to create, at each pel center position, a first pixel from nine pixels of a four pixel group comprising the pel; and at the central position of each pixel group, establishing a second pixel according to four pixels of the pixel group at which the position is located.

2. An imaging apparatus according to claim 1, wherein the polarizing means is a wire grid.

3. The imaging apparatus of claim 1, wherein the polarization angles of the corresponding polarizing means of the four pixels in the set of pixels are 0 °, 45 °, 90 ° and 135 °, respectively.

4. A method of imaging, the method comprising:

acquiring original data of an infrared polarization image;

at the center position of each pixel element, establishing a first pixel according to nine pixels of four pixel groups including the pixel element;

and at the central position of each pixel group, establishing a second pixel according to four pixels of the pixel group at which the position is located.

5. The method according to claim 4, wherein the first pixel is created from nine pixels comprising a group of four pixels of the pixel, in each pixel center position, specifically:

at each pixel center position, calculating a first arithmetic mean value of four pixels having horizontal angles of 0 °, 90 °, 180 ° and 270 ° with respect to the position among nine pixels including four pixel groups of the pixel, calculating a second arithmetic mean value of two pixels having horizontal angles of 45 ° and 225 ° with respect to the position, calculating a third arithmetic mean value of two pixels having horizontal angles of 135 ° and 315 ° with respect to the position, obtaining a Stokes vector, a linear polarized light component in the 0 degree direction and a linear polarized light component in the 45 degree direction according to the pixel value, the first arithmetic mean value, the second arithmetic mean value and the third arithmetic mean value at the pixel position, and acquiring a polarization degree and a polarization angle according to the Stokes vector, and establishing a first pixel according to the 0-degree direction linear polarized light component, the 45-degree direction linear polarized light component, the polarization degree and the polarization angle.

6. The method according to claim 4, wherein the second pixel is established at a central position of each pixel group according to four pixels of the pixel group where the position is located, specifically:

at the central position of each pixel group, according to four pixel values of four different polarization angles in the pixel group, a Stokes vector, a linear polarized light component in the 0-degree direction and a linear polarized light component in the 45-degree direction are obtained, according to the Stokes vector, a polarization degree and a polarization angle are obtained, and a second pixel is established according to the linear polarized light component in the 0-degree direction, the linear polarized light component in the 45-degree direction, the polarization degree and the polarization angle.

Images