JP2021021599A - Time Polarization Correlation Imaging Method and Time Polarization Correlation Imaging System - Google Patents

Abstract

To provide a time polarization correlation imaging method capable of a new time correlation imaging that requires no high-speed voltage modulation on a light source and image sensor, and a system thereof.SOLUTION: In the time polarization correlation imaging method for imaging based on polarization correlation information obtained from reflected light by projecting light from a light source 5 onto a target object 6, the polarized state of the light from the light source 5 is modulated in time by active polarization optics 10 and projected to the target object 6. The reflected light on the target object 6 is detected with a detector 1 after passing through the active polarization optics 10 to thereby obtain polarization correlation information from the detection result.SELECTED DRAWING: Figure 2

Description

The present invention utilizes the fact that the speed of light is known, receives the light bounced off from the target object, and calculates the delay time thereof. The present invention relates to a time polarization correlation imaging method and a time polarization correlation imaging system that enable distance measurement, material, and three-dimensional shape measurement.

In the measurement scene, it is common to give a specific input to the target field and see the response, and obtain the target information from the response generated in response to the given input. A typical example is the correlation detection of the entry / exit relationship. Conventionally, the application has been mostly by a point sensor, but as a surface detection means, a time correlation image sensor is known as a two-dimensional enablement of imaging and correlation detection in real time.

A camera using a time-correlated image sensor has the ability to record the time change of the light that reaches each pixel of the captured image, so that it can be applied differently from a normal camera. For example, it can be put to practical use for measurement and inspection, such as displaying the speed and direction of a moving part in an image from one image data, displaying the material making an object, changes in unevenness, scratches, etc. Research and development aiming at is progressing.

For example, a Time-OF-Flight (TOF) camera that measures distance and velocity data in real time using flight time data rather than relying on brightness data is known. The TOF camera has a light source that emits pulses, and the image sensor records the time change as a time-series signal. Then, it is attracting attention as a means for measuring flight time and independently measuring distance in pixel units.

The image sensor used in the optical flight time measurement method includes a photoelectric conversion unit and a plurality of charge storage units, and discriminates electrons generated in the photoelectric conversion unit by the incoming light at the timing of the incoming light. Then, there is a charge distribution method in which the discriminated electrons are distributed and accumulated in a plurality of charge storage units. The pixel structure of the charge distribution type image pickup device is described in, for example, Patent Document 1.

In addition, the system in which the TOF function is embedded in each pixel of the image sensor emits modulated light energy whose phase is known, and examines the phase shift of the optical signal reflected from the object to the TOF system and returned. For example, Patent Document 2 describes a phase-type TOF system that confirms the depth distance to an object.

Japanese Patent No. 5205002 Japanese Patent No. 45333582 Japanese Unexamined Patent Publication No. 2016-153795 JP-A-2019-015745

The ones described in Patent Documents 1 and 2 are time-correlation imaging devices using an image sensor in which both a CMOS compatible pixel detector for detecting light energy and a related processing circuit configuration are manufactured on one IC. However, it is difficult for the TOF camera to output an ideal emission waveform due to the restriction that the light source needs to be modulated at high speed. Therefore, it is difficult to improve the measurement accuracy.

In addition, the image sensor requires a complicated circuit for each pixel, and it is indispensable to reduce the light receiving area and the sensitivity accordingly. In addition, there are problems such as difficulty in increasing the number of pixels and an increase in manufacturing cost. Further, the prior art is a lock-in detection method using a time window (clock) synchronized with a light source, and an error occurs in the amount of charge accumulated due to switching noise associated with high-speed switching. Then, measures such as dark current, reset noise, and shot noise were required.

Further, those described in Patent Documents 3 and 4 are devices that replace the image pickup element in the time correlation imaging device with a simple circuit. Instead of performing lock-in detection on the circuit, a polarizing plate is installed in front of the light receiving part, and the degree of polarization is changed by an optical phase modulator that synchronizes the light passing through the polarizing plate with the light source to measure the degree of polarization. The time correlation is calculated by the system. However, it is difficult to output an ideal emission waveform due to the restriction that the light source needs to be modulated at high speed. Further, it is difficult in principle to separate the influence of the reflected light on the surface of the object and the internal scattering of the light inside the target object, and the influence of the internal scattering disturbs the result of the time correlation.

An object of the present invention is to solve the above-mentioned problems of the prior art and to provide a time polarization correlation imaging method and a system thereof as a new time correlation imaging that does not require high-speed voltage modulation on a light source and an image sensor.

In order to achieve the above object, the present invention is a time polarization correlation imaging method in which light from a light source is projected onto an object and imaging is performed based on polarization correlation information obtained from the reflected light, and the light from the light source is used. The polarization state is temporally modulated by the active polarization optical system (first active polarization optical system) and projected onto the target object, and the reflected light on the target object is projected onto the target object by the active polarization optical system (second active polarization optical system). After passing through the light, it is detected by a detector, and polarization correlation information is obtained from the detection result.

Further, in the active polarization optical system, the light from the light source is linearly polarized by a polarizing plate, passed through an electro-optical element whose phase is delayed according to an applied voltage, and passed on a 1 / 4λ wave plate to pass the target object. It is desirable to project to.

Further, it is desirable that the active polarizing optical system projects light from the light source onto the target object by passing or reflecting light from a magnetic optical element capable of manipulating the linear polarization direction according to an applied magnetic force.

It is desirable to apply an applied voltage or an applied magnetic force so that the polarized state has two states, one in an arbitrary direction and the other in a direction orthogonal to the polarized state, by the active polarization optical system.

In order to achieve the above object, the present invention is a time polarization correlation imaging system in which light from a light source is projected onto an object and imaging is performed based on polarization correlation information obtained from the reflected light. In the time polarization correlation imaging system, the polarization state of the light from the light source is achieved. From the light source modulated by the active polarization optical system, comprising an active polarization optical system that temporally modulates the light, and a detector that detects light reflected by the target object after passing through the active polarization optical system. The light of the above is projected onto the target object, and polarization correlation information is obtained from the detection result by the detector.

Further, the active polarization optical system includes a polarizing plate that controls the polarization state of light from the light source, a wave plate that controls the phase of light, and electricity that can actively manipulate the polarization state according to an applied voltage or an applied magnetic force. It is desirable to project light on the target object, which includes an optical element or a magnetic optical element and whose polarization state is controlled as a function of an arbitrary time.

Further, it is desirable that an applied voltage is applied to the electro-optical element or the magneto-optical element so that the polarization state is in the horizontal direction and the vertical direction.

Further, it is desirable that the detector is an imaging element in which the photoelectric conversion unit is arranged in an array, and the polarization correlation information having different times in synchronization with the applied voltage is obtained.

Further, the detector is an imaging element in which photoelectric conversion units are arranged in an array, and the polarization is detected by using an asynchronous detector that captures a change in brightness observed by each photoelectric conversion unit as an event and outputs the information. It is desirable to obtain correlation information.

Further, the image pickup element includes a photoelectric conversion unit to which a polarizer having a different direction of linear polarization is applied and a corresponding charge storage unit, and is generated by the photoelectric conversion unit having a different polarization direction due to the reflected light from the target object. It is desirable to store the photocurrent in the charge storage unit.

Further, the image sensor is arranged between the photodiode, which is the photoelectric conversion unit, and the photodiode and the charge storage unit, and has a gate structure that controls the transfer of the photocurrent to the plurality of charge storage units. It is desirable that the CMOS image sensor has.

Further, it is desirable that the polarization correlation information is a phase difference.

In the present invention, the polarization state of light from a light source is time-modulated by an active polarization optical system and projected onto a target object. Then, after passing the reflected light through the active polarization optical system, it is detected by a detector, and time polarization correlation imaging is performed from the detection result. Therefore, time polarization correlation imaging does not require high speed voltage modulation on the light source and the image sensor. As a result, it is easy to improve the measurement accuracy, and the problem of the semiconductor integrated circuit associated with high-speed voltage modulation can be solved for both the light source and the image sensor.

A block diagram showing a basic structure of an active polarizing optical system according to an embodiment of the present invention. Block diagram showing the basic configuration of the time polarization correlation measurement optical system according to one embodiment The figure which shows the polarization state of the active polarization optical system 10 by one Embodiment. Time change diagram of each part according to one embodiment The figure which shows the correlation value of the time polarization correlation when the frequency and the distance are changed. The figure which shows the correlation value of the time correlation when the frequency and the distance are changed.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing the basic structure of the active polarization optical system 10, and FIG. 2 is a block diagram showing the basic configuration of the time polarization correlation measurement optical system.

One embodiment is a new time correlation imaging method that performs imaging based on polarization correlation information obtained from photographing the light reflected from the target object 6. The new time correlation imaging method is a time polarization correlation imaging method that obtains time polarization correlation information by using an active polarization optical system 10 that can change the polarization state by electric drive. And the new time-altered correlation imaging method does not require high-speed voltage modulation on the light source 5 and the image sensor.

Further, in the polarized state by electric drive, the light source 5 may be turned on with a constant brightness and light may be passed through a polarizing plate rotating at high speed. Then, the light that has passed through the polarizing plate is reflected on the target object 6 and is passed through the polarizing plate that is rotating at high speed again. At this time, the polarization state of the light is different between the time of emission and the time of incidence. Therefore, time-polarization correlation imaging is possible by obtaining time-polarization correlation information when the polarizing plate is incident and observing it with an image sensor.

However, it is difficult to mechanically rotate the polarizing plate at a speed at which the moving speed of light can be detected. In one embodiment, the created light polarized in an arbitrary direction is projected onto the target object 6 by using an optical system which can be called an active polarization optical system 10 which can change the polarization state at high speed by electric drive. Then, after passing the reflected light on the target object 6 through the active polarization optical system 10, the detector 1 detects it, and the time polarization correlation information is acquired from the detection result.

Light polarized in an arbitrary direction is produced by the active polarization optical system 10 of FIG. 1, and the produced light having an arbitrary polarization is reflected by the target object 6 as shown in FIG. Then, the reflected light is observed by the detector 1 after passing over the active polarizing optical system 10. In the active polarization optical system 10, the side that projects polarized light onto the target object 6 and transmits it is the first active polarization optical system, and the side that receives the reflected light is the second active polarization optical system. It is also possible to use different active polarizing optical systems on the receiving side and different modulations on the transmitting side and the receiving side, or the same ones.

The active polarization optical system 10 is an optical system capable of creating an arbitrary polarization direction state by voltage driving. In FIG. 1, the light emitted from the light source 5 is linearly polarized by the polarizing plate 2, and the linearly polarized light passes through the electro-optical element 3. The electro-optical element 3 is, for example, a polarizer of Carcel or Pockels, and is an element that delays the phase of the slow-phase axis according to the applied voltage. The slow axis of the electro-optical element 3 and the direction of linearly polarized light are set to 45 degrees. As a result, the light that has passed through the electro-optical element 3 becomes arbitrary elliptically polarized light.

The linearly polarized light in an arbitrary direction can be created by finally passing through the 1 / 4λ wave plate 4 in which the original linearly polarized light direction and the direction of the slow phase axis are aligned. That is, the active polarization optical system 10 can create an arbitrary polarization state by using the electro-optical element 3 whose phase delay can be controlled. This is known as the Sernamon method for measuring the phase delay of liquid crystals and the like.

FIG. 3 is a diagram showing the polarization state of the active polarization optical system 10, and the polarization state of the active polarization optical system 10 can be expressed as a transition of Poincare spheres as shown in FIG. In the Poancare sphere, the polarized state is represented by points on the sphere, the points on the equator represent the linearly polarized state, and the North Pole and South Pole represent circularly polarized light. When the starting point of linear polarization is taken on the x-axis and the y-axis is the rotation axis, the rotation angle in the latitude direction corresponds to the phase delay amount. The Poincare sphere is converted into linearly polarized light, elliptically polarized light, and circularly polarized light by changing the phase delay amount.

The light emitted from the light source 5 is linearly polarized by the polarizing plate 2, and when it passes through the electro-optical element 3, it rotates around the y-axis by the relative phase delay amount ψ of the electro-optical element 3 in the latitude direction. Subsequently, when light passes over the 1 / 4λ wave plate, 90 degree (π / 4) polarized light rotates around the x-axis on the Poancare sphere of FIG.

As a result, linearly polarized light rotated by φ / 2 is obtained. Assuming that the phase delay amount of the electro-optical element 3 is φ (phase difference ψ / 2π), the rotation angle φ in the polarization direction in the active polarization optical system 10 is ψ / 2. Therefore, an arbitrary polarization state can be created by creating a difference in refractive index between the phase-advancing axis and the slow-phase axis of the electro-optical element 3.

The electro-optical element 3 is a polarizing element using a crystalline material, and utilizes an electro-optical effect caused by a force that distorts the position, orientation, or shape of molecules constituting the material. In general, crystals of nonlinear optical materials (organic polymers, lithium niobate, compound semiconductors, etc.) can be modulated by voltage.

Polarization is described by a Jones vector, and when light passes through an optical element, the polarization of the emitted light is the product of the Jones matrix of the optical element and the Jones vector of the incident light. Therefore, assuming that the light observed by the detector 1 uses the Jones vector and the polarization of the reflected light is Ein, the polarization correlation Eout incident on the observed detector is expressed as follows.

FIG. 4 is a time change diagram of each part, and is a graph showing the relationship between the applied voltage applied to the electro-optical element 3 and the time t of the correlation region observed by the detector 1. From the above figure, the applied voltage applied to the electro-optical element 3, the polarized state before the reflection of the light from the light source 5, the polarized state after the reflection, and the correlation region observed by the detector 1 are shown.

As shown in FIG. 4, an applied voltage is applied to the electro-optical element 3 so that the polarization states are in the horizontal direction and the vertical direction, and the polarization state of the light from the light source 5 is temporally modulated by the active polarization optical system 10. Project onto the target object 6. In this case, the polarization state after reflection is such that the active polarization optical system 10 has a time delay of only 2d / C (C is the speed of light, which is 300,000 km / sec), where d is the distance to the target object 6. Incident.

Therefore, the correlation region takes a binary state of 0 and 1 on the time axis. In general time-correlation imaging such as a TOF camera, the light source 5 is made to emit a pulse, and the logical product of the ON state of the pulse wave and the time window of the image sensor is output as a time correlation. The active polarization optical system 10 may apply an applied voltage or an applied magnetic force so that the polarized state has two states, one in an arbitrary direction and the other in a direction orthogonal to the polarized state.

On the other hand, in the case of time polarization correlation imaging according to one embodiment, the correlation value in the polarization direction is output. Therefore, the correlation information is output as an exclusive OR when the polarization states are orthogonal binary values. This doubles the exposure time as compared to time-correlated imaging. In one embodiment, a double SN ratio can be achieved with voltage drive of the same period.

Further, for the time polarization correlation value, the light source is polarized and modulated in the time domain, and the aspect of the change of the object is observed by load integration. That is, since the polarization state of the time polarization correlation value changes in the time domain, the similarity between the reference signal (light source) and the target signal (the signal reflected on the target object and observed on the camera) is obtained.

That is, time polarization correlation imaging is performed by calculating the polarization correlation in the time domain between the light irradiation signal whose reference polarization state is known and the time change of the degree of polarization while the shutter is open. Then, by calculating the time polarization correlation value, the delay time, the phase shift, etc., it is possible to measure the distance, the degree of light scattering of the material, and the three-dimensional shape measurement. For example, when light is scattered inside an object, the material is estimated based on the correlation information between the irradiated light signal and the received light signal.

In imaging, the detector 1 serves as an image sensor and records the light that reaches each pixel. Usually, each pixel has a plurality of photoelectric conversion units (for example, a photodiode or a phototransistor, a detector 1 in FIG. 2) that generates a signal (photocurrent) depending on the light intensity of each pixel. The plurality of pixels are arranged in an array and may be a one-dimensional or two-dimensional array, and the array of pixels may or may not have a rectangular boundary.

For imaging of light obtained in the correlation region observed by the detector 1, for example, the detector 1 includes a photoelectric conversion unit and a plurality of charge storage units, and is based on the reflected light from the target object 6 obtained in the correlation region. An image sensor is used in which the photocurrent generated in the photoelectric conversion unit is distributed and stored in a plurality of charge storage units. That is, the detector 1 uses an image sensor (image sensor) that is arranged between the photodiode and the charge storage unit and has a gate structure that controls the transfer of the photocurrent to the plurality of charge storage units. For example, a CMOS image sensor that has an amplifier for each pixel, converts it into a voltage value, and selects and reads out one pixel at a time.

In addition, the object can be photographed three-dimensionally by combining the data of the distance image with the normal two-dimensional photograph. Furthermore, motion detection is possible by measuring the distance at short time intervals, and motion can be estimated.

Further, in the above-mentioned time polarization correlation imaging method, the active polarization optical system uses a light source having a known polarization state or a light source to which a polarizing plate 2 is applied in order to manipulate the polarization state of light, and has a Kerr effect or a Pockels effect. By combining an electro-optical element 3 that delays the phase of light passing in a specific direction according to an applied voltage and an arbitrary wave plate, it is possible to make a system that arbitrarily modulates the polarization state as a function of time and projects or receives light. it can.

Further, in the above-mentioned time polarization correlation imaging method, the active polarization optical system 10 uses a light source having a known polarization state or a light source to which a polarizing plate 2 is applied in order to manipulate the polarization state of light, and has a Faraday effect or magnetism. It is also possible to make a system in which the polarization state is arbitrarily modulated as a function of time and projected or received by a magnetic optical element that manipulates the polarization state of the light transmitted or reflected according to the applied magnetic force such as the car effect.

Further, in the above-mentioned time polarization correlation imaging method, the active polarization optical system 10 arbitrarily modulates and projects the polarization state as a function of time by a liquid crystal element that directly manipulates the polarization state in order to manipulate the polarization state of light. Alternatively, it may be a system that receives light.

Further, in the above-mentioned time polarization correlation imaging system, the detector 1 includes an imaging element in which photoelectric conversion units are arranged in an array, and obtains polarization correlation information having different times in synchronization with an applied voltage.

In the above-mentioned time polarization correlation imaging system, the detector 1 is an image sensor in which the photoelectric conversion units are arranged in an array, and the change in brightness observed by each photoelectric conversion unit such as the Dynamic Vision Sensor is captured as an event and the information thereof. Includes those that obtain polarization correlation information using an asynchronous detector that outputs.

In the above-mentioned time polarization correlation imaging system, the detector 1 is an imaging element in which photoelectric conversion units are arranged in an array, and uses a polarizing camera in which linear polarization elements in the same direction or different directions are arranged on each detector. Including.

Further, in the above-mentioned time polarization correlation imaging system, it is desirable that the polarization correlation information is the difference in the polarization direction of linearly polarized light.

An example in which the correlation value is estimated by time polarization correlation imaging will be described below.

FIG. 5 is a diagram obtained by simulating the correlation value of the time polarization correlation when the frequency and the distance are changed, and FIG. 6 is a diagram showing the correlation value of the time correlation in the same manner. In the case of the time polarization correlation, the exposure time of the correlation value is twice as long as that of the time correlation as described in FIG. Therefore, in the time polarization correlation imaging, it is possible to observe twice the SN ratio by voltage driving with the same period, and it is possible to improve the estimation accuracy.

1 ... Detector 2 ... Polarizing plate 3 ... Electro-optical element 4 ... 1 / 4λ Wave plate 5 ... Light source 6 ... Target object 10 ... Active polarization optical system

Claims (12)

This is a time-polarization correlation imaging method in which light from a light source is projected onto a target object and imaging is performed based on the polarization correlation information obtained from the reflected light.
The polarization state of the light from the light source is time-modulated by the active polarization optical system and projected onto the target object.
A time-polarization correlation imaging method characterized in that the reflected light on the target object is passed through an active polarization optical system, detected by a detector, and polarization correlation information is obtained from the detection result. In the active polarization optical system, light from the light source is linearly polarized by a polarizing plate, passed through an electro-optical element whose phase is delayed according to an applied voltage, passed over a 1 / 4λ wave plate, and projected onto the target object. The time polarization correlation imaging method according to claim 1, wherein the method is performed. The active polarization optical system according to claim 1, wherein the light from the light source passes or is reflected by a magneto-optical element capable of manipulating the linear polarization direction according to an applied magnetic force and projected onto the target object. Time polarization correlation imaging method. The time polarization correlation imaging method according to claim 2 or 3, wherein an applied voltage or an applied magnetic force is applied so that the polarized state becomes two states in an arbitrary direction and a direction orthogonal to the polarized state by the active polarizing optical system. .. In a time polarization correlation imaging system that projects light from a light source onto a target object and images based on the polarization correlation information obtained from the reflected light.
An active polarized optical system that temporally modulates the polarization state of light from the light source,
A detector that detects light reflected by the target object after passing it through the active polarizing optical system.
With
A time-polarization correlation imaging system characterized in that light from the light source modulated by the active polarization optical system is projected onto the target object and polarization correlation information is obtained from the detection result by the detector. The active polarization optical system is
A polarizing plate that controls the polarization state of light from the light source, a wave plate that controls the phase of light, an electro-optical element or a magnetic optical element that can actively control the polarization state according to an applied voltage or an applied magnetic force.
The time polarization correlation imaging system according to claim 5, wherein light is projected onto the target object in which the polarization state is controlled as a function of an arbitrary time. The time polarization correlation imaging system according to claim 6, wherein an applied voltage is applied to the electro-optical element or the magneto-optical element so that the polarization state is in two states of an arbitrary direction and a direction orthogonal to the arbitrary direction. .. The time polarization correlation imaging according to claim 7, wherein the detector is an imaging element in which photoelectric conversion units are arranged in an array, and obtains polarization correlation information having different times in synchronization with the applied voltage. system. The detector is an imaging element in which the photoelectric conversion units are arranged in an array, and the polarization correlation is performed by using an asynchronous detector that captures the change in brightness observed by each photoelectric conversion unit as an event and outputs the information. The time polarization correlation imaging system according to claim 7, wherein information is obtained. The imaging element includes a photoelectric conversion unit to which a polarizer having different directions of linearly polarized light is applied and a corresponding charge storage unit, and a photocurrent generated in the photoelectric conversion unit having different polarization directions due to reflected light from the target object. The time-polarized light correlation imaging system according to claim 8 or 9, wherein the light is stored in the charge storage unit. The image sensor is
The photodiode, which is the photoelectric conversion unit, and
A gate structure that is arranged between the photodiode and the charge storage unit and controls the transfer of the photocurrent to the plurality of charge storage units.
The time polarization correlation imaging system according to claim 10, further comprising a CMOS image sensor. The time polarization correlation imaging system according to any one of claims 5 to 11, wherein the polarization correlation information is a phase difference.