JP2016053566A - Measuring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a measuring system which can be used for high accurate measurement by suppressing the influence of disturbance light.SOLUTION: The measuring system includes a first light source 101 which generates first light 9001 in which at least one of intensity, a polarization state, and a wavelength is modulated with a first period and emits the first light toward an object 1001, a second light source 102 which generates second light 9002 in which at least one of the intensity, the polarization state, and the wavelength is modulated with a second period and emits the second light, an optical system 109 which mixes the light from the object, based on the first light and the second light, a nonlinear optical crystal 103 which generates third light 9003 having the sum frequency of the frequency of the light from the object, based on the first light and the frequency of the second light from the mixed light by a sum frequency generation phenomenon, and a photodetector 108 which measures the intensity of the third light.SELECTED DRAWING: Figure 1

Description

  The present application relates to a measurement system.

  Technology for visualizing or detecting pedestrians and obstacles (hereinafter sometimes referred to as “subjects”) is important for enhancing the safety of automobiles. As a typical example, an imaging system using a range gate camera is known. The range gate camera is a camera configured to expose (open the gate) for a specific very short time. By combining the range gate camera and pulsed illumination light, it is possible to image only a subject at a predetermined distance. According to the range gate camera, it is possible to eliminate the influence of fog or the like existing in front of the subject. The range gate camera also functions as a Time Of Flight (TOF) imaging system by sequentially changing the timing of opening the gate. A range gate camera is disclosed in Non-Patent Document 1, for example.

Mitsubishi Heavy Industries Technical Report Vol. 42 no. 5P. 212 (2005-12)

  In the conventional technique described above, reflected light from an object by illumination light is detected. When detecting reflected light, disturbance light such as a light source other than illumination light or sunlight affects detection of the reflected light.

  One non-limiting exemplary embodiment of the present application provides a measurement system that suppresses the influence of ambient light and can be used for highly accurate measurements.

  In order to solve the above problem, a measurement system of one embodiment of the present disclosure generates first light in which at least one of intensity, polarization state, and wavelength is modulated in a first period, and the first light is generated. A first light source that emits light toward a subject, and a second light source that generates second light in which at least one of intensity, polarization state, and wavelength is modulated in a second period, and emits the second light And an optical system that mixes the light from the subject based on the first light and the second light, and the mixed light causes the sum frequency generation phenomenon to generate the light based on the first light. A nonlinear optical crystal that generates a third light having a frequency that is the sum of the frequency of light from the subject and the frequency of the second light; and a photodetector that measures the intensity of the third light. .

  Note that comprehensive or specific aspects may be realized by elements, devices, systems, integrated circuits, and methods. In addition, comprehensive or specific aspects may be realized by any combination of elements, devices, systems, integrated circuits, and methods.

  Additional effects and advantages of the disclosed embodiments will become apparent from the specification and drawings. The effects and / or advantages are individually provided by the various embodiments and features disclosed in the specification and drawings, and not all are required to obtain one or more of these.

  According to one aspect of the present disclosure, the influence of ambient light is suppressed, and highly accurate distance measurement is possible.

1 is a schematic diagram of a TOF imaging system 10A according to an exemplary embodiment 1. FIG. It is a figure which shows the spectrum of sunlight. 3 is a flowchart illustrating an operation of a TOF imaging system 10A according to an exemplary embodiment 1. FIG. 6 is a schematic diagram of a TOF imaging system 10B according to an exemplary embodiment 2. FIG. 11 is a diagram showing the intensity of first light 9001. It is a figure which shows the intensity | strength of the reflected light 9111. FIG. It is a figure which shows the intensity | strength of the reflected light 9211. FIG. It is a figure which shows the intensity | strength of the reflected light 9111. FIG. It is a figure which shows the intensity | strength of the reflected light 9211. FIG. It is a figure which shows the intensity | strength of the 2nd light 9002 when it is a 1st phase. FIG. 11 is a diagram showing the intensity of third light 9003. It is a figure which shows the intensity | strength of the reflected light 9111. FIG. It is a figure which shows the intensity | strength of the reflected light 9211. FIG. It is a figure which shows the intensity | strength of the 2nd light 9002 when it is a 2nd phase different from a 1st phase. FIG. 11 is a diagram showing the intensity of third light 9003. 6 is a flowchart illustrating an operation of a TOF imaging system 10B according to an exemplary embodiment 2. FIG. 10 is a schematic diagram of an imaging system 10C according to an exemplary embodiment 3. FIG. 11 is a diagram showing the light intensity of first light 9001. It is a figure which shows the light intensity of the reflected light 9311 from a to-be-photographed object. It is a figure which shows the light intensity of the reflected light 9411 from fog. It is a figure which shows the light intensity of the reflected light 9311. FIG. It is a figure which shows the light intensity of the reflected light 9411. FIG. When the phase of the second light 9002 is the first phase, it is a diagram showing a polarization component (intensity) involved in phase matching in the second light 9002. It is a diagram showing the intensity of third light 9303 generated from reflected light 9311 and second light 9002. It is a diagram showing the intensity of third light 9403 generated from reflected light 9411 and second light 9002. It is a figure which shows the light intensity of the reflected light 9311. FIG. It is a figure which shows the light intensity of the reflected light 9411. FIG. When the phase of the second light 9002 is a second phase different from the first phase, it is a diagram showing a polarization component (intensity) involved in phase matching in the second light 9002. It is a diagram showing the intensity of third light 9303 generated from reflected light 9311 and second light 9002. It is a diagram showing the intensity of third light 9403 generated from reflected light 9411 and second light 9002. It is a figure which shows typically the mode of the multiple scattering of the light by the fog 1041. FIG. 12 is a flowchart illustrating an operation of an imaging system 10C according to an exemplary embodiment 3. FIG. 10 is a schematic diagram of an imaging system 10D according to an exemplary embodiment 4. FIG. 11 is a diagram showing the intensity of first light 9001. It is a figure which shows the intensity | strength of the reflected light 9111. FIG. It is a figure which shows the intensity | strength of the reflected light 9211. FIG. It is a figure which shows the time change of the reflected light 9111 from the to-be-photographed object 1011 near. It is a figure which shows the time change of the reflected light 9211 from the to-be-photographed object 1021. FIG. It is a figure showing the time change of the 2nd light 9002. It is a figure which shows the time change of the 3rd light 9003. FIG.

  In addition to the above-described techniques for visualizing or detecting pedestrians and obstacles, for example, the following techniques are known.

  The first technique is a camera system using a normal image sensor. In this system, an object illuminated by sunlight, road lights, headlights, or the like is imaged by a camera. The camera includes, for example, an optical system including a lens and an image sensor such as a CCD (Charge Coupled Device). The image sensor itself does not have a function of detecting pedestrians and obstacles or a function of determining the distance to them. Pedestrians and obstacles are detected by subjecting the image acquired by the image sensor to image recognition processing. In some cases, estimate the distance to them. Since the distance measuring function is not required for the image sensor itself, an image sensor having a simple structure and a relatively low speed can be used.

  The second technique is a technique called Light Detection and Ranging (LIDAR). LIDAR is a technique for measuring the distance to a subject. In this technique, the time required for the illumination light to be reflected by the subject and the reflected light to reach the detector, that is, the flight time is measured. In order to measure the time of flight of the light, intensity modulation is applied to the illumination light. The detector has a function of detecting a phase difference between the intensity modulation phase of the illumination light and the intensity modulation phase of the reflected light. Since this phase difference corresponds to the time of flight, distance measurement can be performed from this phase difference. However, since only an object existing in the beam irradiation direction can be detected, the spatial resolution is not sufficient. Therefore, in order to ensure the spatial resolution, a scanning mechanism for changing the beam irradiation direction is added.

  The third technique is a TOF imaging system using a TOF image sensor. Each pixel of the TOF image sensor has a function of measuring the intensity phase difference. Each pixel functions as a rider. Each pixel has a distance measuring function and also has an imaging function like a normal camera system. A pedestrian and an obstacle can be imaged by combining the TOF image sensor and an illumination device that illuminates light with intensity modulation over a wide range. At the same time, the distance to them can be measured.

  Hereinafter, problems of the prior art considered by the present inventors will be described.

  The first technique described above can affect imaging during bad weather. For example, if fog or raindrops exist between the subject and the camera, light scattering by them affects the imaging. Therefore, it becomes difficult to clearly image the subject.

  In both the second technique and the third technique described above, the subject is irradiated with intensity-modulated illumination light, and distance measurement is performed by detecting reflected light from the subject.

  However, the subject is not usually illuminated only by intensity-modulated illumination light prepared for ranging. For example, the light is also irradiated by unmodulated light (also referred to as disturbance light) such as the sun, a streetlight, and a car headlight. In order to perform ranging with high accuracy, it is required to remove disturbance light components and extract only modulation components. In principle, it is possible to remove the disturbance light component. For example, only the disturbance light component may be measured, and the difference between the modulation component and the disturbance light component may be taken. However, in reality, when disturbance light is extremely strong such as during the daytime, sensitivity saturation due to the disturbance light occurs, making it difficult to extract the modulation component.

  FIG. 2 shows the spectrum of sunlight. The intensity of sunlight becomes weaker as the wavelength becomes longer. Therefore, in the near infrared region, disturbance light caused by sunlight is less than in the visible region. In particular, sunlight having a wavelength of around 1400 nm and a wavelength of around 1900 nm are absorbed by the atmosphere, and thus hardly reach the ground surface. Therefore, if ranging is performed using light of these wavelengths, the influence of disturbance light can be suppressed. As a result, improvement in ranging accuracy can be expected. The visible range means a wavelength band of approximately 400 nm to 700 nm. The near-infrared region means a wavelength band of approximately 700 nm to 2500 nm.

  However, in order to perform distance measurement using the second and third techniques, the image sensor is required to operate at high speed. For example, the phase difference caused by the optical path difference of 1 m is only 3.3 nanoseconds in terms of time. Therefore, an operation speed of about several hundred MHz is required for the detector or the image sensor. With the second technique, a detector using a compound semiconductor can be used. However, in the third technique, an image sensor using silicon (hereinafter referred to as “silicon image sensor”) must be used as the TOF image sensor. This is because the TOF image sensor requires high-speed operation and complicated arithmetic processing.

  However, the silicon image sensor has a small absorption coefficient on the infrared side. Therefore, the silicon image sensor cannot measure with high sensitivity in the near infrared region. Further, the silicon image sensor does not have sensitivity in a wavelength region where sunlight is lost (hereinafter referred to as “sunlight-depleted wavelength region”). For this reason, the silicon image sensor cannot perform any imaging in the sunlight-missing wavelength region.

  In addition, the above-described range gate camera requires a technique for opening the gate for a limited time. Since a distance resolution of about 1 m is required for in-vehicle applications, it is necessary to limit the time for opening the gate to about several nanoseconds. However, there is no physical shutter that operates at such a high speed. Therefore, it is necessary to support high-speed operation using an electronic shutter of an image sensor that operates at high speed. However, similar to the above-described problem of the third technique, the only image sensor that operates at high speed is a silicon image sensor. Therefore, the operable wavelength range is limited. In view of such problems, the inventor of the present application has come up with a TOF imaging system having a novel structure.

  According to the imaging system according to an aspect of the present disclosure, it is possible to suppress the influence of disturbance light and perform highly accurate distance measurement. In addition, a TOF image sensor that functions even in the near-infrared region with high safety can be provided.

  Hereinafter, more specific embodiments of the present disclosure will be described with reference to the drawings. In the following description, the same reference numerals are assigned to the same or similar components. In addition, overlapping description may be omitted. Note that the imaging system according to the embodiment of the present disclosure is not limited to the one exemplified below.

(Embodiment 1)
Hereinafter, the structure and function of the TOF imaging system 10A will be described with reference to FIGS.

  FIG. 1 schematically shows a configuration of a TOF imaging system 10A according to the present embodiment. Note that this configuration diagram shows only elements necessary for the description of the present disclosure. Moreover, this block diagram is a figure which shows a concept to the last. The actual shape, aspect ratio, scale, etc. of each element are not considered at all.

  The TOF imaging system 10A includes a first light source 101, a second light source 102, a nonlinear optical crystal 103, a first optical system 104, a second optical system 105, a third optical system 106, and a fourth optical system 107. , An image sensor 108, a light mixing element 109, a processor 110, and a controller 111. A light source unit is composed of the first light source 101, the second light source 102, and the like. A controller 111 may be provided in the light source unit. The TOF imaging system 10A functions as a measurement system. The processor 110 processes an output signal from the TOF imaging system 10A. The TOF imaging system 10A may not include the processor 110. In that case, the processor 110 may be incorporated in the controller 111.

  The first light source 101 generates and emits modulated first light 9001. More specifically, the first light source 101 can generate the first light 9001 in which at least one of the intensity, the polarization state, and the wavelength changes periodically on the time axis. The same applies to the second light source 102 described below. The phrase “at least one of the intensity, the polarization state, and the wavelength changes periodically on the time axis” means that at least one of the intensity, the polarization state, and the wavelength changes periodically as time passes. That is, light modulation. The phase of modulation means a position in one cycle at a certain time point in a periodic change. The phase of modulation of the first light 9001 can be set to a predetermined value under the control of the controller 111 described later. In this specification, “the phase of modulation of the first light 9001” may be referred to as “the phase of the first light source 101”.

  In the following description, the first light source 101 generates illumination light having a predetermined wavelength (first light 9001). The illumination light is laser light, for example, and may be pulsed light. The first light source 101 irradiates the subject 1001 with the first light 9001. The first light source 101 may be the same as the light source used in an imaging system using a normal TOF image sensor, for example. That is, the first light source 101 outputs the first light 9001 whose intensity is modulated. In other words, the first light source 101 generates the first light 9001 whose intensity periodically changes on the time axis.

  As the first light source 101, for example, a light source that spontaneously outputs light whose intensity periodically changes on the time axis, such as a mode-locked laser, can be used. Alternatively, the first light source 101 may be realized by combining a light source that can modulate the light intensity with the amount of current to be injected, such as a diode laser, and a mechanism that controls the amount of current. Alternatively, the first light source 101 can be realized by combining a light source that outputs light with a certain intensity and an element that modulates transmitted light, such as an electro-optic modulator or an acousto-optic element. Good. The electro-optic modulator or the acousto-optic element can be disposed between the first optical system 104 and the light source that outputs light with a certain intensity.

  The wavelength of the first light 9001 is set within a range where a sum frequency generation (SFG) phenomenon occurs between the first light 9001 and the second light 9002 by the nonlinear optical crystal 103 described later. If it is this range, arbitrary wavelengths can be selected. This range can be determined by, for example, the type of the nonlinear optical crystal 103, the temperature, and the wavelength of the second light 9002. By appropriately combining these conditions, the wavelength of the first light 9001 can be selected from a wide range from ultraviolet to far infrared.

  Thus, the present disclosure can be implemented in a wide wavelength range. However, since there are advantageous wavelengths for implementing the present disclosure from several viewpoints, the viewpoints and wavelengths will be described. In the implementation of the present disclosure, the wavelength of the first light 9001 can be in the range from ultraviolet to far infrared, and is not limited to the wavelength range shown below.

  The first viewpoint is the sunlight spectrum. Also in the implementation of the present disclosure, as in a normal TOF image sensor system, if light in a wavelength region with little disturbance light is used as illumination light, the accuracy of distance measurement can be improved. That is, it is desirable to select near infrared rather than visible as the wavelength range of the first light 9001. In particular, it is desirable to select the sunlight missing wavelength region as the wavelength region of the first light 9001. As shown in FIG. As will be described later, in the present disclosure, the wavelengths of illumination light of the first and second light sources are different from the wavelength of light reaching the image sensor 108. Therefore, it is possible to select the wavelength of illumination light without worrying about the sensitivity range of the image sensor 108.

  A second aspect is available light intensity. In order to measure the distance to the subject, the subject is irradiated with the first light 9001. The stronger the irradiation intensity, the easier it is to distinguish from ambient light, and the accuracy of distance measurement increases.

  However, the subject includes a pedestrian and a driver of another car. For this reason, the upper limit of the available light intensity is determined in consideration of the influence of the illumination light on the person. For example, if the oncoming vehicle is irradiated with strong visible light, it may hinder driving. The upper limit of allowable light intensity varies depending on the wavelength. The human eye has no sensitivity to the infrared region. Therefore, there is little fear of being dazzled even when strong light is irradiated. In the near-infrared wavelength region of 1.4 micrometers or more, the possibility that light is absorbed by the crystalline lens and forms an image on the retina is low. This wavelength region is called an eye-safe wavelength, and safety is further enhanced by using light in this wavelength region. This is an important viewpoint when the first light 9001 is generated using a laser.

  The first optical system 104 irradiates light to the range imaged by the TOF imaging system 10A. The first light 9001 is applied to the subject 1001 to be measured via the first optical system 104. The first optical system 104 can be configured using optical elements such as a lens, an optical fiber, and a mirror.

  Note that the first light source 101 may have the functions of both the light source and the optical system described above. That is, the first light source 101 may generate the first light 9001 to which intensity modulation has been applied, and emit the first light 9001 directly toward the subject.

  In the present disclosure, the light incident on the nonlinear optical crystal 103 via the light mixing element 109 may be various light generated from the subject 1001 based on the first light 9001. Specifically, the light may be reflected light of the subject 1001, transmitted light, or fluorescent light. That is, the light 9011 is not limited to the reflected light of the subject 1001. However, in this specification, the light 9011 is described as reflected light from the subject 1001.

  The second optical system 105 causes the reflected light 9011 to enter the nonlinear optical crystal 103 via the light mixing element 109. The reflected light 9011 is reflected light of the first light 9001 reflected by the subject 1001. The second optical system 105 plays a role of forming an image of a subject on the image sensor 108 together with a fourth optical system 107 described later. Similarly to the first optical system 104, the second optical system 105 can also be configured using optical elements such as lenses, optical fibers, and mirrors.

  Further, the second optical system 105 has a function of blocking light having the same wavelength as the third light 9003 existing in the photographing environment so as not to enter the nonlinear optical crystal 103. The third light 9003 is sum frequency light generated by SFG in the nonlinear optical crystal 103. Details of the third light 9003 will be described later. The wavelength of the reflected light 9011 and the wavelength of the third light 9003 are different. Therefore, in order to have such a blocking function, an interference type or absorption type optical filter, or a spectral element such as a spectral prism / diffraction grating may be used. However, when it is known that almost no light having the same wavelength as the third light 9003 exists in the photographing environment, the function can be omitted.

  Further, as will be described later, the second optical system 105 includes a polarizer that allows only a component in a specific polarization direction of the reflected light 9011 to pass when the second light 9002 is polarization-modulated light. It is desirable that In the case where the second light 9002 is intensity-modulated or wavelength-modulated light, a polarizer may not be provided.

  The second light source 102 generates second light 9002 in which at least one of intensity, polarization state, and wavelength changes periodically on the time axis. The second light source 102 emits the second light 9002 that has been modulated in any one of intensity, polarization state, and wavelength while switching the phase of the modulation. The phase of modulation of the second light 9002 can be set to a predetermined value under the control of the controller 111 described later. Similarly to the first light 9001, the “phase of modulation of the second light 9002” may be referred to as “phase of the second light source 102”. As the second light source 102, for example, a mode-locked laser or a diode laser can be used in the same manner as the first light source 101.

  The optical path of the second light 9002 is controlled by the light mixing element 109 and enters the nonlinear optical crystal 103. When the reflected light 9011 and the second light 9002 enter the nonlinear optical crystal 103 at the same time, an SFG phenomenon occurs and third light 9003 is generated. The wavelength of the second light 9002 is selected so as to satisfy a so-called phase matching condition so that the SFG phenomenon occurs efficiently. The wavelength of the second light 9002 is selected so as to be different from the wavelength of the first light 9001. The phase matching condition is determined from the type of the selected nonlinear optical crystal 103, the crystal orientation, the crystal temperature, and the wavelength of the first light. The phase matching condition will be described later.

  The second light source 102 generates and emits modulated second light 9002. As the second light source 102, a light source capable of changing a modulation phase or a modulation waveform can be selected. As described above, the second light 9002 may be intensity-modulated light. Alternatively, the second light 9002 may be light whose polarization state is modulated or light whose wavelength is modulated as long as it affects a phase matching condition described later. In other words, the modulation of the second light 9002 may be any one of intensity, polarization state, and wavelength.

  As a method for intensity modulation, the modulation method described for the first light source 101 can be used.

  As a method of modulating the polarization state, for example, a method of changing the polarization plane of linearly polarized light using an electro-optic element can be mentioned. Since this method operates at high speed, it is suitable for implementation of the present disclosure.

  Linearly polarized light can be easily obtained by using, for example, a polarized laser. Alternatively, a non-polarized light may be converted into a linearly polarized light by using a polarizer (for example, a linear polarizer) that passes only light that vibrates only in a specific direction.

  As a method for modulating the polarization state, for example, a method using an electro-optic element made of lithium niobate is suitable because it operates at high speed. Some electro-optic elements can change the angle of the plane of polarization of the transmitted light in accordance with the applied electrical signal. By using such an electro-optic element, the second light 9002 whose polarization state is modulated can be obtained. Note that in the case where low-speed modulation is allowed, a second light 9002 whose polarization state is modulated is obtained by applying a polarizer capable of adjusting the angle of the polarization plane to non-polarized light. be able to.

  Examples of the wavelength modulation method include a method using a wavelength tunable laser. The wavelength tunable laser can change the modulation phase or the modulation waveform, for example, by changing the resonance wavelength of the resonator. Further, the wavelength may be modulated by an electro-optical element. In this case, the modulation phase or the modulation waveform can be changed by changing the electric signal applied to the electro-optical element.

  It is difficult to use the method described above for a light source that is originally modulated, such as a mode-locked laser. However, even in this case, the phase of modulation can be changed by changing the optical path length using a mechanism that can change the optical path length, that is, a so-called optical delay line. As described above, the second light source 102 can change the phase of modulation of the second light 9002. In addition, as described above, in the first light source 101, it is also within the scope of the present disclosure to make the modulation phase of the first light 9001 variable.

  The third optical system 106 causes the second light 9002 emitted from the second light source 102 to enter the nonlinear optical crystal 103 via the light mixing element 109. Similarly to the first and second optical systems 104 and 105, the third optical system 106 is also configured using optical components such as a lens and a curved mirror.

  The reflected light 9011 and the second light 9002 enter the nonlinear optical crystal 103 so as to overlap. By using the light mixing element 109, both optical paths can be efficiently overlapped. The light mixing element 109 reflects one light and allows the other light to pass therethrough. It is desirable to use a so-called dichroic mirror as the light mixing element 109. Alternatively, the difference in polarization between the reflected light 9011 and the second light 9002 may be used to overlap the optical axes of both using a polarizing beam splitter. Alternatively, both optical axes can be overlapped using a half mirror. However, when a half mirror is used, it is considered that the efficiency is low because light that cannot be used when the optical axes of the two mirrors are overlapped.

The nonlinear optical crystal 103 generates light having a wavelength different from that of the incident light. Crystals having such properties are, for example, barium borate (BBO), lithium triborate (LBO), KTP, and lithium niobate. The SFG phenomenon is a phenomenon in which when light of two wavelengths enters the nonlinear optical crystal 103 simultaneously, light having a frequency that is the sum of the two lights is generated. The SFG phenomenon occurs with sufficient efficiency only when the phase matching condition is satisfied. The phase matching condition is given by Equation 1 below.
n3 / λ3 = n1 / λ1 + n2 / λ2 (Formula 1)

  Here, λ1 is the wavelength of the light 9011 generated from the subject based on the first light 9001. Since the wavelength of light does not change in the case of reflection and transmission, the wavelength of the light 9011 is the same as the wavelength of the first light 9001. In the case of fluorescence or the like, the wavelength of the light 9011 can be different from that of the first light 9001. In this case, the wavelength λ1 is the wavelength of fluorescence. In the following description, the case where the light 9011 is reflected light will be described. However, the present disclosure can be similarly implemented when the light 9011 is fluorescent. λ2 is the wavelength of the second light 9002. λ3 is the wavelength of the third light 9003. n1 is the refractive index of the nonlinear optical crystal 103 in the wavelength and polarization state of the light 9011. n2 is the refractive index of the nonlinear optical crystal 103 in the wavelength and polarization state of the second light 9002. n3 is the refractive index of the nonlinear optical crystal 103 at the wavelength and polarization of the third light 9003. n1, n2, and n3 are determined by the type, crystal orientation, and temperature of the nonlinear optical crystal 103. The wavelength of the first light 9001, the wavelength of the second light 9002, and the type, orientation, and temperature of the nonlinear optical crystal 103 are selected so that the phase matching condition is satisfied.

  When the SFG phenomenon occurs in the phase matching condition, the intensity of the generated light is proportional to each intensity of the light of the original two wavelengths. That is, when the third light 9003 is generated from the reflected light 9011 of the first light 9001 and the second light 9002 by the SFG phenomenon, the intensity of the third light 9003 is the intensity of the reflected wave 9011 of the first light 9001. It is proportional to the intensity of the component that satisfies the phase matching and the intensity of the component that satisfies the phase matching of the second light 9002.

  This means that the nonlinear optical crystal 103 has a function of converting the reflected wave 9011 of the first light 9001 into the third light 9003. The conversion efficiency is variable depending on the intensity of the component that satisfies the phase matching of the second light 9002.

  Note that the SFG phenomenon can occur only when the phase matching condition is satisfied. Therefore, even if the light intensity of the second light 9002 is constant, when the wavelength or polarization state is modulated, the SFG phenomenon is only performed while the phase matching condition is satisfied with respect to the change in wavelength or polarization state. Can happen. In addition, while the phase matching condition is satisfied, the third light 9003 having an intensity proportional to the intensity of the light component that satisfies the phase matching condition is generated.

In the SFG phenomenon, the wavelength of the reflected wave 9011, the wavelength of the second light 9002, and the wavelength of the third light 9003 that are the same as the wavelength of the first light 9001 satisfy the relationship of Expression 2 below.
1 / λ3 = 1 / λ1 + 1 / λ2 (Formula 2)

  As apparent from Equation 2, the wavelength of the third light 9003 is always shorter than the wavelengths of the first and second lights 9001 and 9002. For example, when the wavelength of the first light 9001 is 1450 nm and the wavelength of the second light 9002 is 1064 nm, the wavelength of the third light 9003 is 614 nm. Thus, even if the wavelength of the 1st and 2nd light is a wavelength of a near infrared region, the 3rd light 9003 becomes a wavelength of a visible region. Therefore, by converting the wavelength by the nonlinear optical crystal 103, the subject can be irradiated with near-infrared light and the subject can be imaged with visible light. Thereby, an image sensor having sensitivity only in the visible range can be used.

  As long as the phase matching condition of the nonlinear optical crystal 103 is satisfied, two of the wavelength of the first light 9001, the wavelength of the second light 9002, and the wavelength of the third light 9003 can be arbitrarily selected. For example, the wavelength of light to be used can be freely selected based on the intensity of disturbance light, the sensitivity of the image sensor, the light emission efficiency of the light source, the ease of modulation, and the like.

  The fourth optical system 107 has a function of forming an image of the third light 9003 on the image sensor 108. The fourth optical system 107 can be configured using an optical component such as a lens. Note that the image sensor 108 is sensitive to the wavelength of the first light 9001 or the second light 9002. In that case, a spectral element such as an optical filter or a spectral prism / diffraction grating is used to prevent the first light 9001 or the second light 9002 from reaching the light receiving element.

  As the image sensor 108, for example, an organic image sensor and a silicon image sensor can be used. The image sensor 108 receives the third light 9003 and images the subject. The image sensor 108 only needs to have sensitivity to at least the wavelength of the third light 9003. The image sensor 108 has no sensitivity to the wavelengths of the first light 9001 and the second light 9002 that are illumination light (substantially smaller than 1 / 100th of the sensitivity to the wavelength of the third light 9003). It is desirable to include a range having no sensitivity.

  The sensitivity to the wavelengths of the first light 9001 and the second light 9002 may not be strictly zero. The image sensor 108 desirably has higher sensitivity to the wavelength of the third light 9003 than the wavelengths of the first light 9001 and the second light 9002. Signals based on the first light 9001 and the second light 9002 can affect the imaging signal as noise. Therefore, those signal intensities need only be such that they do not affect imaging and distance measurement.

  The image sensor 108 may not be sensitive to both the first light 9011 and the second light 9002. In that case, the fourth optical system 107 does not need a mechanism for removing the first light 9001 or the second light 9002, and the configuration of the fourth optical system 107 can be simplified.

  The image sensor 108 measures the intensity of the third light 9003. Note that the imaging system 10A may be a non-integrated photodetector instead of the image sensor 108. In that case, the non-integrated optical detector measures the intensity of the third light 9003 as a whole. In the present embodiment, the photodetector is exemplified by an image sensor 108 and a non-integrated photodetector.

  The organic image sensor has a photoelectric conversion layer using, for example, phthalocyanines or P3HT, PCBM, and C60. A photoelectric conversion layer using such a material has sensitivity only up to about 650 nm on the long wavelength side, but has higher photoelectric conversion efficiency in this region than a silicon image sensor. When the wavelength of the first light 9001 is 1450 nm and the wavelength of the second light 9002 is 1064 nm, the organic image sensor has no sensitivity for both. Therefore, a mechanism for removing these lights becomes unnecessary.

  On the other hand, the silicon image sensor has sensitivity up to about 1100 nm although it is weak. Therefore, in the example of the combination of wavelengths described above, when the image sensor 108 receives the third light 9003, the second light 9002 becomes noise. Therefore, the fourth optical system 107 may be provided with a mechanism for removing the second light 9002.

  The processor 110 measures the distance to the subject based on the imaging signal acquired by the image sensor 108. Specifically, the processor 110 measures the distance to the subject based on the intensity of the third light 9003.

  The controller 111 controls the entire TOF imaging system 10A. The controller 111 is realized by, for example, a general-purpose computer (for example, a personal computer) or a system dedicated central processing unit (CPU).

  The controller 111 can control the modulation phases of the first light 9001 and the second light 9002, respectively. In other words, the controller 111 can set the phases of the first light source 101 and the second light source 102, respectively. More specifically, the controller 111 can set at least two different phases in the first light source 101 as the modulation phases of the first light 9001. Further, the controller 111 can set at least two phases different from each other as the phase of modulation of the second light 9002 in the second light source 102.

  Thus, the controller 111 can set at least two phases different from each other for each light source. The controller 111 may set a plurality of phases so that the phase of each light source changes continuously. It is within the scope of this disclosure to continuously switch the phase of light modulation. In a specific application, the phase of light modulation may not be changed and the phase may be fixed in both light sources. The configuration will be described later in detail. The controller 111 sets at least two different modulation phases for only one of the first light 9001 and the second light 9002, and fixes the modulation phase for the other. May be.

  The measuring device can be composed of the fourth optical system 107, the image sensor 108, and the like. The measurement apparatus determines a relative phase relationship between the phase of modulation of the reflected light 9011 and the phase of modulation of the second light 9002. Alternatively, the processor 110 may receive the output of the measuring device and determine the relative relationship. The phase relative relationship is obtained according to the distance from the first light source 101 to the subject 1001. The intensity of the third light 9003 can vary depending on the relative relationship. Therefore, the distance to the subject 1001 can be measured based on the intensity of the third light 9003.

  Next, an example of the operation of the TOF imaging system 10A will be described with reference to FIG. Hereinafter, it is assumed that the modulation phase of the first light source 101 is fixed and the modulation phase of the second light source 102 is variable.

  FIG. 3 shows an operation flow of the TOF imaging system 10A. For example, the controller 111 (see FIG. 1) executes the following steps. Specifically, a computer program including an instruction corresponding to each step may be mounted in the memory inside the controller 111, and the controller 111 calls the computer program from the memory and sequentially executes each step. You may make it perform.

  The first light source 101 generates first light 9001 subjected to intensity modulation (step S101A). The first optical system 104 is used to irradiate the subject within the imaging range with the first light 9001 (step S102A). The reflected light 9011 from the subject is collected using the second optical system 105 (step S103A).

  On the other hand, in the second light source 102, the modulation phase is set to the first value (step S110A), and the modulated second light 9002 is generated. The modulation may be any of intensity modulation, polarization modulation, and wavelength modulation. For example, the second light 9002 subjected to polarization modulation is generated (step S104A). The optical path of the second light 9002 is controlled by the third optical system 106 so that the second light 9002 enters the range where SFG occurs in the nonlinear optical crystal 103 (step S105A).

  The reflected light 9011 and the second light 9002 are incident on the nonlinear optical crystal 103 to generate the third light 9003 (step S106A).

  The image sensor 108 is caused to detect an image of the subject by the third light 9003 and output an imaging signal (step S107A).

  The light intensity of the third light 9003 depends on the relative relationship between the phase of intensity modulation of the reflected wave 9011 and the phase of modulation of the second light 9002. On the other hand, the phase of intensity modulation of the reflected wave 9011 depends on the distance from the first light source 101 to the subject 1001. The light intensity of the third light 9003 includes distance information from the first light source 101 to the subject 1001. For this reason, the third light imaging signal acquired by the image sensor 108 includes information on the distance to the subject. This distance information can be extracted by the processor 110.

  However, the light intensity of the third light 9003 also depends on the intensity of the reflected wave 9011, which depends on the reflectance of the subject 1001. The reflectance of the subject 1001 can take various values depending on the material and shape of the subject. For this reason, there is a case where the distance information to the subject 1001 cannot be sufficiently separated by photographing using the modulation phase of the second light 9002 having a single value.

  Further, depending on the intensity modulation waveform of the first light 9011 and the modulation waveform of the second light 9002, a combination of a distance that cannot be measured and a distance that is difficult to distinguish may occur.

  However, the reflectance of the subject 1001 does not depend on the value of the modulation phase of the second light 9002. Therefore, the change in the intensity of the third light 9003 caused by the change in the value of the modulation phase of the second light 9002 is caused by the relative relationship between the phase of the intensity modulation of the reflected wave 9011 and the phase of the modulation of the second light 9002. It is due to change. Therefore, it is possible to separate the distance information by performing imaging while setting the phase of the modulation of the second light 9002 to a plurality of values suitable for extraction of the distance information.

  Further, even when measurement is difficult when the phase of modulation of the second light 9002 is a certain value, there are cases where measurement can be performed by changing the phase of modulation of the second light 9002 to another value.

  For the above reasons, if sufficient distance information cannot be obtained when the modulation phase of the second light 9002 is a certain value, the modulation phase of the second light 9002 is changed to another value and measurement is performed. Just do it. The number of set values of the phase of modulation of the second light 9002 can be determined by the modulation waveform, the type of subject, the accuracy of required distance information, and the like.

  When measuring the modulation phase with multiple setting values, determine the number of required modulation phase setting values in advance and determine whether the number of measured phase setting values has reached the desired value. (Step S108A). If the desired value has been reached, the setting value of the modulation phase is returned to the first value, and the measurement is repeated from step S110A.

  If the desired value has not been reached, the set value of the modulation phase is changed to the next set value (S109), and the measurement is repeated from step S104A.

  The controller 111 repeatedly executes the above steps 101A to 110A until a predetermined condition is satisfied. The acquired imaging signal includes information indicating a change in the light intensity of the third light, which includes distance information to the subject.

  According to the above procedure, the distance information to the subject can be extracted by the processor 110. Further, the processor 110 may perform image analysis such as object recognition based on the acquired imaging signal.

  According to the present embodiment, TOF imaging can be performed using light having a wavelength that the image sensor 108 has no sensitivity as illumination light. In addition, the subject can be detected simultaneously with the distance measurement.

(Embodiment 2)
With reference to FIGS. 4 to 7D, a TOF imaging system 10B in the case of using pulsed light as illumination light will be described.

  FIG. 4 schematically shows the configuration of the TOF imaging system 10B according to this embodiment. The following description will focus on the differences from the first embodiment, and a detailed description of the same contents will be omitted.

  The first light source 101 generates first light 9001 subjected to intensity modulation. The first light 9001 is treated as a pulse train having a narrow pulse width emitted at a constant period.

  As described in Embodiment Mode 1, the wavelength of the first light 9001 can be freely selected. In this embodiment, 1450 nm light is used. This wavelength is a wavelength in the sunlight missing wavelength region, and is an eye-safe wavelength at which image formation on the retina cannot be obtained.

  The object 1001 is irradiated with the first light 9001 using the first optical system 0104. The subject 1001 includes subjects 1021 and 1011 at different distances. The subject 1001 is illuminated by the first light 9001 whose intensity is modulated. At this time, reflection occurs in the subject 1001, and a part of the first light 9001 (reflected light) 9011 travels to the second optical system 105. The reflected light 9011 is incident on the nonlinear optical crystal 103 using the second optical system 105. The reflected light 9011 includes light 9211 and 9111 reflected from the subjects 1021 and 1011, respectively.

  FIG. 5A shows the intensity of the first light 9001. FIG. 5B shows the intensity of the reflected light 9111. FIG. 5C shows the intensity of the reflected light 9211. The time at which the reflected light enters the nonlinear optical crystal 103 differs depending on the distance to the subject. Light 9111 reflected by a nearby subject 1011 enters the nonlinear optical crystal 103 earlier. Light 9211 reflected by the distant subject 1021 is incident slowly by the nonlinear optical crystal 103. That is, the reflected light 9011 has a phase difference corresponding to the distance to the subject.

  The second light source 102 generates a modulated second light 9002. The modulation may be any of intensity modulation, polarization modulation, and wavelength modulation. In this embodiment, modulation of the second light 9002 will be described by taking intensity modulation as an example. The intensity modulation may be pulsed or continuous. Hereinafter, it is assumed that the modulation is pulse modulation, and the period and the pulse width coincide with those of the first light 9001. The period of the modulated second light 9002 is equal to the period of the modulated first light 9001 (including a case where the TOF imaging result is substantially equal to an allowable error range). . Note that, as described in the selection of the wavelength in Embodiment 1, in this embodiment, light of 1064 nm is used as the second light 9002.

  With the modulation phase of the second light 9002 fixed to a certain value, the second light 9002 is incident on the nonlinear optical crystal 103 using the third optical system 106. At this time, the second light 9002 is incident so that the respective reflected lights 9111 and 9211 from the subjects 1011 and 1021 overlap the optical path. Reflected light 9111, reflected light 9211, and second light 9002 enter the nonlinear optical crystal 103. At this time, if the phase of modulation of the reflected light 9111 or the reflected light 9211 and the pulse 9002 of the second light coincide with each other, the two pulses overlap each other. As a result, an SFG phenomenon occurs in the nonlinear optical crystal 103, and third light 9003 is generated.

  On the other hand, if the phase of modulation of the reflected light 9111 or the reflected light 9211 and the pulse 9002 of the second light do not match, the pulses do not overlap. As a result, the SFG phenomenon does not occur in the nonlinear optical crystal 103, and the third light 9003 is not generated.

  FIG. 6A shows the intensity of the reflected light 9111. FIG. 6B shows the intensity of the reflected light 9211. FIG. 6C shows the intensity of the second light 9002 when in the first phase. FIG. 6D shows the intensity of the third light 9003.

  Only the pulse group of the first light 9001 reflected by the subject at a specific distance matches the pulse of the second light 9002 having the first phase, for example. Accordingly, when the fourth optical system 0107 is used to form an image of the third light 9003 on the image sensor 108 and an image is picked up by the image sensor 108, only an image of a subject at a specific distance where the phase of modulation matches is obtained. You can get it. In the illustrated example, the phase of modulation of the reflected light 9111 from the nearby subject 1011 matches the first phase of the second light, and the third light 9003 is generated. Only the image of the near subject 1011 can be acquired, and the image of the distant subject 1021 is not acquired. If the phase of modulation of the second light 9002 is changed, the reflected light from the subject at a different distance and the phase of modulation of the second light 9002 coincide with each other, and an image of the subject at that distance can be acquired.

  FIG. 7A shows the intensity of the reflected light 9111. FIG. 7B shows the intensity of the reflected light 9211. FIG. 7C shows the intensity of the second light 9002 when the second phase is different from the first phase. FIG. 7D shows the intensity of the third light 9003.

  In the illustrated example, the phase of modulation of the reflected light 9211 from the distant subject 1021 matches the second phase, and third light is generated. Only an image of a distant subject 1021 can be acquired, and an image of a nearby subject 1011 is not acquired. As described above, if the phase of modulation of the second light 9002 is changed by the number of distances to be imaged, imaging using the TOF method (TOF imaging) can be realized.

  As described in Embodiment Mode 1, when the modulation of the first light 9001 and the second light 9002 is not pulse modulation, the third light 9003 can always be generated. However, even in that case, the intensity of the third light 9003 changes according to the phase relationship of the modulation between the reflected light 9011 of the first light and the second light 9002. Therefore, TOF imaging can be realized by measuring the intensity change of the third light 9003 when the phase of modulation of the second light 9002 is changed.

  The wavelength of the third light 9003 is given by Equation 2 as described above. If the wavelength of the first light 9001 is 1450 nm and the wavelength of the second light 9002 is 1064 nm, the wavelength of the third light 9003 is 614 nm. When the wavelength is 614 nm, high-sensitivity imaging can be realized using a silicon image sensor or an organic image sensor. In addition, the wavelength 1450 nm of the first light 9001 is a wavelength in the sunlight lacking wavelength region. Therefore, TOF imaging using a high sensitivity sensor can be realized while suppressing the influence of disturbance light.

  As described above, according to the present disclosure, a subject irradiated with light having a certain wavelength can be subjected to TOF imaging using an image sensor having extremely low sensitivity to the wavelength.

  In normal TOF imaging, the image sensor itself needs a phase difference detection function. Therefore, high-speed operation of about several hundred MHz is required. According to the present disclosure, when using pulsed light, the image sensor 108 can detect the phase difference depending on whether or not the third light 9003 can be detected. Alternatively, when continuous light is used, the image sensor 108 can detect the phase difference only by measuring the intensity change. Therefore, the image sensor 108 only needs to have an operation speed capable of responding to the time for switching the phase of the second light 9002, that is, the time for switching the imaging distance. For example, when the positional relationship between the subject and the imaging system is fixed, this time can be made as long as possible in principle. As described above, the operation speed of the image sensor is not in principle.

  In applications where relative motion can occur with a subject such as in-vehicle monitoring, a certain degree of time resolution is required. Therefore, the time required for phase switching is limited. However, according to the present embodiment, for example, when 100 sets of 100 types of distance information are to be acquired per second, 0.1 milliseconds is secured for measurement at each phase. This means that the operating speed of the image sensor may be several tens of kHz. The operation speed of the organic image sensor is lower than that of the silicon image sensor. However, this operation speed is a speed that can be realized even with an organic image sensor.

  Further, the image sensor 108 itself does not need to be provided with a complicated circuit for phase detection. Therefore, an image sensor using a compound semiconductor, which is difficult to integrate complicated circuits, can be used.

  It is obvious that the present disclosure can be applied to a range gate camera. The interval at which the gate is opened can be controlled by the pulse width of the second light 9002. For example, a light source (for example, a femtosecond laser) that emits light having a narrow pulse width up to femtosecond is currently known. Using them, a gate width that is sufficiently narrow in practice can be realized.

  Note that if the image sensor 108 in this embodiment is replaced with a non-integrated optical detector, the imaging system 10B in this embodiment can function as a rider. In that case, the imaging device described in Embodiment 1 functions as a measurement device. When the spatial resolution is not required, this embodiment may be carried out as such. Alternatively, when it is desired to use a scanning mechanism, they may be incorporated in this embodiment.

  Next, an example of the operation of the TOF imaging system 10B will be described with reference to FIG.

  FIG. 8 shows an operation flow of the TOF imaging system 10B. For example, the controller 111 (see FIG. 4) that controls the entire TOF imaging system 10B executes the following steps. Specifically, a computer program including an instruction corresponding to each step may be mounted in the memory inside the controller 111, and the controller 111 reads out the computer program from the memory and sequentially executes each step. You may make it perform.

  The first light source 101 generates first light 9001 subjected to intensity modulation (step S101B). The first optical system 104 is used to irradiate the subject within the imaging range with the first light 9001 (step S102B). The reflected light 9011 from the subject is collected using the second optical system 105 (step S103B).

  On the other hand, in the second light source 102, the modulation phase is set to a value corresponding to the initial measurement distance (step S110B), and the modulated second light 9002 is generated. The modulation may be any of intensity modulation, polarization modulation, and wavelength modulation. For example, the second light 9002 subjected to polarization modulation is generated (step S104B). The optical path of the second light 9002 is controlled by the third optical system 106 so that the second light 9002 enters the range where SFG occurs in the nonlinear optical crystal 103 (step S105B).

  The reflected light 9011 and the second light 9002 are incident on the nonlinear optical crystal 103 to generate the third light 9003 (step S106B).

  The image sensor 108 is caused to detect an image of a subject by the third light 9003 and output an imaging signal (step S107B).

  The light intensity of the third light 9003 depends on the relative relationship between the phase of intensity modulation of the reflected wave 9011 and the phase of modulation of the second light 9002. Since the modulation of the first light 9001 and the modulation of the second light 9002 are both pulsed, the third light 9003 has a specific distance corresponding to the phase value of the modulation of the second light 9002. It is generated only from the reflected light 9011 from the subject 1001 having the relationship.

  The difference in reflectance of the subject 1001 affects the strength of the third light 9003, but does not affect the distance information. Therefore, when both the modulation of the first light 9001 and the modulation of the second light 9002 are pulsed, the distance information can be extracted only by detecting the image of the subject by the third light 9003. it can.

  However, according to this method, only a subject at a specific distance can be photographed with respect to one value of the phase of modulation of the second light 9002. Therefore, when it is desired to measure the distance to a subject within a measurement range having a certain width, it is necessary to sequentially perform imaging by changing the modulation phase of the second light 9002 to a plurality of values. The set number of modulation phases of the second light 9002 matches the number of distances to be measured, which can be determined from the distance measurement range and distance measurement accuracy.

  When measuring the modulation phase with a plurality of set values, the number of required modulation phase values is determined in advance, and it is determined whether the number of measured phase values has reached a desired value. (Step S108B). If the desired value is not reached, the modulation phase value of the second light 9002 is changed to a value corresponding to the next measurement distance (step S109B). Then, the measurement is repeated from step S104B. If the desired value has been reached, the phase of modulation of the second light 9002 is returned to a value corresponding to the initial measurement distance, and the measurement is repeated from step S110B.

  The controller 111 repeatedly executes the above steps S101B to S110B until a predetermined condition is satisfied. The acquired imaging signal includes information indicating a change in the intensity of the third light, which includes distance information to the subject.

  According to the above procedure, the distance information to the subject can be extracted by the processor 110.

  Further, the processor 110 may perform image analysis such as object recognition based on the acquired imaging signal.

  According to the present embodiment, it is possible to perform TOF imaging using light having a wavelength at which the image sensor 108 has no sensitivity as illumination light. In addition, the subject image can be detected simultaneously with the distance measurement.

(Embodiment 3)
An imaging system 10C according to the present embodiment will be described with reference to FIGS. 9 to 12E. The imaging system 10C has substantially the same configuration as the imaging system 10A according to the first embodiment. In the present embodiment, the point that the imaging system 10C functions efficiently in shooting during bad weather such as fog will be described.

  FIG. 9 schematically shows a configuration of an imaging system 10C according to the present embodiment. As illustrated, the subject 1002 includes a fog 1041. The reflected light 9011 from the subject 1002 includes reflected light 9311 from the subject 1031 and reflected light 9411 from fog. Hereinafter, it is assumed that the modulation of the second light 9002 is polarization modulation.

  FIG. 10A shows the intensity of the first light 9001. FIG. 10B shows the intensity of the reflected light 9311 from the subject. FIG. 10C shows the intensity of the reflected light 9411 from the fog.

  The first light source 101 emits first light 9001 subjected to intensity modulation. The modulation may be pulsed or continuous. In this embodiment mode, the first light 9001 is continuously modulated. Continuous modulation is easier to implement than pulse modulation. As described above, the wavelength of the first light 9001 can be freely selected. In this embodiment, 1550 nm light is used. In order to reduce the influence of light scattering due to fog or the like, a wavelength as long as possible is advantageous. The frequency of modulation can be 10 GHz, for example. If the modulator uses an electro-optic element, modulation at this speed is also possible.

  The first optical system 104 irradiates the subject 1002 with the first light 9001. The subject 1002 is irradiated with the intensity-modulated first light 9001. At this time, the first light 9001 is reflected by the subject 1031 such as a pedestrian, an obstacle, and a car, and the fog 1041. A part of the reflected light 9311 from the subject 1031 and a part of the reflected light 9411 from the fog 1041 are both directed to the second optical system 105. The reflected lights 9311 and 9411 are incident on the nonlinear optical crystal 103 via the second optical system 105. In the present embodiment, the second optical system 105 includes a polarizer. Due to the polarizer, only light in a specific polarization state enters the nonlinear optical crystal 103.

  As shown in FIG. 10B, the reflected light 9311 from the subject 1031 enters the nonlinear optical crystal 103 while maintaining the modulation waveform of the original light (first light 9001) on a time scale of about 10 GHz.

  On the other hand, as shown in FIG. 10C, the reflected light 9411 from the fog 1041 does not have the original modulation waveform because the fog 1041 is spatially continuously distributed and the light is multiple-scattered. Therefore, the reflected light 9411 is incident on the nonlinear optical crystal 103 in an averaged state without a high frequency component. For example, the reflected light 9411 by the fog 9411 distributed over a range of 10 m has a spread of about 10 nanoseconds. Therefore, the original 10 GHz modulation information is almost missing.

  The second light source 102 emits the modulated second light 9002. The modulation may be any of intensity modulation, polarization modulation, and wavelength modulation. In this embodiment mode, the second light 9002 is subjected to polarization modulation. In addition, light having a wavelength of 1064 nm is used as the second light 9002. The period of polarization modulation of the second light 9002 and the period of intensity modulation of the first light 9001 are the same. This is because information processing becomes easier when the modulation period is the same. However, the modulation period of the first light 9001 and the modulation period of the second light 9001 may not be the same. For example, the integral multiple of the modulation period of the first light 9001 and the integral multiple of the modulation period of the second light 9002 may be the same. Such a case is substantially equivalent to the case where each is modulated at the least common multiple of the modulation period of the first light 9001 and the modulation period of the second light 9002.

  First, the modulation phase of the second light 9002 is fixed to a certain value (for example, the first phase). The optical path of the second light 9002 is adjusted by the third optical system 106 so that the second light 9002 enters the nonlinear optical crystal 103 in a state where the optical path overlaps with the reflected light 9311 and the reflected light 9411.

  Reflected light 9311, reflected light 9411, and second light 9002 enter the nonlinear optical crystal 103. SFG occurs in the nonlinear optical crystal 103 only when the reflected light 9311 or the reflected light 9411 and the second light 9002 enter at the same time and satisfy the phase matching condition.

  FIG. 11A shows the light intensity of the reflected light 9311. FIG. 11B shows the light intensity of the reflected light 9411. FIG. 11C shows the intensity of light of the polarization component involved in phase matching in the second light 9002 when the phase of the second light 9002 is the first phase. FIG. 11D shows the intensity of the third light 9303 generated based on the reflected light 9311 and the second light 9002. FIG. 11E shows the intensity of the third light 9403 generated based on the reflected light 9411 and the second light 9002.

  As shown in the drawing, the third light 9303 is generated based on the polarized light component of the second light 9002 and the reflected light 9311 in the phase matching. Further, out of the second light 9002, third light 9403 is generated based on the polarized light component involved in the phase matching and the reflected light 9411.

  The phase matching condition depends on the polarization state of the two incident lights. When the first light 9001 is linearly polarized light, only the polarization component of a specific polarization direction in the second light 9002 satisfies the phase matching condition. Therefore, even if the intensity of the second light 9002 is constant, when the polarization modulation is applied to the second light 9002, the intensity of the third light 9003 is modulated as a result. As described above, modulating the polarization component satisfying the phase matching condition in the second light 9002 plays the same role as intensity modulation.

  The first phase is a phase of modulation of the second light 9002 when the peak of the modulation waveform of the reflected light 9311 from the subject 1031 and the peak of the modulation waveform of the second light 9002 overlap. When the phase of modulation of the second light 9002 is set to this phase, the third light 9303 is generated most strongly. Then, the time average intensity of the third light 9303 is maximized. On the other hand, when the modulation phase of the second light 9002 is set so that the peak of the modulation waveform of the reflected light 9311 from the subject 1031 and the valley of the modulation waveform of the second light 9002 overlap. The third light 9303 is the weakest. And the time average intensity becomes the minimum.

  FIG. 12A shows the intensity of the reflected light 9311. FIG. 12B shows the intensity of the reflected light 9411. FIG. 12C shows a polarization component (intensity) related to phase matching in the second light 9002 when the phase of modulation of the second light 9002 is a second phase different from the first phase. . FIG. 12D shows the intensity of the third light 9303 generated based on the reflected light 9311 and the second light 9002. FIG. 12E shows the intensity of the third light 9403 generated based on the reflected light 9411 and the second light 9002.

  The intensity of the third light 9303 shown in FIG. 12D is lower than the intensity of the third light 9303 shown in FIG. 11D.

  Whether the reflected light 9011 and the second light 9002 have an intensifying phase relationship, a weakening phase relationship, or an intermediate relationship depends on the distance to the subject. On the other hand, if the phase of the second light 9002 is changed, the phase relationship between the two changes. In that case, the time average intensity of the third light 9303 generated by the reflected light 9011 from the subject changes regardless of the distance of the subject.

  The reflected light 9411 from the fog lacks the modulation component. Therefore, even if the phase of the second light 9002 is changed, the time average intensity of the third light 9403 generated from the reflected light 9411 does not change. The intensity of the third light 9403 shown in FIG. 12E is substantially the same as the intensity of the third light 9403 shown in FIG. 11E. Moreover, the time average intensity | strength is also substantially the same in both.

  The image sensor 108 is typically a storage type detector. The image sensor 108 outputs the total amount of light incident during the exposure time (integrated value of the amount of light). Therefore, by making the exposure time longer than the modulation period of the second light, an imaging result equivalent to the time average intensity can be obtained without additional processing. In that case, an additional averaging circuit or the like is unnecessary. For example, in the case of 10 GHz modulation, the period is 0.5 nanoseconds. If exposure is performed for 5 nanoseconds, an average value for 10 cycles can be acquired.

  Thus, in principle, the exposure time can be extended without limitation. As the time for averaging (the number of integrations) is longer, noise is reduced, so that the accuracy of distance measurement is improved. However, in practice, it is desirable to set the exposure time so that the relative position with respect to the subject does not change. For example, in the case of an automobile traveling at 100 km / h, the automobile moves only 2.8 mm in 0.1 milliseconds. That is, even if the imaging system according to the present disclosure is installed in a vehicle traveling at a speed of 100 km / h, the relative distance to the subject changes only a few millimeters in 0.1 milliseconds. The change can be ignored. Therefore, for example, it may take 0.1 milliseconds to measure one phase. This means that a sufficient effect can be obtained even if an image sensor operating at a low speed such as an organic image sensor is used.

  The image sensor 108 acquires a first image signal including the subject 1031 and the fog 1041 illustrated in FIG. 9 when the phase of modulation of the second light 9002 is the first phase. Further, it is assumed that the image sensor 108 acquires a second image signal including the subject 1031 and fog when the phase of modulation of the second light 9002 is a second phase different from the first phase. The first image signal and the second image signal both include the subject 1031 and the fog 1041. However, the image signal intensity of the subject 1031 changes depending on the modulation phase, whereas the image signal of the fog 1041 does not change regardless of the modulation phase. Therefore, the image signal based on fog is removed by taking the difference between these image signals. Thus, the influence of multiple scattering of light due to fog can be suppressed, and a differential image in which reflected light from the subject is emphasized can be acquired.

  FIG. 13 is a diagram schematically showing the state of multiple scattering of light by the mist 1041. When the first and second lights are pulse lights, the pulse width dispersion (time dispersion) of the third light 9003 based on the reflected light 9311 from the subject is small. On the other hand, the time dispersion of the third light 9003 based on the reflected light 9411 from the fog is large due to the influence of multiple scattering by the fog 1041. The image sensor 108 may have a function of measuring the time dispersion of the third light. By comparing the time dispersion of the return light, the fog and the subject can be clearly distinguished. In this case, a time resolution for distinguishing dispersion of about several tens of ps is required.

  According to the present embodiment, it is possible to accurately detect a subject (pedestrian, oncoming vehicle, or obstacle) in the fog.

  Next, an example of the operation of the imaging system 10C will be described with reference to FIG.

  FIG. 14 shows an operation flow of the imaging system 10C. For example, the controller 111 (see FIG. 9) that controls the entire imaging system 10C executes the following steps. Specifically, a computer program including an instruction corresponding to each step may be mounted in the memory inside the controller 111, and the controller 111 calls the computer program from the memory and sequentially executes each step. You may make it perform.

  The first light source 101 generates first light 9001 subjected to intensity modulation (step S201). The first optical system 104 is used to irradiate the subject within the imaging range with the first light 9001 (step S202). The reflected light 9011 from the subject is collected using the second optical system 105 (step S203).

  On the other hand, the modulation phase of the second light 9002 is set to the first phase (step S207). The phase of the second light source 102 is controlled to generate modulated second light 9002 (step S208). The optical path of the second light 9002 is controlled by the third optical system 106 so that the second light 9002 enters the range where SFG occurs in the nonlinear optical crystal 103 (step S209).

  The reflected light 9011 and the second light 9002 are incident on the nonlinear optical crystal 103 to generate the third light 9003 (step S204).

  The image sensor 108 images a subject based on the third light 9003. For example, the image sensor 108 acquires the integrated value of the pulse group in the above-described 0.1 milliseconds. The acquired imaging signal is stored in a memory corresponding to the first phase (step S205). The memory is a memory in the image sensor 108, for example.

  It is determined whether or not the phase of modulation of the second light 9002 is the second phase (step S206). When the modulation phase of the second light 9002 is set to the first phase, the modulation phase of the second light 9002 is changed from the first phase to the second phase (step S211). In the state where the modulation phase of the second light 9002 is set to the second phase, the subject based on the third light 9003 is finally imaged in accordance with steps S208, S209, S204, and S205. The image sensor 108 acquires the integrated value of the pulse group for 0.1 milliseconds, for example. The acquired imaging signal is stored in a memory corresponding to the second phase (step S205).

  In step S206, when the modulation phase of the second light 9002 is set to the second phase, the imaging signals corresponding to the first and second phases are stored in the corresponding memories. The image sensor 108 outputs the difference between the two signals to the outside (step S210). For example, when fog or the like is included in the imaging range, the difference between the two can be taken to suppress the influence of multiple scattering of light due to the fog, and a difference image in which reflected light from the subject is emphasized can be acquired.

  The processor 110 detects the subject based on the difference information from the image sensor 108. The processor 110 may perform image analysis such as object recognition.

(Embodiment 4)
With reference to FIGS. 15 to 17D, an imaging system 10D in which the relationship between the phase of modulation of the first light 9001 and the phase of modulation of the second light 9002 is fixed will be described.

  FIG. 15 schematically shows a configuration of an imaging system 10D according to the present embodiment. The configuration of this system is substantially the same as that of the second embodiment. Hereinafter, the description will focus on the differences from the second embodiment, and a detailed description of the same contents will be omitted.

  The first light source 101 generates first light 9001 in which any one of intensity, polarization state, and wavelength periodically changes over time. In this embodiment mode, pulsed light having an intensity only for a short time width at regular time intervals is referred to as first light 9001. The first light 9001 is a light whose polarization state or wavelength is periodically changed so as to be phase-matched with the second light 9002 described later in the nonlinear optical crystal 103 only for a short time width at regular time intervals. It is good. However, the phase of modulation of the first light 9001 is fixed. Hereinafter, an example in which the intensity periodically changes with time will be described.

  The object 1001 is irradiated with the first light 9001 using the first optical system 104. The subject 1001 includes a subject 1021 and a subject 1011 that are at different distances from each other. The subject 1001 is irradiated with first light 9001 in which any one of intensity, polarization state, and wavelength periodically changes over time. Part of the first light 9001 is reflected by the subject 1001, and the reflected light 9011 travels to the second optical system 105. The reflected light 9011 is incident on the nonlinear optical crystal 103 via the second optical system 105. The reflected light 9011 includes reflected light 9211 from the subject 1021 and reflected light 9111 from the subject 1011.

  FIG. 16A shows the intensity of the first light 9001. FIG. 16B shows the intensity of the reflected light 9111. FIG. 16C shows the intensity of the reflected light 9211. The reflected light 9011 includes light reflected from various subjects 1001. As described in the second embodiment, the phase of modulation of each reflected light corresponds to the distance to the subject 1001. As shown in FIGS. 16A to 16C, the reflected light 9111 from the nearby subject 1011 changes as shown in FIG. 16B with respect to the temporal change of the first light 9001. In addition, the reflected light 9211 from the distant subject 1021 changes as shown in FIG. 16C, and the phase is delayed compared to the reflected light 9111.

  The second light source 102 generates the second light 9002 in which any one of the intensity, the polarization state, and the wavelength changes periodically with time. Any of light intensity, polarization state, and wavelength may be changed. The phase of modulation of the second light 9002 is fixed. Hereinafter, similarly to the first light source 101, an example in which the intensity changes with time will be described.

  In this embodiment mode, light having intensity for a certain period is referred to as second light 9002 periodically. Alternatively, in the nonlinear optical crystal 103, light having a polarization state or wavelength that periodically matches the reflected light 9011 of the first light 9001 only for a certain period may be used as the second light 9002. The modulation period of the second light 9002 is made to coincide with the modulation period of the first light 9001. However, with respect to the second light 9002, the width of the period having intensity or the width of the period in which the polarization state or wavelength is phase-matched with the reflected light 9011 does not need to match the pulse width of the first light 9001. . Hereinafter, an example in which the width of the period is 33 nanoseconds will be described.

  FIG. 17A shows the change over time of the intensity of the reflected light 9111 from the near subject 1011. FIG. 17B shows a temporal change in the intensity of the reflected light 9211 from the distant subject 1021. FIG. 17C shows the time change of the intensity of the second light 9002. FIG. 17D shows the time change of the intensity of the third light 9003 generated by the nonlinear optical crystal 103.

  As shown in FIG. 17D, the SFG phenomenon occurs in the nonlinear optical crystal 103 only during the period in which the pulse of the reflected light 9111 or the reflected light 9211 and the pulse of the second light 9002 overlap, and the third light 9003 is generated. At this time, the modulation phases of the reflected light 9111 and the reflected light 9211 correspond to the distances to the subjects 1011 and 1021, respectively. Therefore, if the phase relationship between the modulation phase of the first light 9001 and the modulation phase of the second light 9002 is kept constant, only the subject in the distance range corresponding to the period in which the pulses overlap is imaged. . The 33 nanoseconds described above corresponds to a distance of 10 m to the subject 1001. That is, only a subject within a distance of 10 m can be imaged. Needless to say, the distance range for imaging can be arbitrarily set based on the modulation period of the second light 9002 and the pulse width of the second light 9002.

  Such a technique for imaging only a subject within a specific distance range is useful for safety measures and security applications. For example, if an imaging system 10D set so that only a subject within a distance of 10 m is imaged is mounted on a vehicle, a warning can be issued when an object is imaged. Thus, the measurement system of the present disclosure can be used as a collision prevention measure.

  In addition, if a system is constructed so that an area around the building where people do not normally enter is imaged, imaging is performed only when a suspicious person enters. Only in that case, a warning can be issued and the record can be kept. Thus, the measurement system of the present disclosure can function as a monitoring system.

  The present disclosure further includes a light source unit and a measurement method described in the following items.

[Item 1]
A first light source that generates first light in which at least one of intensity, polarization state, and wavelength is modulated in a first period and emits the first light toward a subject; and intensity, polarization state, and wavelength At least one of them generates a second light modulated with a second period, emits the second light, light from the subject based on the first light, and the second light A light source unit comprising: an optical system that mixes the two lights.

[Item 2]
Due to the sum frequency generation phenomenon, third light having a frequency that is the sum of the frequency of the light from the subject based on the first light and the frequency of the second light is generated from the mixed light. Item 2. The light source unit according to Item 1, further comprising a nonlinear optical crystal.

[Item 3]
Generating first light in which at least one of intensity, polarization state, and wavelength is modulated in a first period, and emitting the first light toward a subject;
Generating second light in which at least one of intensity, polarization state, and wavelength is modulated in a second period, and emitting the second light;
Mixing the light from the subject based on the first light and the second light;
From the mixed light, a third light having a sum frequency of the frequency of the light from the subject based on the first light and the frequency of the second light is generated by a sum frequency generation phenomenon. about,
A measurement method comprising measuring the intensity of the third light.

[Item 4]
Generating first light in which at least one of intensity, polarization state, and wavelength is modulated in a first period, and emitting the first light toward a subject;
Generating second light in which at least one of intensity, polarization state, and wavelength is modulated in a second period, and emitting the second light;
A measurement method comprising measuring a relative relationship between a phase of modulation of light from the subject based on the first light and a phase of modulation of the second light.

  The measurement system according to the present disclosure can be used as, for example, a pedestrian / obstacle detection device or a distance measurement device for in-vehicle use.

10A, 10B, 10C, 10D Imaging system 101 First light source 102 Second light source 103 Nonlinear optical crystal 104 First optical system 105 Second optical system 106 Third optical system 107 Fourth optical system 108 Image sensor 109 Light mixing element 110 Processor 111 Controller 1000, 1001, 1002 Subject 1011 Near subject 1021 Distant subject 1031 Solid subject 1041 Fog 9001 First light 9002 Second light 9003 Third light 9011 First light 9011 Reflected light 9111 First light reflected by a near subject 9211 Third light component generated by a first light 9303 9311 reflected by a far subject 9403 9411 Third light component generated by a solid light 9311 9311 Solid First light reflected by the subject that is 9 411 First light reflected by fog

Claims (12)

A first light source that generates first light in which at least one of intensity, polarization state, and wavelength is modulated in a first period, and emits the first light toward a subject;
A second light source that generates second light in which at least one of intensity, polarization state, and wavelength is modulated in a second period, and emits the second light;
An optical system for mixing the light from the subject based on the first light and the second light;
From the mixed light, a third light having a sum frequency of the frequency of the light from the subject based on the first light and the frequency of the second light is generated by a sum frequency generation phenomenon. A nonlinear optical crystal;
A photodetector for measuring the intensity of the third light;
Including a measurement system.   The said 1st light source produces | generates the 4th light from which the phase of modulation differs from the said 1st light, and switches and emits the said 1st light and the said 4th light. Measuring system.   The said 2nd light source produces | generates the 5th light from which the phase of a modulation | alteration differs from the said 2nd light, and switches and emits the said 2nd light and the said 5th light. Measuring system.   The measurement system according to claim 1, wherein the third light has a visible wavelength.   The measurement system according to claim 1, wherein the photodetector has a higher sensitivity to the third light than the first and second lights. An optical system for condensing the third light on the photodetector;
The measurement system according to claim 1, wherein the photodetector is an image sensor.   The measurement system according to claim 6, wherein the image sensor is an organic image sensor. A processor for processing an output signal from the image sensor;
The processor is
A first imaging signal acquired from the output signal;
The second imaging signal acquired when one of the phase of the modulation of the first light and the phase of the modulation of the second light is different from when the first imaging signal is acquired The measurement system according to claim 7, wherein the subject is detected based on a difference between the subject and the subject. A processor for processing an output signal from the photodetector;
The measurement system according to claim 1, wherein the processor measures a distance to the subject based on the output signal.   The measurement system according to claim 1, wherein the first light has an infrared wavelength. A first light source that generates first light in which at least one of intensity, polarization state, and wavelength is modulated in a first period, and emits the first light toward a subject;
A second light source that generates second light in which at least one of intensity, polarization state, and wavelength is modulated in a second period, and emits the second light;
First information including a relative relationship between the phase of the modulation of the first light and the phase of the modulation of the second light from the light from the subject and the second light based on the first light. An optical element that generates 3 light;
A photodetector for measuring the intensity of the third light;
A measurement system comprising: A first light source that generates first light in which at least one of intensity, polarization state, and wavelength is modulated in a first period, and emits the first light toward a subject;
A second light source that generates second light in which at least one of intensity, polarization state, and wavelength is modulated in a second period, and emits the second light;
A measuring device for measuring a relative relationship between a phase of modulation of light from the subject based on the first light and a phase of modulation of the second light;
A measurement system comprising:

Images