WO2024203081A1 - 受光素子、測距装置 - Google Patents

受光素子、測距装置 Download PDF

Info

Publication number
WO2024203081A1
WO2024203081A1 PCT/JP2024/008545 JP2024008545W WO2024203081A1 WO 2024203081 A1 WO2024203081 A1 WO 2024203081A1 JP 2024008545 W JP2024008545 W JP 2024008545W WO 2024203081 A1 WO2024203081 A1 WO 2024203081A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
receiving element
unit
center
photoelectric conversion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2024/008545
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
雄飛 寄門
創造 横川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Semiconductor Solutions Corp
Original Assignee
Sony Semiconductor Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Semiconductor Solutions Corp filed Critical Sony Semiconductor Solutions Corp
Priority to JP2025510150A priority Critical patent/JPWO2024203081A1/ja
Priority to KR1020257031583A priority patent/KR20250168217A/ko
Publication of WO2024203081A1 publication Critical patent/WO2024203081A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C3/00—Measuring distances in line of sight; Optical rangefinders
    • G01C3/02—Details
    • G01C3/06—Use of electric means to obtain final indication
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481—Constructional features, e.g. arrangements of optical elements
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/486—Receivers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/703—SSIS architectures incorporating pixels for producing signals other than image signals
    • H04N25/705—Pixels for depth measurement, e.g. RGBZ
    • H—ELECTRICITY
    • H10—SEMICONDUCTOR DEVICES; ELECTRIC SOLID-STATE DEVICES NOT OTHERWISE PROVIDED FOR
    • H10F—INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION
    • H10F39/00—Integrated devices, or assemblies of multiple devices, comprising at least one element covered by group H10F30/00, e.g. radiation detectors comprising photodiode arrays
    • H10F39/10—Integrated devices
    • H10F39/12—Image sensors

Definitions

  • This technology relates to light receiving elements and distance measuring devices, for example, light receiving elements and distance measuring devices that perform accurate distance measurements.
  • the distance measurement method used in distance measurement modules is the TOF (Time of Flight) method.
  • TOF imaging elements light is emitted from a light source, hits an object, and is reflected, which is then photoelectrically converted by a photodiode. The signal charge generated by the photoelectric conversion is distributed to two FDs (floating diffusions) by a pair of gate electrodes that are alternately driven (see, for example, Patent Document 1).
  • a technology called pupil correction has been proposed to suppress the occurrence of uneven sensitivity in an image, known as shading, which occurs when there is a large difference between the amount of light incident on pixels at the center of the image plane's angle of view and the amount of light incident on pixels at the edge of the angle of view.
  • pupil correction technology is applied to an image sensor that has a structure in which the signal charge generated by photoelectric conversion is distributed to two FDs by a pair of gate electrodes that are driven alternately, such as the ToF image sensor described above, there is a possibility that the sensitivity of the two FDs will differ. It is desirable to be able to perform appropriate pupil correction for ToF image sensors.
  • This technology was developed in light of these circumstances, and makes it possible to perform appropriate pupil correction for ToF imaging devices.
  • the light receiving element of one aspect of the present technology includes a photoelectric conversion unit that performs photoelectric conversion, first and second charge storage units that store the charge obtained by the photoelectric conversion unit, and an on-chip lens that focuses incident light on the photoelectric conversion unit, and the center of the on-chip lens and the center of the photoelectric conversion unit are separated by a correction amount that is set so that light incident on the center of the on-chip lens is directed to a position that is the same distance from the center of gravity of the first charge storage unit and the second charge storage unit.
  • a distance measuring device includes a light emitting unit that emits irradiated light, and a light receiving element that receives light reflected from an object when the light from the light emitting unit is reflected by the object.
  • the light receiving element includes a photoelectric conversion unit that performs photoelectric conversion, first and second charge storage units that store the charge obtained by the photoelectric conversion unit, and an on-chip lens that focuses incident light on the photoelectric conversion unit.
  • the center of the on-chip lens and the center of the photoelectric conversion unit are separated by a correction amount that is set so that light incident on the center of the on-chip lens is directed to a position that is equal in distance from the center of gravity of the first charge storage unit and the second charge storage unit.
  • the light receiving element includes a photoelectric conversion unit that performs photoelectric conversion, first and second charge storage units that store the charge obtained by the photoelectric conversion unit, and an on-chip lens that focuses incident light on the photoelectric conversion unit.
  • the center of the on-chip lens and the center of the photoelectric conversion unit are separated by a correction amount that is set so that light incident on the center of the on-chip lens is directed toward a position that is the same distance from the center of gravity of the first charge storage unit as it is from the center of gravity of the second charge storage unit.
  • the distance measuring device is configured to include the light receiving element.
  • the distance measuring device may be an independent device or an internal block that constitutes a single device.
  • FIG. 1 is a block diagram showing a configuration example of a distance measuring module according to an embodiment of the present disclosure.
  • FIG. 2 is a diagram illustrating an example of a planar configuration of a light receiving section.
  • FIG. 2 is a diagram illustrating an example of a circuit configuration of a light receiving element.
  • FIG. 11 is a diagram for explaining a distance measuring operation.
  • FIG. 2 is a diagram showing an example of a planar configuration of a light receiving element;
  • FIG. 2 is a diagram illustrating an example of a cross-sectional configuration of a light receiving element.
  • FIG. 2 is a diagram illustrating an example of a circuit configuration of a light receiving element.
  • FIG. 2 is a diagram showing an example of a planar configuration of a light receiving element;
  • FIG. 1 is a block diagram showing a configuration example of a distance measuring module according to an embodiment of the present disclosure.
  • FIG. 2 is a diagram illustrating an example of a planar configuration of a light receiving section
  • FIG. 2 is a diagram illustrating an example of a circuit configuration of a light receiving element.
  • FIG. 2 is a diagram showing an example of a planar configuration of a light receiving element;
  • FIG. 2 is a diagram illustrating an example of a circuit configuration of a light receiving element.
  • FIG. 2 is a diagram showing an example of a planar configuration of a light receiving element;
  • 11 is a diagram for explaining a light receiving element on which pupil correction has been performed;
  • FIG. 11 is a diagram for explaining a light receiving element on which pupil correction has been performed;
  • FIG. 13 is a diagram for explaining the position at which incident light travels.
  • FIG. 11 is a diagram for explaining a method for calculating a correction amount.
  • FIG. 11 is a diagram for explaining a method for calculating a correction amount.
  • FIG. 13 is a diagram for explaining the position of a light-shielding film.
  • FIG. 13 is a diagram for explaining that PLS is equal.
  • FIG. 13 is a diagram for explaining that PLS is equal.
  • FIG. 13 is a diagram showing an example of the arrangement of four charge accumulation units.
  • FIG. 13 is a diagram showing an example of an arrangement of two taps.
  • 11A and 11B are diagrams for explaining an example of an arrangement of I pixels and Q pixels and a process for generating a distance measurement image.
  • 11A and 11B are diagrams for explaining an example of an arrangement of I pixels and Q pixels and a process for generating a distance measurement image.
  • FIG. 1 is a diagram illustrating an example of the configuration of an electronic device.
  • FIG. 1 is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system.
  • 2 is a block diagram showing an example of the functional configuration of a camera head and a CCU.
  • FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system;
  • 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit;
  • FIG. 1 is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system.
  • 2 is a block diagram showing an example of the functional configuration of a camera head and a CCU.
  • FIG. 1 is a block diagram showing an example of a schematic configuration of a vehicle control system
  • 4 is an explanatory diagram showing an example of the installation positions of an outside-vehicle information detection unit and an imaging unit;
  • Fig. 1 is a diagram showing an example of the configuration of a distance measuring module 1 according to an embodiment of the present disclosure. As shown in Fig. 1, the distance measuring module 1 has an irradiation unit 20, a light receiving unit 30, a control unit (irradiation control unit) 40, and a processing unit 60.
  • the irradiation unit 20 has an LED (Light Emitting Diode) light source (not shown) and an optical element (not shown).
  • the wavelength of the irradiated light can be changed by appropriately selecting the LED light source.
  • the irradiation unit 20 is described as irradiating infrared light with a wavelength in the range of 780 nm to 1000 nm, for example, but is not limited to irradiating such infrared light.
  • the irradiation unit 20 can irradiate the target object 800 with irradiation light whose brightness fluctuates periodically in synchronization with a periodic signal such as a rectangular signal supplied from the control unit 40 described below.
  • the light receiving unit 30 receives the reflected light reflected from the object 800.
  • the light receiving unit 30 has a condenser lens (not shown) and a plurality of light receiving elements 10 described later.
  • the condenser lens has a function of collecting the received light to each light receiving element 10.
  • the light receiving element 10 generates an electric charge (e.g., electrons) based on the intensity of the received light, and drives a built-in transistor in synchronization with a periodic signal such as a rectangular signal supplied from the control unit 40 described later, and transfers the generated electric charge to the floating diffusion region FD or the charge accumulation unit MEM. Furthermore, the electric charge transferred to the floating diffusion region FD or the charge accumulation unit MEM is converted into a signal and finally transferred to the processing unit 60.
  • the light receiving element 10 will be described in detail later.
  • the control unit 40 supplies periodic signals to the light emitting unit 20 and the light receiving unit 30 to control the timing of the light irradiation and the driving timing of the transistors.
  • the frequency of the signal can be, for example, 5 to 20 megahertz (MHz), but is not limited to such a frequency.
  • the control unit 40 controls the transistors to operate at different timings, for example differentially.
  • the processing unit 60 acquires a signal from the light receiving unit 30, and based on the acquired signal, can acquire the distance to the target object 800, for example, by the indirect ToF (iToF) method.
  • iToF indirect ToF
  • the light receiving section 30 includes a pixel array section 12, a vertical drive circuit section 32, a column signal processing circuit section 34, a horizontal drive circuit section 36, an output circuit section 38, and a control circuit section 44, which are provided on a semiconductor substrate 200 made of, for example, silicon.
  • the pixel array section 12 has a plurality of light receiving elements 10 arranged two-dimensionally in a matrix (row and column rows and columns) on the semiconductor substrate 200.
  • Each light receiving element 10 has a photoelectric conversion section (photodiode PD) (FIG. 6) that converts light into electric charge (e.g., electrons), and a plurality of pixel transistors (e.g., MOS (Metal-Oxide-Semiconductor) transistors) (FIG. 6).
  • the pixel array section 12 has a plurality of pixels that perform photoelectric conversion of incident light and output a signal corresponding to the resultant electric charge.
  • the pixel transistors can include transistors with various functions, such as a transfer transistor, a selection transistor, a reset transistor, and an amplification transistor.
  • the row direction refers to the horizontal arrangement direction of the light receiving elements 10
  • the column direction refers to the vertical arrangement direction of the light receiving elements 10.
  • the row direction is the left-right direction in FIG. 2, and the column direction is the up-down direction in FIG. 2.
  • pixel drive wiring 42 is wired along the row direction for each row of the matrix-like arrangement of light receiving elements 10, and vertical signal lines 48 are wired along the column direction for each column.
  • the pixel drive wiring 42 transmits a drive signal for driving the light receiving elements 10 when reading out a signal.
  • the vertical drive circuit unit 32 is formed of, for example, a shift register or an address decoder, selects a pixel drive wiring 42, supplies a pulse to the selected pixel drive wiring 42 for driving the light receiving elements 10, and drives all the light receiving elements 10 simultaneously or row by row. For example, the vertical drive circuit unit 32 selects and scans each light receiving element 10 of the pixel array unit 12 in the vertical direction (up and down in FIG. 2) row by row, and supplies a pixel signal based on the charge generated according to the amount of light received by the photodiode PD of each light receiving element 10 to the column signal processing circuit unit 34 (described later) through the vertical signal line 48.
  • the vertical drive circuit unit 32 selects and scans each light receiving element 10 of the pixel array unit 12 in the vertical direction (up and down in FIG. 2) row by row, and supplies a pixel signal based on the charge generated according to the amount of light received by the photodiode PD of each light receiving element 10 to the column signal processing circuit unit 34 (described later) through
  • the column signal processing circuit unit 34 is arranged for each column of the light receiving elements 10, and performs signal processing such as noise removal for each column of signals output from one row of the light receiving elements 10.
  • the column signal processing circuit unit 34 performs signal processing such as CDS (Correlated Double Sampling) and AD (Analog-Digital) conversion to remove fixed pattern noise inherent to the light receiving elements 10.
  • the horizontal drive circuit unit 36 is formed by, for example, a shift register or an address decoder, and by sequentially outputting horizontal scanning pulses, it can select each of the above-mentioned column signal processing circuit units 34 in turn and output a signal from each column signal processing circuit unit 34 to the horizontal signal line 46.
  • the output circuit unit 38 can process and output signals sequentially supplied from each of the column signal processing circuit units 34 described above through the horizontal signal line 46.
  • the output circuit unit 38 can function as a functional unit that performs buffering, for example, or can perform column variation correction, various digital signal processing, and other processes. Buffering refers to temporarily storing signals to compensate for differences in processing speeds and transfer speeds when exchanging signals.
  • the control circuit unit 44 receives an input clock and data instructing the operating mode, etc., and can also output data such as internal information of the light receiving element 10. That is, the control circuit unit 44 generates clock signals and control signals that serve as the basis for the operation of the vertical drive circuit unit 32, column signal processing circuit unit 34, horizontal drive circuit unit 36, etc., based on the vertical synchronization signal, horizontal synchronization signal, and master clock. The control circuit unit 44 then outputs the generated clock signals and control signals to the vertical drive circuit unit 32, column signal processing circuit unit 34, horizontal drive circuit unit 36, etc.
  • ⁇ Equivalent circuit of light receiving element> 3 is an equivalent circuit diagram of the light receiving element 10.
  • the light receiving element 10 has a photodiode PD as a photoelectric conversion element (photoelectric conversion unit) that converts light into electric charge, and a charge discharging transistor OFG (note that the charge discharging transistor OFG is shown as one transistor on the equivalent circuit, but may be composed of multiple transistors electrically connected in parallel).
  • the light receiving element 10 has two each of a distribution transistor VG, a charge storage unit (first charge storage unit, second charge storage unit) MEM, a transfer transistor TG, a floating diffusion region FD, a reset transistor RST, an amplification transistor AMP, and a selection transistor SEL.
  • one of the source and drain of the charge draining transistor OFG is electrically connected to a photodiode PD that generates charge by receiving light. Furthermore, the other of the source and drain of the charge draining transistor OFG is electrically connected to a power supply circuit (power supply potential VDD).
  • the charge draining transistor OFG becomes conductive in response to a voltage applied to its gate, and is able to drain the charge accumulated in the photodiode PD to the power supply circuit (power supply potential VDD).
  • one of the sources or drains of the distribution transistors VG1, VG2 is electrically connected to the photodiode PD, and the other of the sources or drains of the distribution transistors VG1, VG2 is electrically connected to the charge storage units MEM1, MEM2, respectively.
  • the distribution transistors VG1, VG2 become conductive in response to the voltage applied to their gates (first distribution gate, second distribution gate), and can transfer the charge stored in the photodiode PD to the charge storage units MEM1, MEM2, respectively.
  • the charge stored in the photodiode PD can be distributed to one of the two charge storage units MEM1 and MEM2.
  • the two charge storage units MEM1 and MEM2 share one photodiode PD.
  • one of the sources or drains of the transfer transistors TG1, TG2 is electrically connected to the other of the sources or drains of the distribution transistors VG1, VG2 and the charge storage units MEM1, MEM2. Furthermore, the other of the sources or drains of the transfer transistors TG1, TG2 is electrically connected to the floating diffusion regions FD1, FD2.
  • the transfer transistors TG1, TG2 are turned on in response to a voltage applied to their gates (transfer gates), and can transfer the charges stored in the charge storage units MEM1, MEM2 to the floating diffusion regions FD1, FD2. Since there are two charge storage units MEM1, MEM2, the transfer transistors TG1, TG2 can also share one floating diffusion region FD.
  • the floating diffusion regions FD1, FD2 are electrically connected to the gates of the amplification transistors AMP1, AMP2, which convert (amplify) the charge into a voltage and output it as a signal (pixel signal).
  • One of the source/drain of the amplification transistors AMP1, AMP2 is electrically connected to one of the source/drain of the selection transistors SEL1, SEL2, which output the signal obtained by conversion to the signal lines VSL1, VSL2 in accordance with a selection signal.
  • the other of the source/drain of the amplification transistors AMP1, AMP2 is electrically connected to the power supply circuit (power supply potential VDD).
  • the other of the source/drain of the selection transistors SEL1, SEL2 is electrically connected to the signal lines VSL1, VSL2 that transmit the converted voltage as a signal, and is further electrically connected to the column signal processing circuit unit 34 described above. Furthermore, the gates of the selection transistors SEL1, SEL2 are electrically connected to a selection line (not shown) that selects the row to which the signal is output, and is further electrically connected to the vertical drive circuit unit 32 described above. In other words, the charge accumulated in the floating diffusion regions FD1, FD2 is converted into a voltage by the amplification transistors AMP1, AMP2 under the control of the selection transistors SEL1, SEL2, and is output to the signal lines VSL1, VSL2.
  • the floating diffusion regions FD1, FD2 are electrically connected to one of the drains or sources of the reset transistors RST1, RST2 for resetting the accumulated charge.
  • the gates of the reset transistors RST1, RST2 are electrically connected to a reset signal line (not shown), and are further electrically connected to the vertical drive circuit unit 32 described above.
  • the other of the drains or sources of the reset transistors RST1, RST2 are electrically connected to a power supply circuit (power supply potential VDD).
  • the reset transistors RST1, RST2 become conductive in response to a voltage applied to their gates, and can reset (discharge to the power supply circuit (power supply potential VDD)) the charge accumulated in the floating diffusion regions FD1, FD2.
  • a discharge operation is performed to discharge the charge in the photodiode PD. That is, the charge discharge transistors OFG1 and OFG2 are turned on, and the charge in the photodiode PD is discharged to the power supply circuit (power supply potential VDD).
  • the distribution transistors VG1 and VG2 are controlled to operate at different timings (e.g., differentially).
  • the distribution transistor VG1 is turned on, and the charge of the photodiode PD is transferred to the charge storage unit MEM1.
  • the distribution transistor VG2 is turned on, and the charge of the photodiode PD is transferred to the charge storage unit MEM2. That is, the distribution transistors VG1 and VG2 distribute the charge generated in the photodiode PD to the charge storage units MEM1 and MEM2.
  • a discharge operation is performed to discharge the charge in the floating diffusion regions FD1 and FD2. That is, the reset transistors RST1 and RST2 are turned on, and the charge in the floating diffusion regions FD1 and FD2 is discharged to the power supply circuit (power supply potential VDD). After this, the charge (ktc noise) generated in the floating diffusion regions FD1 and FD2 is removed by CDS drive.
  • each light receiving element 10 in the pixel array unit 12 is selected in sequence.
  • the selection transistors SEL1 and SEL2 are turned on.
  • the charges stored in the floating diffusion regions FD1 and FD2 are output as signals to the signal lines VSL1 and VSL2.
  • the distance to the target object 800 can be calculated from the distribution ratio of the charges accumulated in the two floating diffusion regions FD1 and FD2. The principle behind this is briefly explained below.
  • Fig. 4 is an explanatory diagram for explaining the principle of the distance calculation method using the distance measurement module 1, and in detail, shows a schematic diagram of the time variation of the intensity of the irradiated light and the reflected light in the distance measurement module 1.
  • the distance measuring module 1 irradiates light modulated so that the light intensity varies periodically from the irradiation unit 20 towards the object 800.
  • the irradiated light is reflected by the object 800 and detected as reflected light by the light receiving unit 30 of the distance measuring module 1.
  • the detected reflected light (second row from the top in FIG. 4) has a phase difference ⁇ with respect to the irradiated light (first row from the top in FIG. 4), and the phase difference ⁇ becomes larger if the distance from the distance measuring module 1 to the object 800 is farther and becomes smaller if the distance from the distance measuring module 1 to the object 800 is closer.
  • the light receiving element 10 has, for example, the distribution transistors VG1 and VG2 that are differential with each other. Therefore, since the periods during which the distribution transistors VG1 and VG2 operate do not overlap, the charge stored in the photodiode PD is distributed to the charge storage units MEM1 and MEM2 during the periods of the regions 802a and 802b shown in gray in FIG. 4. In detail, the charge distributed to the charge storage units MEM1 and MEM2 is transferred to the floating diffusion regions FD1 and FD2, and is finally converted into a signal corresponding to the area, which is the integral value during the periods of the regions 802a and 802b. Therefore, as is clear from FIG.
  • the distance to the object 800 can be calculated by calculating the phase difference ⁇ based on the difference between the integral value of the region 802a and the integral value of the region 802b. It is also possible to calculate the phase difference ⁇ using the ratio of integral values instead of the difference between integral values, and then calculate the distance.
  • Fig. 5 is a diagram showing an example of the planar configuration of the light receiving element 10 having the circuit configuration shown in Fig. 3, and is a diagram in which the light receiving element 10 is viewed from above the surface of the semiconductor substrate 200. Note that the left-right direction in Fig. 5 corresponds to the row direction (left-right direction) in Fig. 2, and the up-down direction in Fig. 5 corresponds to the column direction (up-down direction) in Fig. 2.
  • an N-type semiconductor region 100 is formed in a P-type semiconductor substrate 200 in the center of the light receiving element 10, and the N-type semiconductor region 100 constitutes a part of the photodiode (photoelectric conversion unit) PD.
  • the gate electrodes (first distribution gate, second distribution gate) 150a, 150b of the distribution transistors VG1, VG2 are arranged so as to be line-symmetric (approximately line-symmetric) with respect to a center line (not shown) that passes through the center point (center) O of the photodiode PD and extends in the vertical direction (column direction) of the light receiving element 10.
  • the distribution transistors VG1, VG2 can distribute the charge generated in the photodiode PD to the charge storage units MEM1, MEM2 described later.
  • the gate electrodes 150a, 150b of the distribution transistors VG1, VG2 are arranged so as to overlap at least a part of the N-type semiconductor region 100.
  • the distribution transistor VG1 is composed of a gate electrode 150a, a gate insulating film (not shown) located between the gate electrode 150a and the semiconductor substrate 200, an N-type semiconductor region 100 as a source region, and N-type semiconductor regions 102a and 102b as drain regions.
  • the N-type semiconductor region 100 as a source region is also used as a photodiode PD, and the N-type semiconductor region as a drain region is also used as a charge storage unit MEM1.
  • the distribution transistor VG2 has a similar configuration to the distribution transistor VG1.
  • the charge storage units MEM1 and MEM2 are provided at positions that are point symmetrical (approximately point symmetrical) with respect to the center point (center) O of the photodiode PD.
  • the charge storage unit MEM1 can be a MOS (Metal-Oxide-Semiconductor) type capacitor that is composed of, for example, an electrode 154a, an insulating film (not shown) provided below the electrode 154a, and an N-type semiconductor region (not shown) provided below the insulating film.
  • the charge storage unit MEM2 also has a similar configuration to the charge storage unit MEM1.
  • the gate electrodes 152a, 152b (first charge discharge gate, second charge discharge gate) of the charge discharge transistors OFG1, OFG2 are arranged so as to be line-symmetric (approximately line-symmetric) with respect to a center line (not shown) that passes through the center point O of the photodiode PD and extends in the left-right direction (row direction) of the light receiving element 10.
  • the charge discharge transistor OFG1 is composed of a gate electrode 152a, a gate insulating film (not shown) located between the gate electrode 152a and the semiconductor substrate 200, an N-type semiconductor region 100 as a source region, and an N-type semiconductor region (not shown) as a drain region.
  • the N-type semiconductor region 100 as a source region is also used as a photodiode PD.
  • the charge discharge transistor OFG2 has a similar configuration to the charge discharge transistor OFG1.
  • the gate electrodes 156a and 156b of the transfer transistors TG1 and TG2 are arranged at positions that are point symmetrical (approximately point symmetrical) with respect to the center point (center) O of the photodiode PD.
  • the gate electrode 156a of the transfer transistor TG1 is arranged adjacent to the gate insulating film (not shown) located between the gate electrode 156a and the semiconductor substrate 200 and the electrode 154a of the charge storage unit MEM1 along the vertical direction (column direction) in FIG. 5
  • the gate electrode 156b of the transfer transistor TG2 is arranged adjacent to the electrode 154b of the charge storage unit MEM2 along the vertical direction (column direction) in FIG.
  • the transfer transistors TG1 and TG2 can transfer the charges stored in the charge storage units MEM1 and MEM2 to the floating diffusion regions FD1 and FD2 described later.
  • the transfer transistor TG1 is composed of a gate electrode 156a, an N-type semiconductor region (not shown) as a source region, and an N-type semiconductor region (not shown) as a drain region.
  • the transfer transistor TG2 has the same configuration as the transfer transistor TG1.
  • the reset transistors RST1, RST2, the amplification transistors AMP1, AMP2, and the selection transistors SEL1, SEL2 are arranged at positions that are point symmetrical (approximately point symmetrical) with respect to the center point (center) O of the photodiode PD.
  • the reset transistor RST1, the amplification transistor AMP1, and the selection transistor SEL1 are arranged adjacent to each other in the left-right direction (row direction) in FIG. 5, and the reset transistor RST2, the amplification transistor AMP2, and the selection transistor SEL2 are also arranged adjacent to each other in the left-right direction (row direction) in FIG. 5.
  • the reset transistor RST1 is composed of a gate electrode 158a, a gate insulating film (not shown) located between the gate electrode 158a and the semiconductor substrate 200, an N-type semiconductor region (not shown) as a source region, and an N-type semiconductor region (not shown) as a drain region.
  • the N-type semiconductor region as a source region is also used as the floating diffusion region FD1
  • the N-type semiconductor region as a drain region is also used as the amplification transistor AMP1.
  • the reset transistor RST2 has a similar configuration to the reset transistor RST1.
  • the amplification transistor AMP1 is composed of a gate electrode 160a, a gate insulating film (not shown) located between the gate electrode 160a and the semiconductor substrate 200, an N-type semiconductor region (not shown) as a drain region, and an N-type semiconductor region (not shown) as a source region.
  • the N-type semiconductor region serving as the drain region is also used as the drain region of the reset transistor RST1.
  • the amplifier transistor AMP2 has the same configuration as the amplifier transistor AMP1.
  • the select transistor SEL1 is composed of a gate electrode 162a, a gate insulating film (not shown) located between the gate electrode 162a and the semiconductor substrate 200, an N-type semiconductor region (not shown) serving as the drain region, and an N-type semiconductor region (not shown) serving as the source region.
  • the N-type semiconductor region serving as the drain region is also used as the source region of the amplifier transistor AMP1.
  • the select transistor SEL2 has the same configuration as the select transistor SEL1.
  • planar configuration of the light receiving element 10 according to this embodiment is not limited to the example shown in FIG. 5, and may include other elements, for example, and is not particularly limited.
  • Fig. 6 ⁇ Cross-sectional structure of light receiving element>
  • the upper side in Fig. 6 corresponds to the back surface side of the semiconductor substrate 200
  • the lower side in Fig. 6 corresponds to the front surface side of the semiconductor substrate 200.
  • the light receiving element 10 has a P-type semiconductor substrate 200 made of, for example, a silicon substrate.
  • N-type semiconductor regions 100a and 100b are formed in the P-type semiconductor substrate 200, thereby forming a photodiode PD in the semiconductor substrate 200.
  • An on-chip lens 208 made of styrene resin, acrylic resin, styrene-acrylic copolymer resin, siloxane resin, or the like is provided above the back surface of the semiconductor substrate 200, into which reflected light from the target object 800 is incident.
  • a planarization film 204 made of, for example, silicon oxide (SiO2), silicon nitride (SiN), silicon oxynitride (SiON), or the like is provided below the on-chip lens 208.
  • an anti-reflection film 202 made of an insulating film is provided below the planarization film 204.
  • the anti-reflection film 202 can be formed of hafnium oxide (HfO2), aluminum oxide (Al2O3), titanium oxide (TiO2), silicon oxide, or the like, or a laminate of these materials.
  • a light blocking film 206 is provided above the anti-reflection film 202, in the boundary region with the adjacent light receiving element 10, to prevent reflected light from the object 800 from entering the adjacent light receiving element 10.
  • the light blocking film 206 is made of a material that blocks light, and can be formed using a metal material such as tungsten (W), aluminum (Al), or copper (Cu).
  • a pixel isolation portion 210a (DTI: Deep Trench Isolation) is provided, which penetrates from the back surface of the semiconductor substrate 200 to partway through the semiconductor substrate 200 along the thickness direction of the semiconductor substrate 200.
  • the pixel isolation portion 210a is composed of, for example, a trench provided in the semiconductor substrate 200 and an insulating film such as silicon oxide or a metal film such as aluminum embedded in the trench.
  • the pixel isolation portion 210a can prevent incident light from entering adjacent light receiving elements 10, and as a result, charge crosstalk between adjacent light receiving elements 10 can be prevented.
  • the distribution transistors VG1 and VG2 are formed on either side of the N-type semiconductor region 100b.
  • the distribution transistors VG1 and VG2 each have a gate electrode 150a and 150b, which is made of, for example, a polysilicon film, provided on the surface of the semiconductor substrate 200.
  • charge storage units MEM1 and MEM2 are provided in the semiconductor substrate 200 so as to sandwich the distribution transistors VG1 and VG2 from the left and right direction.
  • the charge storage units MEM1 and MEM2 can be MOS type capacitors consisting of a laminate of electrodes 154a and 154b made of a metal film or a polysilicon film, an insulating film (not shown) made of an oxide film, and an N-type semiconductor region (shown as MEM1 and MEM2 in FIG. 6).
  • the gate electrodes 156a and 156b of the transfer transistors TG1 and TG2 are provided on the surface of the semiconductor substrate 200.
  • N-type semiconductor regions illustrated as floating diffusion regions FD1 and FD2 are formed in the semiconductor substrate 200 adjacent to the gate electrodes 156a and 156b of the transfer transistors TG1 and TG2.
  • a wiring layer 300 is provided on the surface of the semiconductor substrate 200.
  • the wiring layer 300 includes an insulating film 302 and a metal film 304. Furthermore, an electrode 306 is provided on the surface of the wiring layer 300 opposite the semiconductor substrate 200.
  • a substrate 400 is provided on the surface of the wiring layer 300 opposite the semiconductor substrate 200.
  • the substrate 400 also includes an insulating film 402 and a metal film 404, and an electrode 406 is provided on the surface on the wiring layer 300 side.
  • the electrode 306 of the wiring layer 300 and the electrode 406 of the substrate 400 are formed of copper (Cu) or the like, and can be joined to each other by being in contact with each other.
  • the cross-sectional configuration of the light receiving element 10 is not limited to the example shown in FIG. 6, and may include other elements, for example, and is not particularly limited.
  • Fig. 7 is another equivalent circuit diagram of the light receiving element 10
  • Fig. 8 is a diagram showing an example of the planar configuration of the light receiving element 10 in the circuit configuration shown in Fig. 7.
  • the light receiving element 10 shown in FIG. 7 has a configuration in which the charge storage units MEM1 and MEM2 and the transfer transistors TG1 and TG2 are removed from the light receiving element 10 shown in FIG. 3.
  • the charge generated in the photodiode PD is directly distributed to the floating diffusion regions FD1 and FD2 by the distribution transistors VG1 and VG2.
  • the gate electrodes 150a, 150b of the distribution transistors VG1, VG2 are arranged to be line-symmetric (approximately line-symmetric) with respect to a center line (not shown). Therefore, by adopting a configuration in which the charge storage units MEM1, MEM2 and the transfer transistors TG1, TG2 are not provided in this manner, the light receiving element 10 can be made even finer.
  • Fig. 9 is an equivalent circuit diagram of a light receiving element 10
  • Fig. 10 is a diagram showing an example of a planar configuration of the light receiving element 10 having the circuit configuration shown in Fig. 9.
  • the light receiving element 10 has four combinations of a distribution transistor VG, a charge storage unit MEM, a transfer transistor TG, a floating diffusion region FD, a reset transistor RST, an amplification transistor AMP, and a selection transistor SEL.
  • a distribution transistor VG a charge storage unit MEM
  • a transfer transistor TG a transfer transistor TG
  • a floating diffusion region FD a floating diffusion region FD
  • a reset transistor RST an amplification transistor AMP
  • SEL selection transistor
  • the distribution transistors VG1, VG2, VG3, and VG4 of the light receiving element 10 are arranged so as to be linearly symmetrical (approximately linearly symmetrical) with respect to a center line that passes through the center point O of the photodiode PD and extends in the vertical direction (column direction) of the light receiving element 10.
  • Charge discharge transistors OFG are arranged between each of the distribution transistors VG1, VG2, VG3, and VG4.
  • the charge accumulation units MEM1 to MEM4 are arranged around the distribution transistors VG1, VG2, VG3, and VG4, with the photodiode PD at the center.
  • FIG. 11 is an equivalent circuit diagram of a light receiving element 10 having eight distribution transistors VG1 to VG8, and Figure 12 is a diagram showing an example of a planar configuration of the light receiving element 10 in the circuit configuration shown in Figure 11.
  • the light receiving element 10 has eight combinations of a distribution transistor VG, a charge storage unit MEM, a transfer transistor TG, and a floating diffusion region FD.
  • the light receiving element 10 has one reset transistor RST, one amplification transistor AMP, and one selection transistor SEL.
  • Such a light receiving element 10 can adopt a planar configuration as shown in FIG. 12.
  • the gate electrodes 150 of the distribution transistors VG1 to VG8 are arranged so as to be line-symmetric (approximately line-symmetric) with respect to a center line that passes through the center point O of the photodiode PD and extends in the vertical direction (column direction) of the light receiving element 10.
  • the charge storage units MEM1 to 8 are also arranged so as to be line-symmetric (approximately line-symmetric) with respect to a center line that passes through the center point O of the photodiode PD and extends in the vertical direction (column direction) of the light receiving element 10.
  • the light receiving element 10 described above has a configuration including two, four, or eight charge storage units MEM, and is structured so that charges are distributed to these charge storage units MEM. Such light receiving elements 10 are arranged in an array in the pixel array section 12.
  • pupil correction In order to eliminate any difference in sensitivity between the light receiving elements 10 at the center of the angle of view and those at the edges of the angle of view and to maintain a constant sensitivity, and to suppress deterioration in sensitivity of the light receiving elements 10 at the edges of the angle of view and maximize their sensitivity, for example, there is a technology known as pupil correction, which aligns the optical axis of the on-chip lens 208 with the opening of the photodiode PD at the center of the imaging surface (center of the angle of view), and shifts the position of the photodiode PD to match the direction of the chief ray as it moves toward the edges of the angle of view.
  • FIG. 13 shows an example of the cross-sectional configuration of a light receiving element 10 that has been subjected to pupil correction at the end of the angle of view.
  • FIG. 13 shows the on-chip lens 208, the planarization film 204, and the semiconductor substrate 200 (photodiode PD), and also shows the charge storage units MEM1 and MEM2 formed on the bottom surface of the semiconductor substrate 200 in the figure.
  • incident light is incident on the photodiode PD from an oblique direction, as shown by the arrow in FIG. 13.
  • the center of the on-chip lens 208 and the center of the photodiode PD are shifted by the amount of correction.
  • the center of the on-chip lens 208 is shifted by the amount of correction to the left of the center of the photodiode PD.
  • interface M1 The interface between the planarization film 204 and the semiconductor substrate 200 is referred to as interface M1.
  • Incident light that passes through the center of the on-chip lens 208 passes through the center P1 of the photodiode PD at interface M1.
  • the central axis P1 passing through the center P1 of the photodiode PD is shown by a dotted line. Also in FIG. 13, a distribution with a certain light intensity is shown by a dotted line to explain the spread of light within the photodiode PD.
  • region 231-1 The region to the left of the central axis P1 where the incident light is distributed at a predetermined light intensity or higher is designated as region 231-1, and the region to the right of the central axis P1 where the incident light is distributed at a predetermined light intensity or higher is designated as region 231-2.
  • region 231-2 the region to the left of the central axis P1 where the incident light is distributed at a predetermined light intensity or higher is designated as region 231-2.
  • the components that are photoelectrically converted in the diffusion layer of the charge storage unit MEM and the components that are photoelectrically converted in the surrounding wells and penetrate into the diffusion layer of the charge storage unit MEM by diffusion, so-called PLS, are larger in the charge storage unit MEM2 than in the charge storage unit MEM1.
  • a difference may occur between the PLS of the charge storage unit MEM1 and the PLS of the charge storage unit MEM2.
  • the distance measurement accuracy may decrease.
  • pupil correction is performed on the light incident surface side of the photodiode PD so that the chief ray is incident on the center P1 of the photodiode PD.
  • pupil correction is performed on the surface opposite the light incident surface side of the photodiode PD so that the chief ray is incident on position P2 of the photodiode PD.
  • the on-chip lens 208 is disposed at a position shifted further to the left in the figure than the on-chip lens 208 of the light receiving element 10 shown in FIG. 13.
  • the amount of pupil correction in the light receiving element 10 shown in FIG. 14 is greater than the amount of pupil correction in the light receiving element 10 shown in FIG. 13.
  • the amount of pupil correction becomes greater the light that passes through the center of the on-chip lens 208 is incident not on the center P1 on the incident light surface side of the photodiode PD, but on a position shifted to the left of center P1 in the example shown in FIG. 13.
  • position P2 is a position on a line where the distance from the position of the center of gravity of the charge storage unit MEM1 is the same as the distance from the position of the center of gravity of the charge storage unit MEM2.
  • regions 231-1 and 231-2 are both positioned shifted to the left in the figure.
  • the distance between region 231-1 and charge storage unit MEM1 and the distance between region 231-2 and charge storage unit MEM2 become approximately the same, making it possible to suppress differences in PLS.
  • FIG. 15 is a diagram showing an example of the planar configuration at the interface M2 between the semiconductor substrate 200 and the wiring layer 300 (not shown in FIGS. 6 and 14) in the light receiving element 10 of FIG. 14.
  • the charge storage unit MEM1 is disposed on the left side of the light receiving element 10
  • the charge storage unit MEM2 is disposed on the right side.
  • the center of gravity of the charge storage unit MEM1 is defined as center of gravity P21
  • the center of gravity of the charge storage unit MEM2 is defined as center of gravity P22.
  • the line where the distance L1 from the center of gravity P21 of the charge storage unit MEM1 and the distance L2 from the center of gravity P22 of the charge storage unit MEM2 are equal passes through the center P11 of the line connecting the centers of gravity P21 and P22, and is perpendicular to the line, which is the perpendicular bisector L3.
  • Point P2 ( Figure 14) is on this perpendicular bisector L3.
  • the perpendicular bisector L3 is a line passing through the center of the light receiving element 10.
  • the charge storage units MEM are positioned at equal distances from the center O of the light receiving element 10. Therefore, in the case of a light receiving element 10 having such a planar configuration, the perpendicular bisector L3 is a line passing through the center of the light receiving element 10.
  • the center of the light receiving element 10 can also be said to be the center of gravity of the light receiving element 10, so the perpendicular bisector L3 is a line that passes through the center of gravity of the light receiving element 10, and can also be said to be a line that is the center of gravity of the charge storage unit MEM1 and the charge storage unit MEM2.
  • the distance between region 231-1 and charge storage unit MEM1 and the distance between region 231-2 and charge storage unit MEM2 can be made approximately the same size.
  • Fig. 16 is a schematic diagram of a cross section of the light receiving element 10, and shows a state in which a semiconductor substrate 200, a planarizing film 204, and an on-chip lens 208 are laminated.
  • the thickness of the semiconductor substrate 200 is thickness D, and the combined thickness of the planarization film 204 and the on-chip lens 208 (the thickness of the upper layer structure of the semiconductor substrate 200) is thickness H.
  • X represents the amount of pupil correction, and represents the distance between the center of the on-chip lens 208 and the center of the semiconductor substrate 200 when viewed in a plan view.
  • the semiconductor substrate 200 is made of silicon (Si) will be described.
  • the angle of incidence of the incident light with respect to the on-chip lens 208 is defined as angle ⁇ air
  • the angle of refraction of the light in the upper layer structure of the semiconductor substrate 200 is defined as angle ⁇ SiO
  • angle ⁇ Si the angle of refraction of the light in the semiconductor substrate 200
  • nsi is the refractive index of silicon, and for example, the refractive index for infrared light (940 nm) is about 3.6. If the refractive index of the upper layer structure of the semiconductor substrate 200 is nSiO, then nSiO is about 1.45.
  • the pupil correction amount X is calculated by the following equation (4).
  • equation (4) differs in that it is equation (5) to which D ⁇ tan( ⁇ si) has been added.
  • the pupil correction amount is calculated taking into account the thickness of the semiconductor substrate 200.
  • the value calculated by equation (4) is a value equivalent to half the length P of one side of the light receiving element 10.
  • the on-chip lens 208 is positioned so that, in a plan view, the center of the on-chip lens 208 is separated from the center of the semiconductor substrate 200 by the correction amount X.
  • the amount of pupil correction found by such a calculation is gradually larger from the center of the angle of view to the edge of the angle of view, because the incident angle ⁇ air differs depending on the angle of view position.
  • pupil correction may be performed only in the X-axis direction or Y-axis direction in the pixel array section 12, or pupil correction may be performed taking into account both the X-axis direction and the Y-axis direction.
  • the present invention can also be applied to a light receiving element 10 having four or eight charge storage units MEM.
  • the position of the perpendicular bisector L3 or point P2 is set according to the arrangement of the multiple charge storage units MEM, and the correction amount is set so that the principal ray travels to the perpendicular bisector L3 or point P2.
  • a light receiving element 10 that does not include a charge storage unit MEM
  • a light receiving element 10 that has a structure in which the charge from the photodiode PD described with reference to Figures 7 and 8 is directly distributed to the floating diffusion region FD
  • the charge storage unit MEM described above can be read as the floating diffusion region FD and the correction amount can be set.
  • the pupil correction amount is calculated in this manner, and is applied to the on-chip lens 208 to set the center position of the on-chip lens 208.
  • a correction amount corresponding to the correction amount of the on-chip lens 208 is also applied to the light-shielding film 206 to set its position.
  • the light receiving element 10 shown in FIG. 6 is an example of a cross-sectional configuration of a light receiving element 10 located at the center of the angle of view.
  • the light shielding film 206 is disposed between the photodiodes PD to prevent the incident light from leaking into the adjacent photodiodes PD.
  • the light shielding film 206 is disposed on the pixel separation section 210a that separates the pixels. In other words, it is disposed between the on-chip lenses 208.
  • FIG. 17 is a simplified diagram of the cross-sectional configuration of the light receiving element 10 shown in FIG. 6, and shows an example of the cross-sectional configuration of the light receiving element 10 located at the end of the angle of view.
  • the light shielding film 206 is located between the on-chip lenses 208.
  • the pupil correction amount determined as described above is applied to the on-chip lenses 208, and the center of the on-chip lenses 208 is positioned at a position shifted from the center of the photodiode PD.
  • a pupil correction amount is also applied to the light-shielding film 206, and its position is shifted.
  • the light-shielding film 206 is disposed near the center of the interface M1 of the photodiode PD. This position corresponds to between the on-chip lens 208 and the adjacent on-chip lens 208.
  • the light-shielding film 206 is located between the charge storage units MEM1 and MEM2 in a plan view.
  • the light-shielding film 206 is located near the center of the pixel because the position is at the edge of the angle of view, where the amount of pupil correction is large.
  • the light-shielding film 206 may be located on the charge storage unit MEM1 or the charge storage unit MEM2.
  • pupil correction is also performed on the light shielding film 206, and it is positioned at an appropriate position.
  • Fig. 18 When the pupil correction amount at the edge of the angle of view is calculated in this manner and the pupil correction amount is applied, the effect shown in Fig. 18 can be obtained.
  • a of Fig. 18 when PLS is measured for a light-receiving element 10 designed so that, at interface M2 of the light-receiving element 10, charge accumulation unit MEM1 is on the left side in the figure, charge accumulation unit MEM2 is on the right side in the figure, and perpendicular bisector L3, which is the target position for the principal ray, is in the vertical direction at the center, the result shown in B of Fig. 18 can be obtained.
  • the vertical axis represents PLS and the horizontal axis represents the position within the angle of view of the light receiving element 10, with the left side of the figure corresponding to the left side of the angle of view and the right side of the figure corresponding to the right side within the angle of view.
  • the difference is small.
  • the difference is large, so it can be seen that by applying this technology, it is possible to suppress the occurrence of differences in the PLS of charge storage unit MEM1 and charge storage unit MEM2.
  • a in Figure 19 shows an example in which the charge storage units MEM1 and MEM2 are arranged in a vertical direction.
  • a in Figure 19 when PLS is measured in a light receiving element 10 designed so that the charge storage unit MEM1 is on the upper side of the figure and the charge storage unit MEM2 is on the lower side of the interface M2 of the light receiving element 10, and the perpendicular bisector L3, which is the target position for the principal ray, is in the horizontal direction in the center, the results shown in B and C in Figure 19 are obtained.
  • the vertical axis represents PLS and the horizontal axis represents the position within the angle of view of the light receiving element 10, with the left side of the figure corresponding to the left side of the angle of view and the right side of the figure corresponding to the right side within the angle of view.
  • the difference when comparing the PLS of the charge storage unit MEM1 and the charge storage unit MEM2 at the left angle of view, the difference is small.
  • the difference when comparing the PLS of the charge storage unit MEM1 and the charge storage unit MEM2 at the right angle of view, the difference is small, indicating that by applying this technology, it is possible to suppress the occurrence of differences in the PLS of the charge storage unit MEM1 and the charge storage unit MEM2.
  • the left side of the horizontal axis corresponds to the lower side of the angle of view, and the right side corresponds to the upper side within the angle of view.
  • the difference is small.
  • charge storage unit MEM1 and charge storage unit MEM2 are arranged in the vertical direction, so it can be seen that by applying this technology, it is possible to suppress the occurrence of differences in the PLS of charge storage unit MEM1 and charge storage unit MEM2.
  • FIG. 20 shows an example in which four charge storage units MEM1 to MEM4 are arranged at the four corners of the light receiving element 10.
  • Charge storage unit MEM1+charge storage unit MEM4 ⁇ charge storage unit MEM2+charge storage unit MEM3 This relationship is utilized, and by adjusting the addition, the difference in PLS can be offset.
  • the light receiving element 10 is configured to include a tap A and a tap B (Tap A and Tap B) as shown in Fig. 21.
  • the tap A is configured to include a charge storage unit MEM1
  • the tap B is configured to include a charge storage unit MEM2.
  • the light receiving element 10 is configured to include tap A and tap B, a detection signal with a phase of 0° is obtained at tap A, and a detection signal with a phase of 180° is obtained at tap B.
  • the in-phase component with respect to the modulated light wave is called I pixel data
  • the pixel that obtains I pixel data is called an I pixel.
  • the light receiving element 10 is configured to include tap A and tap B, a detection signal with a phase of 90° is obtained at tap A, and a detection signal with a phase of 270° is obtained at tap B.
  • the orthogonal component to the modulated light wave is called Q pixel data
  • the pixel that obtains the Q pixel data is called a Q pixel.
  • the distance measurement method using the light receiving element 10 with two taps shown in Figure 21 will be explained.
  • a method called 2 tap 4 phase will be explained as an example.
  • the method is called 2 tap 4 phase when measuring the distance to a specified object by using irradiation light with four phase differences with two taps, or receiving light with four phase differences.
  • first exposure is performed at tap A to obtain a phase signal with a phase of 0°, and then the phase signal with a phase of 0° is read out.
  • tap B exposure is performed at tap B to obtain a phase signal with a phase of 180°, and then the phase signal with a phase of 180° is read out.
  • the photodetector 10 functions as an I pixel, and the first accumulation and readout are performed to obtain the first frame.
  • the light receiving element 10 functions as a Q pixel, and the second accumulation and readout are performed to obtain the second frame.
  • the light receiving element 10 functions as a -I pixel, and the third accumulation and readout is performed to obtain the third frame.
  • the light receiving element 10 functions as a -Q pixel, and accumulation and readout are performed a fourth time to obtain the fourth frame.
  • the third and fourth frames acquired by the light receiving element 10 functioning as the -I pixel and -Q pixel are acquired to cancel the mismatch between the taps, that is, the difference in PLS between the charge storage unit MEM1 and the charge storage unit MEM2 described above.
  • one distance measurement image can be generated by the process shown in FIG. 23.
  • the light receiving element 10 having tap A and tap B exposure is first performed at tap A to obtain a phase signal with a phase of 0°, and then the phase signal with a phase of 0° is read out.
  • tap B exposure is performed at tap B to obtain a phase signal with a phase of 180°, and then the phase signal with a phase of 180° is read out.
  • the light receiving element 10 functions as an I pixel, and the first accumulation and readout are performed to obtain the first frame.
  • the light receiving element 10 functions as a Q pixel, and the second accumulation and readout are performed to obtain the second frame.
  • one ranging image can be generated. Because there is no need to acquire out-of-phase frames, it is possible to shorten the time required to generate one ranging image.
  • the light receiving element 10 can be applied to a distance measuring module as described above, and can also be applied to various electronic devices such as imaging devices such as digital still cameras and digital video cameras with distance measuring functions, and smartphones with distance measuring functions.
  • FIG. 24 is a block diagram showing an example of the configuration of a smartphone as an electronic device to which this technology is applied.
  • the smartphone 601 is configured by connecting a distance measurement module 602, an image capture device 603, a display 604, a speaker 605, a microphone 606, a communication module 607, a sensor unit 608, a touch panel 609, and a control unit 610 via a bus 611.
  • the control unit 610 has the functions of an application processing unit 621 and an operation system processing unit 622 by the CPU executing a program.
  • the distance measurement module 1 in FIG. 1 is applied to the distance measurement module 602.
  • the distance measurement module 602 is placed on the front side of the smartphone 601, and by performing distance measurement on the user of the smartphone 601, it is possible to output the depth value of the surface shape of the user's face, hands, fingers, etc. as the distance measurement result.
  • the imaging device 603 is disposed on the front side of the smartphone 601, and captures an image of the user of the smartphone 601 as a subject, thereby obtaining an image of the user. Although not shown, the imaging device 603 may also be disposed on the back side of the smartphone 601.
  • the display 604 displays an operation screen for performing processing by the application processing unit 621 and the operation system processing unit 622, images captured by the imaging device 603, and the like.
  • the speaker 605 and the microphone 606 output the voice of the other party and pick up the voice of the user, for example, when making a call using the smartphone 601.
  • the communication module 607 performs network communication via communication networks such as the Internet, public telephone line networks, wide area communication networks for wireless mobile devices such as so-called 4G and 5G lines, WANs (Wide Area Networks) and LANs (Local Area Networks), and short-range wireless communication such as Bluetooth (registered trademark) and NFC (Near Field Communication).
  • the sensor unit 608 senses speed, acceleration, proximity, etc., and the touch panel 609 acquires touch operations by the user on the operation screen displayed on the display 604.
  • the application processing unit 621 performs processing for providing various services through the smartphone 601. For example, the application processing unit 621 can perform processing for creating a computer graphics face that virtually reproduces the user's facial expression based on the depth value supplied from the distance measurement module 602, and displaying the face on the display 604. The application processing unit 621 can also perform processing for creating, for example, three-dimensional shape data of any three-dimensional object based on the depth value supplied from the distance measurement module 602.
  • the operation system processing unit 622 performs processing to realize the basic functions and operations of the smartphone 601. For example, the operation system processing unit 622 can perform processing to authenticate the user's face and unlock the smartphone 601 based on the depth value supplied from the distance measurement module 602. The operation system processing unit 622 can also perform processing to recognize, for example, a user's gesture based on the depth value supplied from the distance measurement module 602, and input various operations according to the gesture.
  • the smartphone 601 configured in this manner, by applying the above-mentioned distance measurement module 1 as the distance measurement module 602, it is possible to perform processes such as measuring and displaying the distance to a specified object, and creating and displaying three-dimensional shape data of the specified object.
  • the technology according to the present disclosure (the present technology) can be applied to various products.
  • the technology according to the present disclosure may be applied to an endoscopic surgery system.
  • FIG. 25 is a diagram showing an example of the general configuration of an endoscopic surgery system to which the technology disclosed herein (the present technology) can be applied.
  • an operator (doctor) 11131 is shown using an endoscopic surgery system 11000 to perform surgery on a patient 11132 on a patient bed 11133.
  • the endoscopic surgery system 11000 is composed of an endoscope 11100, other surgical tools 11110 such as an insufflation tube 11111 and an energy treatment tool 11112, a support arm device 11120 that supports the endoscope 11100, and a cart 11200 on which various devices for endoscopic surgery are mounted.
  • the endoscope 11100 is composed of a lens barrel 11101, the tip of which is inserted into the body cavity of the patient 11132 at a predetermined length, and a camera head 11102 connected to the base end of the lens barrel 11101.
  • the endoscope 11100 is configured as a so-called rigid scope having a rigid lens barrel 11101, but the endoscope 11100 may also be configured as a so-called flexible scope having a flexible lens barrel.
  • the tip of the tube 11101 has an opening into which an objective lens is fitted.
  • a light source device 11203 is connected to the endoscope 11100, and light generated by the light source device 11203 is guided to the tip of the tube by a light guide extending inside the tube 11101, and is irradiated via the objective lens towards an object to be observed inside the body cavity of the patient 11132.
  • the endoscope 11100 may be a direct-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.
  • An optical system and an image sensor are provided inside the camera head 11102, and reflected light (observation light) from the object being observed is focused onto the image sensor by the optical system.
  • the image sensor converts the observation light into an electric signal corresponding to the observation light, i.e., an image signal corresponding to the observed image.
  • the image signal is sent to the camera control unit (CCU: Camera Control Unit) 11201 as RAW data.
  • CCU Camera Control Unit
  • the CCU 11201 is composed of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., and controls the overall operation of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera head 11102, and performs various types of image processing on the image signal, such as development processing (demosaic processing), in order to display an image based on the image signal.
  • a CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • the display device 11202 under the control of the CCU 11201, displays an image based on the image signal that has been subjected to image processing by the CCU 11201.
  • the light source device 11203 is composed of a light source such as an LED (light emitting diode), and supplies illumination light to the endoscope 11100 when photographing the surgical site, etc.
  • a light source such as an LED (light emitting diode)
  • the input device 11204 is an input interface for the endoscopic surgery system 11000.
  • a user can input various information and instructions to the endoscopic surgery system 11000 via the input device 11204.
  • the user inputs an instruction to change the imaging conditions (type of irradiation light, magnification, focal length, etc.) of the endoscope 11100.
  • the treatment tool control device 11205 controls the operation of the energy treatment tool 11112 for cauterizing tissue, incising, sealing blood vessels, etc.
  • the insufflation device 11206 sends gas into the body cavity of the patient 11132 via the insufflation tube 11111 to inflate the body cavity in order to ensure a clear field of view for the endoscope 11100 and to ensure a working space for the surgeon.
  • the recorder 11207 is a device capable of recording various types of information related to the surgery.
  • the printer 11208 is a device capable of printing various types of information related to the surgery in various formats such as text, images, or graphs.
  • the light source device 11203 that supplies illumination light to the endoscope 11100 when photographing the surgical site can be composed of a white light source composed of, for example, an LED, a laser light source, or a combination of these.
  • a white light source composed of, for example, an LED, a laser light source, or a combination of these.
  • the white light source is composed of a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so that the white balance of the captured image can be adjusted in the light source device 11203.
  • the light source device 11203 may be controlled to change the intensity of the light it outputs at predetermined time intervals.
  • the image sensor of the camera head 11102 may be controlled to acquire images in a time-division manner in synchronization with the timing of the change in the light intensity, and the images may be synthesized to generate an image with a high dynamic range that is free of so-called blackout and whiteout.
  • the light source device 11203 may be configured to supply light of a predetermined wavelength band corresponding to special light observation.
  • special light observation for example, by utilizing the wavelength dependency of light absorption in body tissue, a narrow band of light is irradiated compared to the light irradiated during normal observation (i.e., white light), and a specific tissue such as blood vessels on the surface of the mucosa is photographed with high contrast, so-called narrow band imaging is performed.
  • fluorescence observation may be performed in which an image is obtained by fluorescence generated by irradiating excitation light.
  • excitation light is irradiated to body tissue and fluorescence from the body tissue is observed (autofluorescence observation), or a reagent such as indocyanine green (ICG) is locally injected into the body tissue and excitation light corresponding to the fluorescence wavelength of the reagent is irradiated to the body tissue to obtain a fluorescent image.
  • the light source device 11203 may be configured to supply narrow band light and/or excitation light corresponding to such special light observation.
  • FIG. 26 is a block diagram showing an example of the functional configuration of the camera head 11102 and CCU 11201 shown in FIG. 25.
  • the camera head 11102 has a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405.
  • the CCU 11201 has a communication unit 11411, an image processing unit 11412, and a control unit 11413.
  • the camera head 11102 and the CCU 11201 are connected to each other via a transmission cable 11400 so that they can communicate with each other.
  • the lens unit 11401 is an optical system provided at the connection with the lens barrel 11101. Observation light taken in from the tip of the lens barrel 11101 is guided to the camera head 11102 and enters the lens unit 11401.
  • the lens unit 11401 is composed of a combination of multiple lenses including a zoom lens and a focus lens.
  • the imaging unit 11402 may have one imaging element (a so-called single-plate type) or multiple imaging elements (a so-called multi-plate type).
  • each imaging element may generate an image signal corresponding to each of RGB, and a color image may be obtained by combining these.
  • the imaging unit 11402 may be configured to have a pair of imaging elements for acquiring image signals for the right eye and the left eye corresponding to a 3D (dimensional) display. By performing a 3D display, the surgeon 11131 can more accurately grasp the depth of the biological tissue in the surgical site.
  • multiple lens units 11401 may be provided corresponding to each imaging element.
  • the imaging unit 11402 does not necessarily have to be provided in the camera head 11102.
  • the imaging unit 11402 may be provided inside the lens barrel 11101, immediately after the objective lens.
  • the driving unit 11403 is composed of an actuator, and moves the zoom lens and focus lens of the lens unit 11401 a predetermined distance along the optical axis under the control of the camera head control unit 11405. This allows the magnification and focus of the image captured by the imaging unit 11402 to be adjusted appropriately.
  • the communication unit 11404 is configured with a communication device for transmitting and receiving various information to and from the CCU 11201.
  • the communication unit 11404 transmits the image signal obtained from the imaging unit 11402 as RAW data to the CCU 11201 via the transmission cable 11400.
  • the communication unit 11404 also receives control signals for controlling the operation of the camera head 11102 from the CCU 11201, and supplies them to the camera head control unit 11405.
  • the control signals include information on the imaging conditions, such as information specifying the frame rate of the captured image, information specifying the exposure value during imaging, and/or information specifying the magnification and focus of the captured image.
  • the imaging conditions such as the frame rate, exposure value, magnification, and focus may be appropriately specified by the user, or may be automatically set by the control unit 11413 of the CCU 11201 based on the acquired image signal.
  • the endoscope 11100 is equipped with the so-called AE (Auto Exposure) function, AF (Auto Focus) function, and AWB (Auto White Balance) function.
  • the camera head control unit 11405 controls the operation of the camera head 11102 based on a control signal from the CCU 11201 received via the communication unit 11404.
  • the communication unit 11411 is configured with a communication device for transmitting and receiving various information to and from the camera head 11102.
  • the communication unit 11411 receives an image signal transmitted from the camera head 11102 via the transmission cable 11400.
  • the communication unit 11411 also transmits to the camera head 11102 a control signal for controlling the operation of the camera head 11102.
  • the image signal and the control signal can be transmitted by electrical communication, optical communication, etc.
  • the image processing unit 11412 performs various image processing operations on the image signal, which is the RAW data transmitted from the camera head 11102.
  • the control unit 11413 performs various controls related to the imaging of the surgical site, etc. by the endoscope 11100, and the display of the captured images obtained by imaging the surgical site, etc. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102.
  • the control unit 11413 also causes the display device 11202 to display the captured image showing the surgical site, etc., based on the image signal that has been image-processed by the image processing unit 11412. At this time, the control unit 11413 may recognize various objects in the captured image using various image recognition techniques. For example, the control unit 11413 can recognize surgical tools such as forceps, specific body parts, bleeding, mist generated when the energy treatment tool 11112 is used, etc., by detecting the shape and color of the edges of objects included in the captured image. When the control unit 11413 causes the display device 11202 to display the captured image, it may use the recognition result to superimpose various types of surgical support information on the image of the surgical site. By superimposing the surgical support information and presenting it to the surgeon 11131, the burden on the surgeon 11131 can be reduced and the surgeon 11131 can proceed with the surgery reliably.
  • various image recognition techniques such as forceps, specific body parts, bleeding, mist generated when the energy treatment tool 11112 is used, etc.
  • the transmission cable 11400 that connects the camera head 11102 and the CCU 11201 is an electrical signal cable that supports electrical signal communication, an optical fiber that supports optical communication, or a composite cable of these.
  • communication is performed wired using a transmission cable 11400, but communication between the camera head 11102 and the CCU 11201 may also be performed wirelessly.
  • the technology according to the present disclosure can be applied to various products.
  • the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as an automobile, an electric vehicle, a hybrid electric vehicle, a motorcycle, a bicycle, a personal mobility device, an airplane, a drone, a ship, or a robot.
  • FIG. 27 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology disclosed herein can be applied.
  • the vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001.
  • the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside vehicle information detection unit 12030, an inside vehicle information detection unit 12040, and an integrated control unit 12050.
  • Also shown as functional components of the integrated control unit 12050 are a microcomputer 12051, an audio/video output unit 12052, and an in-vehicle network I/F (Interface) 12053.
  • the drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs.
  • the drive system control unit 12010 functions as a control device for a drive force generating device for generating the drive force of the vehicle, such as an internal combustion engine or a drive motor, a drive force transmission mechanism for transmitting the drive force to the wheels, a steering mechanism for adjusting the steering angle of the vehicle, and a braking device for generating a braking force for the vehicle.
  • the body system control unit 12020 controls the operation of various devices installed in the vehicle body according to various programs.
  • the body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, a power window device, or various lamps such as headlamps, tail lamps, brake lamps, turn signals, and fog lamps.
  • radio waves or signals from various switches transmitted from a portable device that replaces a key can be input to the body system control unit 12020.
  • the body system control unit 12020 accepts the input of these radio waves or signals and controls the vehicle's door lock device, power window device, lamps, etc.
  • the outside-vehicle information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000.
  • the image capturing unit 12031 is connected to the outside-vehicle information detection unit 12030.
  • the outside-vehicle information detection unit 12030 causes the image capturing unit 12031 to capture images outside the vehicle, and receives the captured images.
  • the outside-vehicle information detection unit 12030 may perform object detection processing or distance detection processing for people, cars, obstacles, signs, or characters on the road surface, based on the received images.
  • the imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of light received.
  • the imaging unit 12031 can output the electrical signal as an image, or as distance measurement information.
  • the light received by the imaging unit 12031 may be visible light, or may be invisible light such as infrared light.
  • the in-vehicle information detection unit 12040 detects information inside the vehicle.
  • a driver state detection unit 12041 that detects the state of the driver is connected.
  • the driver state detection unit 12041 includes, for example, a camera that captures an image of the driver, and the in-vehicle information detection unit 12040 may calculate the driver's degree of fatigue or concentration based on the detection information input from the driver state detection unit 12041, or may determine whether the driver is dozing off.
  • the microcomputer 12051 can calculate control target values for the driving force generating device, steering mechanism, or braking device based on information inside and outside the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040, and output control commands to the drive system control unit 12010.
  • the microcomputer 12051 can perform cooperative control aimed at realizing the functions of an Advanced Driver Assistance System (ADAS), including vehicle collision avoidance or impact mitigation, following driving based on the distance between vehicles, maintaining vehicle speed, vehicle collision warning, or vehicle lane departure warning.
  • ADAS Advanced Driver Assistance System
  • the microcomputer 12051 can also perform cooperative control for the purpose of autonomous driving, which allows the vehicle to travel autonomously without relying on the driver's operation, by controlling the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the outside vehicle information detection unit 12030 or the inside vehicle information detection unit 12040.
  • the microcomputer 12051 can also output control commands to the body system control unit 12030 based on information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 can control the headlamps according to the position of a preceding vehicle or an oncoming vehicle detected by the outside information detection unit 12030, and perform cooperative control aimed at preventing glare, such as switching from high beams to low beams.
  • the audio/image output unit 12052 transmits at least one output signal of audio and image to an output device capable of visually or audibly notifying the occupants of the vehicle or the outside of the vehicle of information.
  • an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are exemplified as output devices.
  • the display unit 12062 may include, for example, at least one of an on-board display and a head-up display.
  • FIG. 28 shows an example of the installation position of the imaging unit 12031.
  • the imaging unit 12031 includes imaging units 12101, 12102, 12103, 12104, and 12105.
  • the imaging units 12101, 12102, 12103, 12104, and 12105 are provided, for example, at the front nose, side mirrors, rear bumper, back door, and upper part of the windshield inside the vehicle cabin of the vehicle 12100.
  • the imaging unit 12101 provided at the front nose and the imaging unit 12105 provided at the upper part of the windshield inside the vehicle cabin mainly acquire images of the front of the vehicle 12100.
  • the imaging units 12102 and 12103 provided at the side mirrors mainly acquire images of the sides of the vehicle 12100.
  • the imaging unit 12104 provided at the rear bumper or back door mainly acquires images of the rear of the vehicle 12100.
  • the imaging unit 12105 provided at the upper part of the windshield inside the vehicle cabin is mainly used to detect leading vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, etc.
  • FIG. 28 shows an example of the imaging ranges of the imaging units 12101 to 12104.
  • Imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose
  • imaging ranges 12112 and 12113 indicate the imaging ranges of the imaging units 12102 and 12103 provided on the side mirrors, respectively
  • imaging range 12114 indicates the imaging range of the imaging unit 12104 provided on the rear bumper or back door.
  • an overhead image of the vehicle 12100 viewed from above is obtained by superimposing the image data captured by the imaging units 12101 to 12104.
  • At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information.
  • at least one of the imaging units 12101 to 12104 may be a stereo camera consisting of multiple imaging elements, or an imaging element having pixels for phase difference detection.
  • the microcomputer 12051 can obtain the distance to each solid object within the imaging ranges 12111 to 12114 and the change in this distance over time (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104, and can extract as a preceding vehicle, in particular, the closest solid object on the path of the vehicle 12100 that is traveling in approximately the same direction as the vehicle 12100 at a predetermined speed (e.g., 0 km/h or faster). Furthermore, the microcomputer 12051 can set the inter-vehicle distance that should be maintained in advance in front of the preceding vehicle, and perform automatic braking control (including follow-up stop control) and automatic acceleration control (including follow-up start control). In this way, cooperative control can be performed for the purpose of automatic driving, which runs autonomously without relying on the driver's operation.
  • automatic braking control including follow-up stop control
  • automatic acceleration control including follow-up start control
  • the microcomputer 12051 classifies and extracts three-dimensional object data on three-dimensional objects, such as two-wheeled vehicles, ordinary vehicles, large vehicles, pedestrians, utility poles, and other three-dimensional objects, based on the distance information obtained from the imaging units 12101 to 12104, and can use the data to automatically avoid obstacles.
  • the microcomputer 12051 distinguishes obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see.
  • the microcomputer 12051 determines the collision risk, which indicates the risk of collision with each obstacle, and when the collision risk is equal to or exceeds a set value and there is a possibility of a collision, it can provide driving assistance for collision avoidance by outputting an alarm to the driver via the audio speaker 12061 or the display unit 12062, or by forcibly decelerating or steering to avoid a collision via the drive system control unit 12010.
  • At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays.
  • the microcomputer 12051 can recognize a pedestrian by determining whether or not a pedestrian is present in the captured image of the imaging units 12101 to 12104. The recognition of such a pedestrian is performed, for example, by a procedure of extracting feature points in the captured image of the imaging units 12101 to 12104 as infrared cameras, and a procedure of performing pattern matching processing on a series of feature points that indicate the contour of an object to determine whether or not it is a pedestrian.
  • the audio/image output unit 12052 controls the display unit 12062 to superimpose a rectangular contour line for emphasis on the recognized pedestrian.
  • the audio/image output unit 12052 may also control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.
  • a system refers to an entire device that is made up of multiple devices.
  • the present technology can also be configured as follows.
  • a photoelectric conversion unit that performs photoelectric conversion; a first charge storage unit and a second charge storage unit that store the charge obtained by the photoelectric conversion unit; an on-chip lens that focuses incident light on the photoelectric conversion unit, a light-receiving element in which the center of the on-chip lens and the center of the photoelectric conversion unit are separated by a correction amount that is set so that light incident on the center of the on-chip lens is directed to a position whose distance from the center of gravity of the first charge accumulation unit is equal to that of the center of gravity of the second charge accumulation unit.
  • a light-shielding film is further provided between the on-chip lens and the photoelectric conversion unit, The light receiving element according to any one of (1) to (4), wherein the light shielding film is located between the first charge accumulation portion and the second charge accumulation portion in a plan view.
  • a light emitting unit that emits irradiation light; a light receiving element that receives light reflected by an object from the light emitting unit, The light receiving element is A photoelectric conversion unit that performs photoelectric conversion; a first charge storage unit and a second charge storage unit that store the charge obtained by the photoelectric conversion unit; an on-chip lens that focuses incident light on the photoelectric conversion unit, The center of the on-chip lens and the center of the photoelectric conversion unit are separated by a correction amount that is set so that light incident on the center of the on-chip lens is directed to a position whose distance from the center of gravity of the first charge accumulation unit is equal to that of the center of gravity of the second charge accumulation unit.
  • Range finding module 1 Range finding module, 2 Correlation, 4 2 taps, 10 Light receiving element, 12 Pixel array section, 20 Irradiation section, 30 Light receiving section, 32 Vertical drive circuit section, 34 Column signal processing circuit section, 36 Horizontal drive circuit section, 38 Output circuit section, 40 Control section, 42 Pixel drive wiring, 44 Control circuit section, 46 Horizontal signal line, 48 Vertical signal line, 60 Processing section, 100 N-type semiconductor region, 150 Gate electrode, 200 Semiconductor substrate, 202 Anti-reflection film, 204 Planarization film, 206 Light shielding film, 208 On-chip lens 300 wiring layer, 302 insulating film, 304 metal film, 306 electrode, 400 substrate, 402 insulating film, 404 metal film, 406 electrode, 601 smartphone, 602 distance measurement module, 603 imaging device, 604 display, 605 speaker, 606 microphone, 607 communication module, 608 sensor unit, 609 touch panel, 610 control unit, 611 bus, 621 application processing unit, 622 operation system processing unit, 800 object

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Electromagnetism (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Solid State Image Pick-Up Elements (AREA)
PCT/JP2024/008545 2023-03-30 2024-03-06 受光素子、測距装置 Ceased WO2024203081A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2025510150A JPWO2024203081A1 (https=) 2023-03-30 2024-03-06
KR1020257031583A KR20250168217A (ko) 2023-03-30 2024-03-06 수광 소자 및 측거 장치

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023055901 2023-03-30
JP2023-055901 2023-03-30

Publications (1)

Publication Number Publication Date
WO2024203081A1 true WO2024203081A1 (ja) 2024-10-03

Family

ID=92905514

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/008545 Ceased WO2024203081A1 (ja) 2023-03-30 2024-03-06 受光素子、測距装置

Country Status (3)

Country Link
JP (1) JPWO2024203081A1 (https=)
KR (1) KR20250168217A (https=)
WO (1) WO2024203081A1 (https=)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006040963A1 (ja) * 2004-10-15 2006-04-20 Konica Minolta Holdings, Inc. 固体撮像装置
JP2020113573A (ja) * 2019-01-08 2020-07-27 キヤノン株式会社 光電変換装置
WO2020241151A1 (ja) * 2019-05-24 2020-12-03 ソニーセミコンダクタソリューションズ株式会社 固体撮像装置及び測距装置
WO2021100826A1 (ja) * 2019-11-19 2021-05-27 ソニーセミコンダクタソリューションズ株式会社 受光素子、測距モジュール
WO2021125116A1 (ja) * 2019-12-18 2021-06-24 ソニーセミコンダクタソリューションズ株式会社 受光装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3114379C (en) 2011-12-06 2023-06-27 Dolby Laboratories Licensing Corporation Device and method of improving the perceptual luminance nonlinearity - based image data exchange across different display capabilities

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006040963A1 (ja) * 2004-10-15 2006-04-20 Konica Minolta Holdings, Inc. 固体撮像装置
JP2020113573A (ja) * 2019-01-08 2020-07-27 キヤノン株式会社 光電変換装置
WO2020241151A1 (ja) * 2019-05-24 2020-12-03 ソニーセミコンダクタソリューションズ株式会社 固体撮像装置及び測距装置
WO2021100826A1 (ja) * 2019-11-19 2021-05-27 ソニーセミコンダクタソリューションズ株式会社 受光素子、測距モジュール
WO2021125116A1 (ja) * 2019-12-18 2021-06-24 ソニーセミコンダクタソリューションズ株式会社 受光装置

Also Published As

Publication number Publication date
KR20250168217A (ko) 2025-12-02
JPWO2024203081A1 (https=) 2024-10-03

Similar Documents

Publication Publication Date Title
JP7586709B2 (ja) 固体撮像装置および電子機器
JP7620005B2 (ja) 撮像装置及び電子機器
TWI857044B (zh) 成像元件及距離測量裝置
WO2021235101A1 (ja) 固体撮像装置
WO2022009627A1 (ja) 固体撮像装置および電子機器
JP7696830B2 (ja) 受光素子及び受光装置
JP7613865B2 (ja) 受光素子及び受光装置
JP7705805B2 (ja) 撮像装置および電子機器
KR20230157329A (ko) 촬상 소자 및 촬상 장치
WO2024095832A1 (en) Photodetector, electronic apparatus, and optical element
KR102924715B1 (ko) 수광 소자 및 수광 장치
WO2019176302A1 (ja) 撮像素子および撮像素子の製造方法
WO2024203081A1 (ja) 受光素子、測距装置
WO2022158170A1 (ja) 光検出素子および電子機器
TWI889735B (zh) 受光元件及受光裝置
EP4694177A1 (en) Light detection device and electronic apparatus
US20260075976A1 (en) Photodetection device and electronic apparatus
US20250081643A1 (en) Solid-state imaging device and electronic device
EP4694174A1 (en) Optical detection device and electronic apparatus
EP4694173A1 (en) Photodetector and electronic apparatus
WO2024057805A1 (ja) 撮像素子および電子機器
KR20260042558A (ko) 이미징 디바이스
WO2024166667A1 (ja) 光検出装置及び電子機器
TW202523158A (zh) 光檢測裝置
WO2024262370A1 (ja) 光検出装置及び電子機器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24779216

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025510150

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025510150

Country of ref document: JP

ENP Entry into the national phase

Ref document number: 1020257031583

Country of ref document: KR

Free format text: ST27 STATUS EVENT CODE: A-0-1-A10-A15-NAP-PA0105 (AS PROVIDED BY THE NATIONAL OFFICE)

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24779216

Country of ref document: EP

Kind code of ref document: A1