WO2022016412A1 - 深度信息图像采集装置和电子设备 - Google Patents

深度信息图像采集装置和电子设备 Download PDF

Info

Publication number
WO2022016412A1
WO2022016412A1 PCT/CN2020/103458 CN2020103458W WO2022016412A1 WO 2022016412 A1 WO2022016412 A1 WO 2022016412A1 CN 2020103458 W CN2020103458 W CN 2020103458W WO 2022016412 A1 WO2022016412 A1 WO 2022016412A1
Authority
WO
WIPO (PCT)
Prior art keywords
phase modulation
pixel
array
angle
pixel array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/103458
Other languages
English (en)
French (fr)
Inventor
袁晓龙
姚国峰
沈健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Goodix Technology Co Ltd
Original Assignee
Shenzhen Goodix Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Goodix Technology Co Ltd filed Critical Shenzhen Goodix Technology Co Ltd
Priority to CN202080034583.3A priority Critical patent/CN114365019B/zh
Priority to PCT/CN2020/103458 priority patent/WO2022016412A1/zh
Publication of WO2022016412A1 publication Critical patent/WO2022016412A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B5/00—Optical elements other than lenses
    • G02B5/20—Filters

Definitions

  • the embodiments of the present application relate to the field of computers, and more particularly, to a depth information image acquisition apparatus and electronic equipment.
  • capturing images with depth information is increasingly used in smart connected devices and smart wearable devices, such as smartphones, virtual reality (VR) or augmented reality (AR) devices.
  • VR virtual reality
  • AR augmented reality
  • the current depth information measurement solutions have many problems, such as the need for additional active light sources, sensitivity to environmental interference, and complex calculations, which hinder the large-scale commercial use of depth measurement devices.
  • the embodiments of the present application provide a depth information image acquisition device and electronic equipment, which can reduce the power consumption of the equipment and help to improve the effect of outdoor use.
  • a depth information image acquisition device comprising: a plurality of two-dimensional optical phase modulation structures, each of the two-dimensional optical phase modulation structures in the plurality of two-dimensional optical phase modulation structures includes a substrate and a a boss on the substrate, each of the two-dimensional optical phase modulation structures is used to perform spatial phase modulation on the optical signal reflected by the object; an optical sensor is arranged below the plurality of two-dimensional optical phase modulation structures, so The optical sensor includes a plurality of pixel point arrays, the plurality of pixel point arrays have a one-to-one correspondence with the plurality of two-dimensional optical phase modulation structures, and one pixel point array in the plurality of pixel point arrays is used for receiving The optical signal modulated by the corresponding two-dimensional optical phase modulation structure, the one pixel point array includes a plurality of pixel points, and the intensity distribution of the optical signal received by the plurality of pixel points is used to determine the incident incident to the corresponding optical signal.
  • the present application can perform depth information detection without an additional light source, which can reduce the power consumption of the device; and this detection method is less disturbed by the environment, which is beneficial to improve the effect of outdoor use.
  • the present application can directly introduce a layer of two-dimensional optical phase modulation structure on the structure of the traditional image detection chip, the structure is simple, and the introduced two-dimensional optical phase modulation structure does not require complicated optical design like a traditional full-field camera.
  • the cross section of the boss is a center-symmetrical figure.
  • the centrally symmetric boss shape introduces a relatively small signal error in the angle recognition process, which is beneficial to improve the accuracy of the detection angle.
  • the boss is a cylindrical structure.
  • the center of the boss coincides with the center of the corresponding pixel array.
  • the center of the boss has an offset relative to the center of the corresponding pixel array, so that the incident angle determined by the corresponding pixel array is the actual incident angle of the light signal - the main angle of incidence of light.
  • the unshifted boss is used to directly detect the absolute value of the actual incident angle of the incident light in the main coordinate system of the optical system, and the shifted boss is used to detect the actual incident light relative to the imaging optical system.
  • the relative value which is beneficial to reduce the computational complexity, can be directly used to judge under-focus or over-focus.
  • the ratio between the lateral size of the boss and the size of one pixel is between 0.5 and 1.5.
  • the height of the boss is between ⁇ /4 ⁇ 5 ⁇ , where ⁇ is the wavelength of the light signal incident on the boss in vacuum.
  • the distance H between the plurality of two-dimensional optical phase modulation structures and the optical sensor is less than 10 ⁇ m.
  • the distance H less than 10 ⁇ m can ensure that the detection angle is monotonous within the angle detection range, which is beneficial to improve the accuracy of the detection angle.
  • the substrate in a two-dimensional optical phase modulation structure overlaps with the projection of its corresponding pixel array in a direction perpendicular to the surface of the substrate.
  • the one pixel point array is an N ⁇ N pixel point array, and N is a positive integer greater than or equal to 2.
  • the one pixel point array is a 3 ⁇ 3 pixel point array.
  • the 3 ⁇ 3 pixel point array has high angle detection range and detection accuracy.
  • the one pixel array is a 2 ⁇ 2 pixel array.
  • the 2 ⁇ 2 pixel point array makes the entire phase modulation structural unit use less total pixels, and more of the phase modulation structural unit can be placed on the imaging optical sensor of the same area for participating in angle detection, which can improve the image resolution with depth information. Rate.
  • the sum of the light signals received by the one pixel array is used to generate the image of the object.
  • the optical sensor is a back-illuminated optical sensor.
  • the optical signal does not have to pass through the metal wiring layer of the pixel array circuit before reaching the pixel array, which can reduce the interference and crosstalk problems of the optical signal.
  • the geometric dimensions of the bosses in the plurality of two-dimensional optical phase modulation structures are all the same.
  • the plurality of two-dimensional optical phase modulation structures include bosses with at least two geometric sizes.
  • a filter structure is further included, the filter structure is disposed between the plurality of two-dimensional optical phase modulation structures and the optical sensor, and the filter structure is used to detect the object.
  • the reflected light signal is filtered.
  • Adding a filter structure can reduce the spectral width of the incident light, and a narrower spectrum is beneficial to improve the accuracy of angle detection.
  • a two-dimensional optical phase modulation structure corresponds to a color filter.
  • the filter structure includes a mosaic filter.
  • Mosaic filters can be used to generate color images of objects.
  • the filter structure includes a Bayer filter array.
  • the filter structure includes filters of different colors, and the geometric dimensions of the bosses of the two-dimensional optical phase modulation structure corresponding to the filters of different colors are different, so that different two-dimensional optical phase modulation The structures can detect the same range of incident angles of optical signals.
  • a microlens array is also included, the microlens array is arranged between the plurality of two-dimensional optical phase modulation structures and the optical sensor, and the microlenses in the microlens array are connected to the optical sensor.
  • the pixels in the optical sensor have a one-to-one correspondence, and the microlens array is used for converging the optical signals modulated by the plurality of two-dimensional optical phase modulation structures to the optical sensor.
  • the incident angle includes an azimuth angle and an inclination angle
  • the inclination angle represents an angle between the optical signal and a direction perpendicular to a plane where the plurality of two-dimensional optical phase modulation structures are located
  • the azimuth angle The angle represents the included angle between the projection of the optical signal on the plane where the plurality of two-dimensional optical phase modulation structures are located and the coordinate axis on the plane.
  • an electronic device including: a depth information image acquisition apparatus according to the first aspect or any possible implementation manner of the first aspect.
  • it further includes a processor connected to the depth information image acquisition device, and the processor is configured to determine, according to the intensity of the light signal received by the pixel points in the one pixel point array, incident to the depth information image acquisition device.
  • the incident angle of the optical signal of the corresponding two-dimensional optical phase modulation structure, the incident angle includes an azimuth angle and an inclination angle.
  • the processor is configured to calculate the azimuth according to the following formula
  • X G represents the barycentric coordinate on the X axis of the intensity of the optical signal received by the pixel in the X direction
  • Y G represents the barycentric coordinate on the Y axis of the intensity of the optical signal received by the pixel in the Y direction.
  • the directions are two directions perpendicular to each other on the plane where the two-dimensional optical phase modulation structure is located.
  • the one pixel array is a 3 ⁇ 3 pixel array
  • the processor is configured to calculate the barycentric coordinates (X G , Y G ) by the following formula:
  • X i,j represents the coordinates of the pixel point in the ith row and the jth column in the X direction
  • Y i,j represents the coordinate in the Y direction of the pixel point in the ith row and the jth column
  • P i,j represents the th The intensity of the optical signal received by the pixel point in row i and column j, where i and j are both positive integers.
  • the one pixel array is a 2 ⁇ 2 pixel array
  • the processor is configured to calculate the barycentric coordinates (X G , Y G ) by the following formula:
  • X i,j represents the coordinates of the pixel point in the ith row and the jth column in the X direction
  • Y i,j represents the coordinate in the Y direction of the pixel point in the ith row and the jth column
  • P i,j represents the th The intensity of the optical signal received by the pixel point in row i and column j, where i and j are both positive integers.
  • the center coordinates determined by the above formula are relatively accurate.
  • the one pixel array is a 3 ⁇ 3 pixel array
  • the processor is configured to calculate the azimuth angle according to the following formula
  • G y represents the intensity gradient in the Y direction obtained according to the Sobel operator
  • G x represents the intensity gradient in the X direction obtained according to the Sobel operator.
  • the processor is configured to determine the tilt angle ⁇ according to a pre-established correspondence between ⁇ and f( ⁇ ), and the processor is configured to determine f( ⁇ ) according to the following formula:
  • ⁇ P x represents the normalized power difference in the X direction
  • ⁇ P y represents the normalized power difference in the Y direction.
  • the one pixel array is a 3 ⁇ 3 pixel array
  • the processor is configured to determine the normalized power difference between the pixels in the X direction and the Y direction by the following formula:
  • P i,j represents the intensity of the light signal received by the pixel point in the i-th row and the j-th column.
  • the normalized power difference determined by the above formula is relatively accurate.
  • the one pixel array is a 2 ⁇ 2 pixel array
  • the processor is configured to determine the normalized power difference between the pixels in the X direction and the Y direction by the following formula:
  • P i,j represents the intensity of the light signal received by the pixel point in the i-th row and the j-th column.
  • the normalized power difference determined by the above formula is relatively accurate.
  • FIG. 1 is a schematic partial structure diagram of a depth information image acquisition device provided by an embodiment of the present application.
  • FIG. 2 is a schematic view of the shape of several embodiments of the bosses in the device shown in FIG. 1 .
  • FIG. 3 is a top view of a depth information image acquisition device provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of another partial structure of a depth information image acquisition device provided by an embodiment of the present application.
  • 5-7 are distribution diagrams of a pixel point array provided by an embodiment of the present application.
  • FIG. 8 is a schematic diagram of another partial structure of a depth information image acquisition device provided by an embodiment of the present application.
  • FIG. 9 and FIG. 10 are schematic diagrams of the azimuth angle and the tilt angle of the optical signal incident on the optical phase modulation structure provided by the embodiment of the present application.
  • FIG. 11 is a schematic diagram of the X direction and the Y direction in the 2 ⁇ 2 pixel point array provided by the embodiment of the present application.
  • FIG. 12 is a simulation curve diagram of f( ⁇ ) provided by an embodiment of the present application.
  • FIG. 13 is a schematic flowchart of a method for determining depth information of an object image provided by an embodiment of the present application.
  • FIG. 14 is a schematic block diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 15 is a schematic structural diagram of a lens imaging process provided by an embodiment of the present application.
  • 3D imaging technology is now increasingly used in mobile smart connected devices and smart wearable devices, such as smartphones, VR devices or AR devices.
  • Detection methods that utilize a single detector (without the need for an artificial light source) also include traditional light-field cameras, which require the computation of depth information through comparative analysis of multiple simultaneously acquired images, the use of sophisticated machine learning or image reconstruction techniques, or Additional signal detection devices consisting of active illuminators and sensors are also required to recover depth information.
  • the embodiments of the present application provide a depth information image acquisition device, which does not need to introduce an additional active optical signal transmission device and an additional signal detection device, and can reduce the influence of environmental interference and equipment power consumption, especially suitable for mobile smart device.
  • the present technology may be used in imaging applications that require or may benefit from enhanced image depth detection and other three-dimensional imaging capabilities. For example to allow the user to change the focus, viewpoint and/or depth of field of the captured scene image.
  • the present techniques may be applied or implemented in various types of three-dimensional imaging systems and methods, including but not limited to light field imaging applications using plenoptic descriptions, ranging applications by comparative analysis of chromaticity dependence of diffraction, and single Sensor single image depth acquisition application.
  • the embodiments of the present application may be applied to a camera device of an electronic device, for example, a front/rear camera of a smartphone, a camera device of a security probe, and the like.
  • the depth information image acquisition device 100 may include a plurality of two-dimensional phase modulation structures and optical sensors.
  • a plurality of two-dimensional optical phase modulation structures are used for spatial phase modulation of the optical signal reflected by the object.
  • the optical sensor can be arranged under a plurality of two-dimensional phase modulation structures, the optical sensor includes a plurality of pixel point arrays, and the plurality of pixel point arrays have a one-to-one correspondence with the plurality of two-dimensional optical phase modulation structures, that is, one pixel
  • the dot array corresponds to a two-dimensional optical phase modulation structure.
  • One pixel array 120 of the plurality of pixel arrays is configured to receive an optical signal modulated by a corresponding two-dimensional optical phase modulation structure 110, and the intensity of the optical signal received by the pixels in the one pixel array 120 It is used to determine the incident angle of the light signal incident on the corresponding two-dimensional optical phase modulation structure 110 , and the incident angle is used to determine the depth information of the image of the imaging object.
  • the two-dimensional optical phase modulation structure means that the optical signal after passing through the two-dimensional phase modulation structure will be distributed in the two-dimensional space.
  • Spatial phase modulation can be understood as an optical path difference generated when an optical signal passes through different refractive index regions of a two-dimensional optical phase modulation structure, thereby achieving phase modulation of the optical signal.
  • the intensity of the optical signal can also be understood as the power of the optical signal.
  • FIG. 1 shows a schematic diagram of a two-dimensional optical phase modulation structure 110 and a pixel array 120 .
  • the apparatus 100 in this embodiment of the present application may include multiple structures shown in FIG. 1 .
  • the core of the two-dimensional optical phase modulation structure is a two-dimensional refractive index modulation pattern (as shown in Figure 1 , the boss structure shown in Figure 1). Affects the energy distribution of incident light on different pixel points of the pixel point array. Taking Fig. 1 as an example, due to the different refractive indices of the boss and the air, the light passing through the boss and the air layer around the boss will be refracted differently, resulting in an optical path difference.
  • the present application utilizes the Taber effect in wave optics, and the periodic structure of multiple two-dimensional optical phase modulation structures introduces periodic modulation of incident light.
  • the modulated incident light undergoes self-coherence, which produces an "image copy" of the above-mentioned periodic structure at the position where the coherence is enhanced.
  • the present application uses the relationship between the spatial distribution of the light field and the angle of incidence to detect the angle of incidence.
  • the depth information image acquisition in the embodiment of the present application does not require changing the structure of the original optical sensor, and only needs to add a layer of two-dimensional optical phase modulation structure on top of the original optical sensor after the fabrication is completed.
  • the energy distribution of the light signal with different incident angles on the pixel points is different, and the embodiment of the present application can determine the angle information of the incident light according to the energy distribution.
  • the embodiment of the present application can perform depth information detection without an additional light source, which can reduce the power consumption of the device; and this detection method is less disturbed by the environment, which is conducive to improving the effect of outdoor use.
  • the present application can directly introduce a layer of two-dimensional optical phase modulation structure on the structure of the traditional image detection chip, the structure is simple, and the introduced two-dimensional optical phase modulation structure does not require complicated optical design like a traditional full-field camera.
  • the present application can use a single image obtained by a single detector to perform image depth information detection and image three-dimensional reconstruction, and use a passive sensing mode with less power to perform its functions.
  • the device of the embodiment of the present application can be implemented without an active light source, the device can still be compatible with the active light source.
  • the structure of the device is simple and low-cost to integrate into existing image sensor hardware and manufacturing processes; it is comparable to conventional complementary metal oxide semiconductor (CMOS) and charged coupled device (CCD) image sensors. Compatible; and eliminates the need for multiple components, such as being equipped with an active lighting system for depth detection.
  • CMOS complementary metal oxide semiconductor
  • CCD charged coupled device
  • the present application uses the principle of phase modulation to perform angle detection, and good measurement results can also be obtained by using small-sized pixels (pixels of wavelength or sub-wavelength).
  • phase modulation structure For convenience of description, the two-dimensional optical phase modulation structure is simply referred to as a phase modulation structure below.
  • the phase modulation structure of the embodiment of the present application can perform phase modulation on incident light, and the energy distribution of the modulated optical signal is related to the incident angle of the optical signal.
  • the phase modulation structure 110 may include a substrate 111 and a boss 112 disposed on the substrate 111, and the substrate 111 and the boss 112 may be used to phase modulate the incident light signal.
  • the phase modulation structure may include a substrate and an array of small holes disposed on the substrate, and the substrate and the array of small holes may be used to phase modulate the incident light signal.
  • the plurality of phase modulation structures 110 may be periodically arranged along the X direction and the Y direction, and together with the plurality of pixel point arrays 120 form the depth information image acquisition device of the present application.
  • the X direction and the Y direction may be any two mutually perpendicular directions on the plane where the phase modulation structure is located.
  • the X direction and the Y direction may be the two directions shown in FIG. 4 or FIG. 11 .
  • phase modulation structure of the present application will be described below by taking the substrate and the boss as examples.
  • the cross section of the boss may be a center-symmetrical figure, for example, the cross section of the boss may be a circle, a square, a rhombus, a regular hexagon or a regular octagon, etc., as shown in FIG. 2 .
  • the cross section of the boss is circular, that is, the boss has a cylindrical structure.
  • the centrally symmetric boss shape introduces a relatively small signal error in the angle recognition process, which is beneficial to improve the accuracy of the detection angle.
  • the symmetry of the light intensity distribution will be better, the easier it is to determine the center of the light intensity distribution, and the determined center of the light intensity distribution is also more accurate, so that the determined angle information will be more accurate.
  • the parameters of the phase modulation structure can include the position of the center of the boss, the geometric size of the boss, the distance between the phase modulation structure and the pixel array, the size of the substrate, etc. These parameters can affect the detection angle range and/or detection accuracy of the device .
  • the design of the parameters of the phase modulation structure is described below.
  • the center of the boss in one phase modulation structure may coincide with the center of the pixel array corresponding to the one phase modulation structure.
  • the center of the boss can be coincident with the center of the 3 ⁇ 3 pixel point array.
  • the center of the boss 112 may coincide with the center of the intermediate pixel point of the pixel point array 120 , wherein the center of the intermediate pixel point is also the center of the pixel point array 120 .
  • the centers of all the bosses may be set on the centers of the corresponding pixel arrays.
  • FIG. 3 shows 9 pixel point arrays, each pixel point array is a 3 ⁇ 3 pixel point array, and the center of each boss is located at the center of the corresponding pixel point array.
  • the center of the boss may also have an offset relative to the center of the corresponding pixel array, and the offset may be determined according to the chief ray of the imaging optical system. Specifically, the magnitude of the offset may be determined according to the position of a phase modulation structure on the entire optical sensor and the incident angle of the chief ray at the position jointly determined by the imaging optical system, so that the corresponding pixel array The determined incident angle is the actual incident angle of the optical signal - the chief ray incident angle.
  • the imaging optical system may include a main lens, and due to the converging effect of the main lens in the imaging optical system, the chief ray angle (CRA) of the optical signal received by the phase modulation structures at different positions is different.
  • CRA chief ray angle
  • the inclination angle of the chief ray of the optical signal received by the phase modulation structure located in the middle position is 0, but the inclination angle of the chief ray of the optical signal received by the phase modulation structure located at the edge is greater than 0, and the closer to the edge position, the inclined angle of the chief ray The bigger the angle.
  • the embodiment of the present application can also perform translation optimization on the center of the boss according to the CRA at different positions.
  • the actual device is balanced due to the CRA.
  • the change of ⁇ causes the response shift of the pixel points of the pixel point array at different positions, and the boss can be shifted according to the CRA to balance the relative value of the obtained angle with respect to the CRA.
  • the non-shifted boss is used to directly detect the absolute value of the actual incident angle of the incident light in the main coordinate system of the optical system
  • the shifted boss is used to detect the actual incident light relative to the imaging main coordinate system of the imaging optical system.
  • the relative value of the incident angle of the light, the offset scheme is beneficial to reduce the computational complexity, and can be directly used to judge underfocus or overfocus.
  • the specific offset at the position of the boss with a certain distance can be calculated by simulation, that is, only the offset of part of the boss is calculated, and the offset of the rest of the boss is calculated. It can be calculated according to the difference algorithm, so that there is no need to calculate the offset of the boss at each position, which can reduce the complexity of the calculation.
  • the geometric dimension of the boss may include the lateral dimension of the boss and the height of the boss.
  • the ratio between the lateral size of the boss and the size of one pixel may be between 0.5 and 1.5.
  • the ratio between the lateral size of the boss and the size of a pixel may be approximately equal to 1:1.
  • the specific ratio can be optimized according to the whole device and the specific situation. The reason is that when the pixel points are arranged at a certain point, the lateral size of the boss determines the duty cycle of the phase modulation structure, and the duty cycle can effectively detect the inclination angle of the incident light. range is strongly correlated.
  • the lateral dimension here may refer to the dimension of the cross section of the boss along the direction of the optical sensor surface. If the shape of the boss is circular, the lateral dimension of the boss can be the diameter of the boss; if the shape of the boss is square, the lateral dimension of the boss can be the side length of the boss.
  • the height of the boss can be between ⁇ /4 ⁇ 5 ⁇ , where ⁇ is the wavelength of the light signal incident on the boss in vacuum.
  • Too low boss height will increase the difficulty of processing, and too high boss height will block light incident from a large angle.
  • the light field distribution of the light will affect the angle detection range. Therefore, the height of the boss can be determined according to the actual required angle detection range.
  • the height of the boss is related to the wavelength of the optical signal, the height of the boss will be adjusted accordingly if the wavelength of the optical signal incident on the boss is different.
  • a filter structure may also be provided above the phase modulation structure, and the filter structure may include filters of different colors.
  • the wavelengths of the optical signals of the two bosses are different, and the height of the optimum boss may also be different for the different wavelengths.
  • the bosses of each phase modulation structure can be selected to have the same height and lateral size, but this may lose the detection accuracy in certain incident angle ranges. Optimum boss height and lateral dimensions for the same detection angle range.
  • the geometric dimensions of the bosses in the plurality of phase modulation structures may all be the same, or the geometric dimensions of the bosses in the plurality of phase modulation structures may not be exactly the same, that is, the plurality of phase modulation structures may include bosses with at least two geometric sizes .
  • the distance H between the plurality of phase modulation structures and the optical sensor may be related to at least one of the period of the boss, the wavelength of the optical signal incident on the one phase modulation structure, and the pixel size.
  • the plurality of phase modulation structures The distance H between the structure and the optical sensor may be determined according to at least one of the period of the boss, the wavelength of the light signal incident on the one phase modulation structure, and the pixel size.
  • the period P of the bosses is the distance between any two adjacent bosses in the device, as shown in FIG. 3 .
  • the H between the plurality of phase modulation structures and the optical sensor may refer to the distance between the upper surface of the boss and the upper surface of the pixel array, as shown in FIG. 1 and FIG. 4 .
  • the distance H between multiple phase modulation structures and the optical sensor is less than 10 ⁇ m, which can ensure that the detection angle is monotonous within the angle detection range, which is beneficial to improve the accuracy of the detection angle.
  • the calculation process of the angle information will be described in detail below.
  • the size of the substrate in a phase modulation structure may be consistent with the size of the pixel array corresponding to the one phase modulation structure, that is, the substrate in a two-dimensional optical phase modulation structure and its corresponding pixel points
  • the projections of the arrays in a direction perpendicular to the substrate surface overlap.
  • the projections of the substrate 111 and the pixel array 120 in a direction perpendicular to the surface of the substrate overlap.
  • the substrate and the boss in the embodiments of the present application are all light-transmitting materials, which can transmit incident light signals and make the light signals reach the optical sensor. After the structure reaches the optical signal of the phase modulation structure.
  • the phase modulation structure can be formed by using nano-imprint technology, laser direct writing technology, or a combination of photolithography and plasma etching technology on a transparent optical substrate.
  • the material of the transparent optical substrate can be selected from fused silica, glass substrate, silicon dioxide, etc., which is not specifically limited in this application and does not emphasize the process manufacturing of the phase modulation structure.
  • optical phase modulation structures can remove excess material on thick substrates to form thin substrates and relatively convex bosses.
  • modulation structures of different sizes can be designed.
  • patterns of different sizes only need to be designed on the layout, and bosses of different sizes can be obtained by one photolithography.
  • the photosensitive unit of the pixel can be realized based on CMOS technology or CCD technology.
  • the pixel point array in this embodiment of the present application may be an N ⁇ N pixel point array, where N is a positive integer greater than or equal to 2.
  • N is a positive integer greater than or equal to 2.
  • the symmetry of the N ⁇ N pixel point array is good, which is beneficial to reduce the computational complexity.
  • the pixel point array may also be an N ⁇ M pixel point array, and N and M are not equal and are both positive integers.
  • the period of the phase modulation structure in the X and Y directions (P in FIG. 3 ) is equal to the pixel size of a pixel array (ie, the sum of N single pixel sizes).
  • the period P of the phase modulation structure is equal to the length of one pixel array.
  • one pixel array may be a 3 ⁇ 3 pixel array, or one pixel array may be a 2 ⁇ 2 pixel array.
  • All pixel points in a pixel point array form an imaging pixel point, and the sum of the light signals received by a pixel point array is used to generate an image of the object.
  • the sum of the intensities of optical signals sensed by multiple pixels in a pixel array is the sensing intensity of one imaging pixel, that is, each pixel in a single pixel array will sense a photosensitive signal according to incident light, such as representing The photosensitive signal of the light intensity, and the sum of the photosensitive signals of multiple pixels is output as the photosensitive signal of the imaging pixel, and the depth information detected by the pixel array is the depth information of the imaging pixel.
  • the imaging pixel not only detects the intensity of the incident light but also detects the angle information of the incident light, and the array formed by the periodic arrangement of multiple imaging pixels is this.
  • the novel image sensing device of the embodiment of the application can simultaneously capture two-dimensional image information and depth information of an object image.
  • the boss scheme of the embodiment of the present application has a high pixel utilization rate.
  • the signal addition of the pixel points can also be used for imaging; while the groove scheme can also detect the incident light. Angle information, but a part of the pixels need to be lost for angle detection, and all the pixels cannot be used for imaging.
  • the optical sensor of the embodiment of the present application may be a back-illuminated optical sensor.
  • the optical signal does not need to pass through the metal wiring layer of the pixel array circuit before reaching the pixel array, which can reduce the interference of the optical signal and crosstalk problem.
  • an optical micro lens array, an optical filter structure, an optical baffle (grid), an optical anti-reflection film, etc. may be sequentially arranged above the pixel point array.
  • adding an optical filter structure to the device 100 is a preferred solution, which can improve the accuracy of angle detection, but is not a necessary solution, and the present invention can still be used for wide-spectrum incident light, such as natural light.
  • the optical filter structures corresponding to a single phase modulation structure are of the same type, and the optical filter structures corresponding to different types of phase modulation structures can be of different types or of the same type.
  • the depth information image acquisition device may further include a filter structure 140, and the filter structure may be arranged above the entire optical sensor. Specifically, the filter structure 140 may be arranged between the phase modulation structure 110 and the pixel points. Between the arrays 120, the filter structure 140 can be used to filter the light signal reflected by the object.
  • the filter structure here may be a filter structure in an RGB camera. After adding the filter structure, the depth information image acquisition device can obtain color image information and depth information of the object.
  • Adding a filter structure can reduce the spectral width of the incident light, and a narrower spectrum is beneficial to improve the accuracy of angle detection.
  • this is not a mandatory solution, and broad-spectrum incident light or white light is still suitable for this application.
  • the depth information image acquisition device may not include a filter structure, and if the filter structure is not included, the depth information image acquisition device can obtain a grayscale image and depth information of an object.
  • the optical filter structure in the present application can transmit optical signals of specific wavelengths, and filter optical signals of non-specific wavelengths.
  • the filter structure including a red color filter as an example, the red color filter can transmit red light signals and filter non-red light signals.
  • the filter structure may include filters of one color, that is, all the filters above the pixel array have the same color, and the filter structure is a monochromatic filter.
  • the filters arranged above each pixel array are red filters, and the 4 pixel arrays can be used to receive the color filters filtered by the red filters. Red light signal.
  • the filter structure may include filters of multiple colors.
  • the filter structure can choose to transmit one of the light signals such as red (R) light, green (G) light, blue (B) light, etc., or can choose to transmit from ultraviolet (UV) to terahertz ( Any wavelength in the THz) band range is the optical signal of the central wavelength.
  • the multi-color filters may be arranged in a mosaic filter array, that is, the filter structure may include a mosaic filter array, so that the plurality of pixel point arrays can generate a color image of the object.
  • the filter structure may include a bayer filter array. As shown in FIG. 6 and FIG. 7 , the filter structure can be arranged according to RGB Bayer, and the corresponding pixel point array is used to sense RGB light signals.
  • One pixel array can correspond to a color filter.
  • pixel array 1 can be used to receive red light signals
  • pixel array 2 and pixel array 3 can be used to receive green light signals.
  • the pixel array 4 can be used to receive blue light signals
  • the 4 pixel arrays shown in FIG. 6 and FIG. 7 can be used to generate a color image of the object.
  • the embodiments of the present application are not limited to Bayer-type patterns, but can be applied to any appropriate mosaic color pattern, for example, the mosaic color pattern can be RGB-IR, RGB-W, CYGM, CYYM, RGBE, RGBW, and so on.
  • R represents red
  • G represents green
  • B represents blue
  • C represents cyan
  • Y represents yellow
  • M represents magenta
  • IR represents infrared
  • E represents emerald color
  • W represents white light.
  • the pixel array can be used to collect light signals of one color.
  • the color of the filter above one pixel array can also include multiple colors, and the pixel array can be used to receive light signals of multiple colors, that is, two pixels in the pixel array can receive light signals of multiple colors. Light signals of different colors.
  • the angular range of the optical signal that can be detected by the phase modulation structure is related to the wavelength of the incident light, for the case where the filter structure includes filters of different colors, the geometric dimensions of the bosses corresponding to the filters of different colors can be different. , so that the range of incident angles of optical signals that can be detected by different phase modulation structures is the same.
  • the geometric dimensions of the bosses of the multiple phase modulation structures can also be set to be the same. This is because although the range of incident angles of optical signals that can be detected by different phase modulation structures is different, the present application can use the smallest range of incident angles to detect depth information.
  • the depth information image acquisition device may further include a microlens array 130 , and the microlens array 130 may be disposed between the phase modulation structure 110 and the pixel array 120 for modulating the phase modulation structure 110 The resulting optical signals are collected to the pixel array 120 .
  • the microlenses in the microlens array have a one-to-one correspondence with the pixels in the optical sensor, that is, one pixel can correspond to one microlens, and one microlens can converge the optical signal modulated by the phase modulation structure to the corresponding pixel point. As shown in FIG. 4 , the microlenses 131 correspond to the pixel points 121 .
  • the depth information image acquisition device may further include an anti-reflection coating (ARC), and the anti-reflection coating is beneficial to improve the intensity of the light signal received by the optical sensor.
  • ARC anti-reflection coating
  • the anti-reflection film can be arranged on the surface of any device in the depth information image acquisition device. For example, in order to increase the amount of light passing through the phase modulation structure, an anti-reflection film may be provided on the upper surface of the phase modulation structure. For another example, in order to increase the amount of light received by the pixel array, an anti-reflection film may be provided on the surface of the optical sensor.
  • the microlens array 130 may be disposed below the phase modulation structure 110, and the filter structure 140 may be disposed between the microlens array 130 and the pixel array 120, and an upper surface of the pixel array 120 may be provided with a Layer anti-reflection coating.
  • the depth information image acquisition device may further include a main lens, which may also be referred to as an imaging lens, the main lens is used to image the object, and the main lens may be arranged above all the phase modulation structures.
  • the plurality of phase modulation structures are used for receiving the optical signal passing through the main lens, and performing phase modulation on the optical signal, and the optical sensor is used for receiving the phase-modulated optical signal.
  • the depth information image acquisition device 200 may include a phase modulation structure 210 , a microlens array 220 , a filter structure 230 , a spacer 240 , an antireflection film 250 and a pixel array 260 .
  • the phase modulation structure 210 includes a substrate 211 and a boss 212 disposed above the substrate 211 ; the pixel array 260 may include a photosensitive unit 261 .
  • the phase modulation structure 210 may be disposed above the pixel array 260 , the microlens array 220 may be disposed between the phase modulation structure 210 and the pixel array 260 , and the filter structure 230 may be disposed between the microlens array 220 and the pixel array 260 In between, the anti-reflection film 250 may be disposed on the upper surface of the pixel array 260 .
  • an embodiment of the present application further provides an electronic device 400 , the electronic device 400 has a depth information image acquisition apparatus 410 and a processor 420 , and the depth information image acquisition apparatus 410 may be any of the above-described ones A depth information image acquisition device.
  • the processor 410 can be configured to determine the incident angle of the optical signal incident on the corresponding phase modulation structure according to the intensity of the optical signal received by the pixels in a pixel array.
  • the incident angle in this embodiment of the present application may include an azimuth angle and an inclination angle
  • the depth information image acquisition device may detect the azimuth angle and inclination angle of the incident light signal.
  • the phase modulation structure of grooves if the angle information in two directions is to be detected, grooves with different axis directions need to be designed, and this method will sacrifice the resolution, and the angle information in two directions needs to be calculated by interpolation method.
  • the present application adopts the phase modulation structure of the boss, and the azimuth angle and the inclination angle of the object can be simultaneously detected through a phase modulation structure, and the structure is simple and does not require complicated optical design.
  • the azimuth angle can represent the angle between the projection of the incident optical signal on the plane where the phase modulation structure is located and the coordinate axis on the plane, and the tilt angle can represent the angle between the optical signal and the direction perpendicular to the plane where the phase modulation structure is located.
  • ⁇ represents the inclination angle, Indicates the azimuth angle.
  • the X axis and the Y axis represent any two mutually perpendicular directions on the plane where the boss is located, and the Z axis represents the direction perpendicular to the plane where the boss is located.
  • the tilt angle ⁇ represents the angle between the optical signal and the Z axis, the azimuth angle Indicates the angle between the optical signal and the X-axis.
  • the X axis and the Y axis are preferably directions extending along the arrangement direction of the pixel array, as shown in FIG. 4 .
  • the X axis and the Y axis are preferably the directions extending along the diagonal of the pixel array, as shown in FIG. 11 .
  • the processor 410 can calculate the azimuth angle according to the following formula
  • X G represents the barycentric coordinate on the X axis of the intensity of the optical signal received by the pixel in the X direction
  • Y G represents the barycentric coordinate on the Y axis of the intensity of the optical signal received by the pixel in the Y direction.
  • the directions are two directions perpendicular to each other on the plane where the pixel array is located.
  • the position coordinates of the middle pixel can be set to the zero position (0,0), and the processor 410 can calculate the barycentric coordinates (X G , Y G ) by the following formula :
  • X i,j represents the coordinates of the pixel point in the ith row and the jth column in the X direction
  • Y i,j represents the coordinate in the Y direction of the pixel point in the ith row and the jth column
  • P i,j represents the th The intensity of the optical signal received by the pixel point in row i and column j, where i and j are both positive integers.
  • both the X axis and the Y axis are directions consistent with the arrangement direction of the pixel points, as shown in FIG. 4 .
  • the above uses the pixel points (1,2), (2,1), (2,3), (3,2) as examples to calculate the barycentric coordinates.
  • the barycentric coordinates calculated in this way are relatively accurate, but the embodiments of the present application do not Limited to this, other pixels can also be used to calculate the barycentric coordinates.
  • the processor 410 is configured to calculate the barycentric coordinates (X G , Y G ) by the following formula:
  • X i,j represents the coordinates of the pixel point in the ith row and the jth column in the X direction
  • Y i,j represents the coordinate in the Y direction of the pixel point in the ith row and the jth column
  • P i,j represents the th The intensity of the optical signal received by the pixel point in row i and column j, where i and j are both positive integers.
  • the X axis and the Y axis are directions forming an included angle of 45° with the arrangement direction of the pixel points, as shown in FIG. 11 .
  • the barycentric coordinates calculated by formula (4) and formula (5) are relatively accurate, but the embodiments of the present application are not limited to this.
  • the barycentric coordinates can also be calculated using the following formula:
  • the azimuth angle It can also be calculated by the Sobel operator.
  • the processor 410 may calculate the azimuth according to the following formula
  • G y represents the intensity gradient in the Y direction obtained according to the Sobel operator (sobel operator) method
  • G x represents the intensity gradient in the X direction obtained according to the Sobel operator.
  • Sobel operator can be as follows:
  • P represents the pixel intensity of a pixel array.
  • the azimuth angle is calculated by the above method Afterwards, the processor 410 may determine the tilt angle ⁇ according to the pre-established correspondence between ⁇ and f( ⁇ ), and the processor 410 may determine f( ⁇ ) according to the following formula:
  • ⁇ P x represents the normalized power difference in the X direction
  • ⁇ P y represents the normalized power difference in the Y direction.
  • the sum of the optical power difference of the pixel point in the Y direction and the optical power difference of the pixel point in the X direction can be expressed as a function of the tilt angle ⁇ , as follows:
  • Incident light azimuth When it is 0 degrees, the sum of the optical power difference of the pixel point in the Y direction and the optical power difference of the pixel point in the X direction ⁇ P y ( ⁇ ,0)+ ⁇ P x ( ⁇ ,0) and the azimuth angle is the sum of the power differences at other angles Satisfy a certain optical power conservation relationship.
  • Equation (14) Equation (14)
  • Equation (15) Equation (15)
  • the normalized power differences ⁇ P x and ⁇ P y can be calculated according to the optical power received by the pixels in the pixel array, and the azimuth angle It can be calculated according to the method described above, so that f( ⁇ ) can be calculated according to formula (16).
  • ⁇ can be obtained through computer simulation or experimental detection. After f( ⁇ ) is calculated by the above formula, ⁇ can be obtained according to the corresponding relationship between ⁇ and f( ⁇ ). .
  • the processor 410 can be used to determine ⁇ P x and ⁇ P y according to the following formulas:
  • Equation (17) and Equation (18) is only one implementation determining ⁇ P x ⁇ P y and obtain in this way is calculated by ⁇ P x ⁇ P y and more accurate, but the present disclosure is not limited thereto, the following can also formula to determine ⁇ P x and ⁇ P y .
  • the processor 410 can be used to determine ⁇ P x and ⁇ P y according to the following formulas:
  • Equation (21) and equation (22) is only one implementation determining ⁇ P x ⁇ P y and obtain in this way is calculated by ⁇ P x ⁇ P y and more accurate, but the present disclosure is not limited thereto, the following can also formula to determine ⁇ P x and ⁇ P y .
  • the 2 ⁇ 2 pixel array makes the entire phase modulation structural unit use less total pixels, and more phase modulation structural units can be placed on the imaging optical sensor of the same area for participating in angle detection, which can improve the Image resolution with depth information.
  • the 3 ⁇ 3 pixel point array has high angle detection range and detection accuracy, but it will sacrifice a certain image spatial resolution.
  • the specific pixel array to be used can be determined according to actual needs.
  • the pixel array is not limited to the above two forms, and a 4 ⁇ 4 pixel array or a 5 ⁇ 5 pixel array can also be used.
  • an embodiment of the present application further provides a method for calculating incident light angle information. As shown in FIG. 13 , the method may be executed by the processor 410 .
  • a phase modulation structure and its corresponding pixel array may be collectively referred to as an angle-sensitive detection unit.
  • the angle-sensitive detection unit selects different color filters to form angle-sensitive detection units of different colors.
  • the height of the boss is within a certain range, the above energy distribution phenomenon can be generated, but the same boss height can detect the azimuth angle of incident light of different wavelengths. and the accuracy of the tilt angle have different effects, preferably the heights of the bosses of the angle-sensitive detection units with different wavelengths can be optimized through numerical simulation.
  • the height H of the phase modulation structure from the pixel point photosensitive unit is strongly related to the detection range of the incident light inclination angle. If H is too large, f( ⁇ ) will appear similar periodic distribution with the incident light inclination angle ⁇ , that is, multiple inclination angles. Corresponding to the same f( ⁇ ) value, so for the same f( ⁇ ) value, multiple ⁇ values will be generated by inversion. Therefore, when designing the size of H, it is necessary to keep f( ⁇ ) monotonic within the designed detection range.
  • the specific value of H is related to many factors such as the period of the boss, the wavelength of the incident light, and the size of the pixel point. It can be obtained through simulation optimization, and H is generally less than 10um.
  • the pixel size is 2um
  • the ratio between the lateral size of the boss and the pixel size is 1:1
  • the height of the boss is 0.344um
  • H is 2.644um
  • the inclination angle ⁇ can be obtained.
  • the detection range is ⁇ 23 degrees
  • the detection accuracy of the azimuth angle is less than ⁇ 5 degrees.
  • the detection angle range in the embodiment of the present application may refer to the range within which the ⁇ value corresponding to f( ⁇ ) is unique.
  • a single angle-sensitive detection unit can obtain the incident light angle information of each angle-sensitive detection unit through the angle detection method of the present application, and can obtain the incident azimuth angle and the inclination angle at the same time, and use the angle information to form image depth detection and three-dimensional image reconstruction. .
  • z f is the distance between the imaging object plane and the main plane of the optical system in the ideal imaging process
  • z s is the distance between the photosensitive sensor and the main plane of the optical system in the ideal imaging process
  • f Represents the focal length of the imaging optical system.
  • the point on the object plane is ideally imaged on the image plane, and a clear two-dimensional image can be formed by placing the photosensitive sensor on the image plane.
  • the optical system forms a blurred image.
  • the size of the blurred image point b w can be obtained.
  • the actual object can be obtained by the following formula
  • the distance u from the object-side principal point of the optical system can be obtained, that is, the depth information of the image of the imaging object.
  • D is the pupil aperture
  • the object image can be determined by the object distance. depth information.
  • the directions of the light signals incident on the light-sensitive surface of the solid line and the dotted line are exactly opposite, so it can be judged whether it is under-focus or over-focus according to the direction of the incident light signal.
  • the incident angle, especially the inclination angle ⁇ , of the imaging beam of the object point located before and after the alignment plane incident on the angle-sensitive detection unit on the photosensitive sensor can be clearly resolved.
  • a software functional unit If implemented in the form of a software functional unit and sold or used as a stand-alone product, it may be stored in a computer-readable storage medium.
  • the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that make contributions to the prior art or the parts of the technical solutions, and the computer software products are stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application.
  • the aforementioned storage medium includes: a U disk, a removable hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk and other media that can store program codes.
  • division of units, modules or components in the apparatus embodiments described above is only a logical function division, and other division methods may be used in actual implementation.
  • multiple units, modules or components may be combined or integrated.
  • To another system, or some units or modules or components can be ignored, or not implemented.
  • the above-mentioned units/modules/components described as separate/display components may or may not be physically separated, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units/modules/components may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Solid State Image Pick-Up Elements (AREA)

Abstract

一种深度信息图像采集装置和电子设备（400），结构简单，能够降低设备功耗且有利于提高户外使用效果。该装置包括：多个二维光学相位调制结构（110），每个二维光学相位调制结构（110）包括基板（111）和设置在基板（111）上的凸台（112），且用于对物体反射的光信号进行空间相位调制；光学传感器，设置在多个二维光学相位调制结构（110）的下方，且包括多个像素点阵列（120），多个像素点阵列（120）与多个二维光学相位调制结构（110）一一对应，一个像素点阵列（120）用于接收经过与其对应的二维光学相位调制结构（110）调制后的光信号，一个像素点阵列（120）包括多个像素点，多个像素点接收的光信号的强度分布用于确定入射至对应的二维光学相位调制结构（110）的光信号的入射角度，入射角度用于确定物体图像的深度信息。

Description

深度信息图像采集装置和电子设备 技术领域
本申请实施例涉及计算机领域，并且更具体地，涉及一种深度信息图像采集装置和电子设备。
背景技术
目前采集具有深度信息的图像越来越多地应用于智能互联设备以及智能可穿戴设备中，例如智能手机、虚拟现实(virtual reality，VR)或增强现实(augmented reality，AR)设备中。但是目前的深度信息测量方案存在很多问题，例如，需要额外的主动式光源、对环境干扰敏感、计算复杂等问题，这阻碍了深度测量装置的大规模商用。
发明内容
本申请实施例提供了一种深度信息图像采集装置和电子设备，能够降低设备功耗且有利于提高户外使用效果。
第一方面，提供了一种深度信息图像采集装置，包括：多个二维光学相位调制结构，所述多个二维光学相位调制结构中的每个二维光学相位调制结构包括基板以及设置在基板上的凸台，所述每个二维光学相位调制结构用于对所述物体反射的光信号进行空间相位调制；光学传感器，设置在所述多个二维光学相位调制结构的下方，所述光学传感器包括多个像素点阵列，所述多个像素点阵列与所述多个二维光学相位调制结构具有一一对应关系，所述多个像素点阵列中的一个像素点阵列用于接收经过与其对应的二维光学相位调制结构调制后的光信号，所述一个像素点阵列包括多个像素点，所述多个像素点接收的光信号的强度分布用于确定入射至所述对应的二维光学相位调制结构的光信号的入射角度，所述入射角度用于确定所述物体图像的深度信息。
本申请不需要额外的光源即可进行深度信息检测，能够降低设备功耗；且这种检测方式受环境干扰较小，有利于提高户外使用效果。
本申请可以在传统图像探测芯片的结构上，直接引入一层二维光学相位调制结构，结构简单，所引入的二维光学相位调制结构也无需像传统全场相 机那样进行复杂的光学设计。
在一些可能的实现方式中，所述凸台的横截面为中心对称图形。
中心对称的凸台形状在角度识别过程中引入的信号误差比较小，有利于提高探测角度的准确性。
在一些可能的实现方式中，所述凸台为圆柱形结构。
在一些可能的实现方式中，所述凸台的中心与与其对应的像素点阵列的中心重合。
在一些可能的实现方式中，所述凸台的中心相对于与其对应的像素点阵列的中心具有偏移量，使得所述对应的像素点阵列确定的入射角度为光信号的实际入射角度-主光线入射角。
未经偏移的凸台为直接检测入射光在光学系统主坐标系中实际的入射角度绝对值，经过偏移后的凸台用于检测实际入射光线相对于成像光学系统成像主光线入射角的相对值，这有利于降低计算复杂度，可直接用于判断欠焦还是过焦。
在一些可能的实现方式中，所述凸台的横向尺寸与一个像素点的尺寸之间的比值介于0.5～1.5之间。
在一些可能的实现方式中，所述凸台的高度位于λ/4～5λ之间，λ为入射至所述凸台的光信号在真空中的波长。
在一些可能的实现方式中，所述多个二维光学相位调制结构与光学传感器之间的距离H小于10μm。
距离H小于10μm能够保证探测角度在角度探测范围内单调，有利于提高探测角度的准确性。
在一些可能的实现方式中，一个二维光学相位调制结构中的基板与与其对应的像素点阵列在垂直于所述基板表面的方向上的投影重叠。
在一些可能的实现方式中，所述一个像素点阵列为N×N像素点阵列，N为大于或等于2的正整数。
N×N像素点阵列对称性较好，有利于降低计算的复杂度。
在一些可能的实现方式中，所述一个像素点阵列为3×3像素点阵列。
3×3像素点阵列具有较高的角度探测范围和探测精度。
在一些可能的实现方式中，所述一个像素点阵列为2×2像素点阵列。
2×2像素点阵列使得整个相位调制结构单元使用总像素点较少，同样面 积的成像光学传感器上可以放置更多的该相位调制结构单元用于参与角度探测，可提升具有深度信息的图片分辨率。
在一些可能的实现方式中，所述一个像素点阵列接收的光信号的总和用于生成所述物体的图像。
在一些可能的实现方式中，所述光学传感器为背照式光学传感器。
采用背照式光学传感器，光信号在到达像素点阵列之前不必穿过像素点阵列电路的金属布线层，能够减小光信号的干扰和串扰问题。
在一些可能的实现方式中，所述多个二维光学相位调制结构中的凸台的几何尺寸均相同。
在一些可能的实现方式中，所述多个二维光学相位调制结构包括至少两种几何尺寸的凸台。
在一些可能的实现方式中，还包括滤光结构，所述滤光结构设置在所述多个二维光学相位调制结构与所述光学传感器之间，所述滤光结构用于对所述物体反射的光信号进行过滤。
加入滤光结构可以减小入射光的光谱宽度，较窄的光谱有利于提升角度探测的准确性。
在一些可能的实现方式中，一个二维光学相位调制结构对应一种颜色的滤光片。
在一些可能的实现方式中，所述滤光结构包括马赛克滤光片。
马赛克滤光片可用于生成物体的彩色图像。
在一些可能的实现方式中，所述滤光结构包括拜耳滤光阵列。
在一些可能的实现方式中，所述滤光结构包括不同颜色的滤光片，不同颜色的滤光片对应的二维光学相位调制结构的凸台的几何尺寸不同，使得不同二维光学相位调制结构能够探测的光信号入射角度的范围相同。
在一些可能的实现方式中，还包括微透镜阵列，所述微透镜阵列设置在所述多个二维光学相位调制结构与所述光学传感器之间，所述微透镜阵列中的微透镜与所述光学传感器中的像素点具有一一对应的关系，所述微透镜阵列用于将经过所述多个二维光学相位调制结构调制后的光信号汇聚至所述光学传感器。
在一些可能的实现方式中，所述入射角度包括方位角和倾斜角，所述倾斜角表示光信号与垂直于所述多个二维光学相位调制结构所在平面的方向 的夹角，所述方位角表示光信号在所述多个二维光学相位调制结构所在平面的投影与所述平面上的坐标轴的夹角。
第二方面，提供了一种电子设备，包括：如第一方面或第一方面中任一可能的实现方式中的深度信息图像采集装置。
在一些可能的实现方式中，还包括处理器，连接至所述深度信息图像采集装置，所述处理器用于根据所述一个像素点阵列中的像素点接收的光信号的强度，确定入射至所述对应的二维光学相位调制结构的光信号的入射角度，所述入射角度包括方位角和倾斜角。
在一些可能的实现方式中，所述处理器用于根据以下公式计算方位角
Figure PCTCN2020103458-appb-000001
Figure PCTCN2020103458-appb-000002
其中，X G表示X方向的像素点接收的光信号的强度在X轴上的重心坐标，Y G表示Y方向的像素点接收的光信号的强度在Y轴上的重心坐标，X方向与Y方向为二维光学相位调制结构所在平面上相互垂直的两个方向。
在一些可能的实现方式中，所述一个像素点阵列为3×3像素点阵列，所述处理器用于通过以下公式计算重心坐标(X G，Y G)：
Figure PCTCN2020103458-appb-000003
Figure PCTCN2020103458-appb-000004
其中，X i,j表示第i行第j列的像素点在X方向上的坐标，Y i,j表示第i行第j列的像素点在Y方向上的坐标，P i,j表示第i行第j列的像素点接收的光信号的强度，i和j均为正整数。
在一些可能的实现方式中，所述一个像素点阵列为2×2像素点阵列，所述处理器用于通过以下公式计算重心坐标(X G，Y G)：
Figure PCTCN2020103458-appb-000005
Figure PCTCN2020103458-appb-000006
其中，X i,j表示第i行第j列的像素点在X方向上的坐标，Y i,j表示第i行第j列的像素点在Y方向上的坐标，P i,j表示第i行第j列的像素点接收的光信号的强度，i和j均为正整数。
通过上述公式确定的中心坐标比较准确。
在一些可能的实现方式中，所述一个像素点阵列为3×3像素点阵列，所述处理器用于根据以下公式计算方位角
Figure PCTCN2020103458-appb-000007
Figure PCTCN2020103458-appb-000008
其中，G y表示根据索贝尔算子得到的Y方向的强度梯度，G x表示根据索贝尔算子得到的X方向的强度梯度。
在一些可能的实现方式中，所述处理器用于根据预先建立的θ与f(θ)之间的对应关系确定倾斜角θ，且所述处理器用于根据以下公式确定f(θ)：
Figure PCTCN2020103458-appb-000009
Figure PCTCN2020103458-appb-000010
表示方位角，ΔP x表示X方向的归一化功率差，ΔP y表示Y方向的归一化功率差。
在一些可能的实现方式中，所述一个像素点阵列为3×3像素点阵列，所述处理器用于通过以下公式确定X方向和Y方向的像素点的归一化功率差：
Figure PCTCN2020103458-appb-000011
Figure PCTCN2020103458-appb-000012
P i,j表示第i行第j列的像素点接收的光信号的强度。
通过上述公式确定的归一化功率差比较准确。
在一些可能的实现方式中，所述一个像素点阵列为2×2像素点阵列，所述处理器用于通过以下公式确定X方向和Y方向的像素点的归一化功率差：
Figure PCTCN2020103458-appb-000013
Figure PCTCN2020103458-appb-000014
P i,j表示第i行第j列的像素点接收的光信号的强度。
通过上述公式确定的归一化功率差比较准确。
附图说明
图1是本申请实施例提供的深度信息图像采集装置的局部结构示意图。
图2是图1所示装置中凸台的几种实施例的形状示意图。
图3是本申请实施例提供的深度信息图像采集装置的俯视图。
图4是本申请实施例提供的深度信息图像采集装置的另一种局部结构示意图。
图5-图7是本申请实施例提供的一种像素点阵列的分布图。
图8是本申请实施例提供的深度信息图像采集装置的另一种局部结构示意图。
图9和图10是本申请实施例提供的入射至光学相位调制结构的光信号的方位角和倾斜角的示意图。
图11是本申请实施例提供的2×2像素点阵列中的X方向和Y方向的示意图。
图12是本申请实施例提供的一种f(θ)的仿真曲线图。
图13是本申请实施例提供的一种确定物体图像的深度信息的方法的示意性流程图。
图14是本申请实施例提供的一种电子设备的示意性框图。
图15是本申请实施例提供的透镜成像过程的示意性结构图。
具体实施方式
三维成像技术目前越来越多地应用于移动智能互联设备以及智能可穿戴设备中，例如智能手机、VR设备或AR设备中。
当下热门技术基于光飞行时间(time-of-flight，TOF)和结构光(Structure Light)技术，需要利用脉冲的、图案的或调制的连续红外光主动照射场景，额外的红外光源和探测器的使用会增加功耗以及整个装置的空间尺寸。另外，这些技术往往对干扰信号、镜面反射和环境红外光敏感，影响户外使用效果。基于压缩感知和非线性重建的随机二进制掩模和编码孔径技术，大量图像字典和计算费用阻碍了其大规模商用。利用单探测器(不需要人工光源)的探测方法还有传统光场相机，该技术需要通过对多个同时采集的图像的比较分析、利用复杂的机器学习或图像重建技术来计算深度信息，或者还需要使用有源照明器和传感器组成的额外信号探测装置配合来恢复深度信息。
基于此，本申请实施例提供一种深度信息图像采集装置，无需引入额外 的主动式光信号发射装置以及额外信号探测装置，且能够减少受环境干扰的影响，降低设备功耗，尤其适用于移动智能设备。
本技术可以用于需要或可受益于增强的图像深度探测和其它三维成像能力的成像应用。例如以允许用户改变捕获的场景图像的焦点、视点和/或景深。本技术可以应用于或实施在各种类型的三维成像系统和方法中，包括但不限于使用全光描述的光场成像应用、通过衍射的色度依赖性的比较分析的测距应用、以及单传感器单图像深度采集应用。本申请实施例可以应用于电子设备的摄像装置，例如，智能手机的前置/后置摄像头、安防探头的摄像装置等。
下面结合图1-图12，对本申请实施例的深度信息图像采集装置进行描述。
该深度信息图像采集装置100可以包括多个二维相位调制结构和光学传感器。
多个二维光学相位调制结构用于对物体反射的光信号进行空间相位调制。
光学传感器可以设置在多个二维相位调制结构的下方，该光学传感器包括多个像素点阵列，该多个像素点阵列与该多个二维光学相位调制结构具有一一对应关系，即一个像素点阵列对应一个二维光学相位调制结构。该多个像素点阵列中的一个像素点阵列120用于接收经过与其对应的一个二维光学相位调制结构110调制后的光信号，该一个像素点阵列120中的像素点接收的光信号的强度用于确定入射至该对应的二维光学相位调制结构110的光信号的入射角度，该入射角度用于确定成像物体的图像的深度信息。
二维光学相位调制结构指的是经过该二维相位调制结构后的光信号会在二维空间上进行分布。
空间相位调制可以理解为光信号经过二维光学相位调制结构的不同折射率区域时产生光程差，从而达到对光信号的相位调制。
可以理解的是，光信号的强度也可以理解为光信号的功率。
图1示出的是一个二维光学相位调制结构110与一个像素点阵列120的示意图，本申请实施例中的装置100可以包括多个图1所示的结构。
该二维光学相位调制结构的核心为具有二维的折射率调制图案(如图1所示的凸台结构)，光经过不同折射率区域形成光程差进而调制入射光的空 间相位分布，进而影响入射光在像素点阵列不同像素点上的能量分布。以图1为例，由于凸台和空气的折射率不同，光经过凸台和经过凸台周围的空气层会发生不同的折射，从而会形成光程差。
本申请利用波动光学中的泰伯效应，多个二维光学相位调制结构的周期性结构引入对入射光的周期性调制。经过调制的入射光发生自相干，在相干增强的位置产生上述周期性结构的“图像副本”，入射光角度不同会影响光程差，通过自相干进而影响调制光的光场周期性空间分布，本申请利用光场的空间分布与入射角的关系探测入射角。
另外，本申请实施例的深度信息图像采集可以无需改变原有的光学传感器的结构，仅需在原有的光学传感器制作完成后，在其顶部增加一层二维光学相位调制结构即可。
不同入射角度的光信号在像素点上的能量分布不同，本申请实施例可以根据该能量分布确定入射光的角度信息。
本申请实施例不需要额外的光源即可进行深度信息检测，能够降低设备功耗；且这种检测方式受环境干扰较小，有利于提高户外使用效果。
本申请可以在传统图像探测芯片的结构上，直接引入一层二维光学相位调制结构，结构简单，所引入的二维光学相位调制结构也无需像传统全场相机那样进行复杂的光学设计。
本申请可利用单一探测器获得的单一图像进行图像深度信息探测和图像三维重构，采用较少功率来执行其功能的无源感测模式。虽然本申请实施例的装置无需主动光源即可实现，但装置仍然可以与主动光源相兼容。装置的结构简单且低成本地集成到现有的图像传感器硬件和制造工艺中；与常规互补金属氧化物半导体(complementary metal oxide semiconductor，CMOS)和充电耦合设备(charged coupled device，CCD)图像传感器的兼容；并且消除了对多个部件的需求，诸如配备有用于深度检测的有源照明系统。
本申请利用相位调制原理进行角度检测，使用小尺寸像素点(波长量级或亚波长量级像素点)也可以取得较好的测量结果。
为方便描述，下文将二维光学相位调制结构简称为相位调制结构。
本申请实施例的相位调制结构能够对入射光进行相位调制，且调制后的光信号的能量分布与光信号的入射角度相关。
例如，如图1所示，该相位调制结构110可以包括基板111和设置在基 板111上的凸台112，该基板111和凸台112可用于对入射光信号进行相位调制。又如，该相位调制结构可以包括基板以及设置在基板上的小孔阵列，该基板和小孔阵列可用于对入射光信号进行相位调制。
多个相位调制结构110可以沿着X方向和Y方向周期性排列，并和多个像素点阵列120共同形成本申请的深度信息图像采集装置。其中，X方向和Y方向可以为相位调制结构所在平面的任意两个相互垂直的方向。例如，X方向和Y方向可以为图4或图11所示的两个方向。
下面以基板和凸台为例，对本申请的相位调制结构进行描述。
凸台的横截面可以是中心对称图形，例如，凸台的横截面可以是圆形、正方形、菱形、正六边形或正八边形等，如图2所示。
优选地，凸台的横截面为圆形，即凸台为圆柱形结构。
中心对称的凸台形状在角度识别过程中引入的信号误差比较小，有利于提高探测角度的准确性。相比于非中心对称的凸台结构，光信号经过中心对称的凸台结构后，光强分布的对称性会越好，越容易确定光强分布的中心，且确定的光强分布的中心也比较准确，从而使得确定的角度信息也会更准确。
相位调制结构的参数可以包括凸台中心的位置、凸台的几何尺寸、相位调制结构与像素点阵列之间的距离、基板的尺寸等，这些参数能够影响装置的探测角度范围和/或探测精度。下面对相位调制结构的参数的设计进行描述。
一个相位调制结构中凸台的中心可以与该一个相位调制结构对应的像素点阵列的中心重合。假设像素点阵列为3×3像素点阵列，则凸台的中心可以与3×3像素点阵列中心重合。具体地，如图1所示，凸台112的中心可以与像素点阵列120的中间像素点的中心重合，其中该中间像素点的中心也是该像素点阵列120的中心。
作为一种实现方式，所有的凸台的中心可以均设置在与其对应的像素点阵列的中心上。如图3所示，图3示出了9个像素点阵列，每个像素点阵列为3×3像素点阵列，每个凸台的中心均位于各自对应的像素点阵列的中心。
凸台的中心也可以相对于与其对应的像素点阵列的中心具有偏移量，该偏移量的大小可以是根据成像光学系统的主光线确定的。具体地，该偏移量的大小可以是根据一个相位调制结构在整个光学传感器上的位置与成像光 学系统共同决定的该位置上的主光线入射角确定的，从而使得所述对应的像素点阵列确定的入射角度为光信号的实际入射角度-主光线入射角。
成像光学系统可以包括主透镜，由于成像光学系统中主透镜的汇聚作用，不同位置的相位调制结构接收的光信号的主光线入射角(chief ray angle，CRA)不同。例如，位于中间位置的相位调制结构接收的光信号的主光线倾斜角为0，但是位于边缘的相位调制结构接收的光信号的主光线倾斜角大于0，并且，越靠近边缘位置，主光线倾斜角越大。
除了按照上文描述的将凸台的中心与与其对应的像素点阵列的中心对齐之外，本申请实施例还可以将凸台的中心根据不同位置的CRA进行平移优化，实际装置为平衡由于CRA的变化引起不同位置的像素点阵列的像素点的响应偏移，可以根据CRA将凸台进行偏移，以平衡得到的角度相对于CRA的相对数值。
如果所有位置的凸台均位于与其对应的像素点阵列的中心，则每个位置感测的角度＝入射光的实际入射角度(光学系统坐标系下)；而如果每个位置的凸台的中心均根据CRA进行了一定的偏移，则每个位置感测的角度＝入射光的实际入射角度(光学系统坐标系下)-每个位置的CRA。
也就是说，未经偏移的凸台为直接检测入射光在光学系统主坐标系中实际的入射角度绝对值，经过偏移后的凸台用于检测实际入射光线相对于成像光学系统成像主光线入射角的相对值，经过偏移后的方案有利于降低计算复杂度，可直接用于判断欠焦还是过焦。
在计算凸台的偏移量的过程中，可通过仿真计算具有一定间距的凸台位置处的具体偏移量，即仅计算部分凸台的偏移量，其余位置的凸台的偏移量可以根据差值算法计算得到，这样无需计算每个位置处的凸台的偏移量，能够降低计算的复杂度。
本申请实施例中，凸台的几何尺寸可以包括凸台的横向尺寸和凸台的高度。
凸台的横向尺寸与一个像素点的尺寸之间的比值可以在0.5～1.5之间。例如，凸台的横向尺寸可以与一个像素点的尺寸之间的比值约等于1:1。具体的比值可以根据整个装置以及具体情况进行优化，原因是在像素点排布一定时，凸台的横向尺寸决定了相位调制结构的占空比，该占空比对入射光倾斜角的有效探测范围强相关。
这里的横向尺寸可以指凸台的横截面沿光学传感器表面的方向上的尺寸。如果凸台的形状为圆形，则凸台的横向尺寸可以为凸台的直径；如果凸台的形状为正方形，则凸台的横向尺寸可以为凸台的边长。
凸台的高度可以位于λ/4～5λ之间，λ为入射至凸台的光信号在真空中的波长。
过低的凸台高度会增加加工难度，过高的凸台高度会对大角度入射的光形成遮挡作用，本申请实施例的凸台高度可以根据实际需要进行选择，凸台高度会影响调制后光的光场分布，会影响角度探测范围，因此，凸台的高度可以根据实际需要的角度探测范围来确定。
由于凸台的高度与光信号的波长有关，因此，入射至凸台的光信号的波长不同，凸台的高度也会因此而调整。例如，本申请实施例还可以在相位调制结构的上方设置滤光结构，该滤光结构可以包括不同颜色的滤光片，如果两个凸台上方的滤光片的颜色不同，则入射至这两个凸台的光信号的波长不同，最佳凸台的高度也可能因此波长的不同而不同。
其中，每个相位调制结构的凸台可以选择为同一高度和横向尺寸，但是这样可能会损失某些入射角范围内的探测精度，优选地是根据不同感光颜色的角度探测单元通过仿真计算设计满足相同探测角度范围的最佳凸台高度和横向尺寸。
多个相位调制结构中的凸台的几何尺寸可以均相同，或者多个相位调制结构中的凸台的几何尺寸可以不完全相同，即多个相位调制结构可以包括至少两种几何尺寸的凸台。
单一几何尺寸的凸台，对于探测多点强光源物体时，深度探测过程会出现误差，在凸台最佳设计值附近选择两种或三种尺寸，可有效降低多点强光源成像误差问题。
多个相位调制结构与光学传感器之间的距离H可以与凸台的周期、入射至该一个相位调制结构的光信号的波长、像素点尺寸中的至少一个相关，换句话说，多个相位调制结构与光学传感器之间的距离H可以是根据凸台的周期、入射至该一个相位调制结构的光信号的波长、像素点尺寸中的至少一个确定的。
其中，凸台的周期P为装置中任意两个相邻凸台之间的间距，如图3所示。
多个相位调制结构与光学传感器之间的H可以指凸台的上表面与像素点阵列的上表面之间的距离，如图1和图4所示。
通过仿真优化得出，多个相位调制结构与光学传感器之间的距离H小于10μm能够保证探测角度在角度探测范围内单调，有利于提高探测角度的准确性。下文会结合角度信息的计算过程进行详细描述。
本申请实施例中，一个相位调制结构中基板的尺寸可以与该一个相位调制结构对应的像素点阵列的尺寸一致，也就是说，一个二维光学相位调制结构中的基板与与其对应的像素点阵列在垂直于所述基板表面的方向上的投影重叠。如在图1所示的结构中，基板111与像素点阵列120在垂直于基板表面的方向上的投影重叠。
本申请实施例中的基板和凸台均为透光材料，能够透过入射光信号，并使该光信号到达光学传感器，入射光信号指的是经物体反射的光信号经主透镜、滤光结构后到达相位调制结构的光信号。
相位调制结构可以通过在透明光学基板上利用纳米压印技术，激光直写技术，或光刻加等离子刻蚀组合技术等方式加工形成。透明光学基板的材料可选熔融石英、玻璃基板、二氧化硅等，本申请不做具体限定且并不强调该相位调制结构的工艺制造。
例如，光学相位调制结构可以在厚的基板上去除多余的材料，形成薄的基板和相对凸出的凸台。
为了更好地探测物体的深度信息，可以设计不同尺寸的调制结构，本申请仅需在版图上设计不同尺寸的图案，通过一次光刻即可得到不同尺寸的凸台。
像素点的感光单元可基于CMOS技术或CCD技术来实现。
本申请实施例中的像素点阵列可以为N×N像素点阵列，N为大于或等于2的正整数。N×N像素点阵列对称性较好，有利于降低计算的复杂度。
当然，像素点阵列也可以为N×M像素点阵列，N和M不相等，且均为正整数。
对于N×N像素点阵列，相位调制结构在X方向与Y方向上的周期(如图3中的P)等于一个像素点阵列的像素点尺寸(即N个单一像素点尺寸之和)。
以图3为例，由于每个凸台的中心均位于与其对应的像素点阵列的中心， 且每个像素点阵列的大小均相同，所以相位调制结构的周期P等于一个像素点阵列的长度。
优选地，一个像素点阵列可以为3×3像素点阵列，或一个像素点阵列为2×2像素点阵列。
一个像素点阵列中的所有像素点组成一个成像像素点，一个像素点阵列接收的光信号的总和用于生成所述物体的图像。一个像素点阵列中的多个像素点感测的光信号的强度之和为一个成像像素点的感测强度，即单个像素点阵列中每个像素点都会根据入射光感应得到感光信号，比如代表光强度的感光信号，而多个像素点的感光信号之和输出作为该成像像素点的感光信号，该一个像素点阵列检测的深度信息为该一个成像像素点的深度信息。
将单个像素点阵列看作为一个成像像素点，该成像像素点除了探测入射光强度外还同时探测了该入射光的角度信息，而多个成像像素点周期性地排布形成的阵列即为本申请实施例的新型图像传感器件，可同时捕获物体图像的二维图像信息和深度信息。
本申请实施例的凸台方案像素点利用率高，一个像素点阵列中的像素点除了可以探测角度外，像素点的信号相加还可用于成像；而凹槽方案虽然也能探测入射光的角度信息，但是需要损失一部分像素点用于角度探测，像素点不能全部用于成像。
本申请实施例的光学传感器可以为背照式光学传感器，采用背照式光学传感器，光信号在到达像素点阵列之前不必穿过像素点阵列电路的金属布线层，能够减小光信号的干扰和串扰问题。
可选地，本申请实施例可以在像素点阵列上方依次设置光学微透镜阵列(micro lens array)、光学滤光结构、光学隔板(grid)、光学减反射膜(anti-reflection film)等。其中，在装置100中加入光学滤光结构为优选方案，可以提升角度探测的准确性，但是并非必选方案，宽光谱入射光，比如自然光仍然可以使用本发明。在本申请中单个相位调制结构对应的光学滤光结构为同一种，不同类型相位调制结构对应的光学滤光结构可以选择不同类型，也可以为同一类型。
如图4所示，深度信息图像采集装置还可以包括滤光结构140，该滤光结构可以设置在整个光学传感器的上方，具体地，该滤光结构140可以设置在相位调制结构110与像素点阵列120之间，该滤光结构140可用于对物体 反射的光信号进行过滤。
这里的滤光结构可以是RGB相机中的滤光结构，加入滤光结构后，该深度信息图像采集装置可以获得物体的彩色图像信息以及深度信息。
加入滤光结构可以减小入射光的光谱宽度，较窄的光谱有利于提升角度探测的准确性。但是这并非必选方案，宽光谱入射光或白光仍然适用于本申请。
可以理解的是，深度信息图像采集装置可以不包括滤光结构，如果不包括滤光结构，则深度信息图像采集装置可以得到物体的灰度图像以及深度信息。
本申请中的滤光结构可以透过特定波长的光信号，而过滤非特定波长的光信号。以滤光结构包括红色滤光片为例，该红色滤光片可以透过红色光信号，而过滤非红色光信号。
在一种可能的实现方式中，该滤光结构可以包括一种颜色的滤光片，即所有的像素点阵列上方的滤光片的颜色均相同，该滤光结构为单色滤光片。
例如，在图5所示的4个像素点阵列中，每个像素点阵列上方设置的滤光片均为红色滤光片，该4个像素点阵列可用于接收经过红色滤光片过滤后的红色光信号。
在另一种可能的实现方式中，该滤光结构可以包括多种颜色的滤光片。该滤光结构可以选择透过红(R)光、绿(G)光、蓝(B)光等光信号中的一种，也可以根据实际应用选择透过从紫外(UV)到太赫兹(THz)波段范围内的任意波长为中心波长的光信号。
该多种颜色的滤光片可以按照马赛克滤光阵列进行排列，即该滤光结构可以包括马赛克滤光阵列，使得该多个像素点阵列能够生成物体的彩色图像。
作为一种优选的方式，该滤光结构可以包括拜耳(bayer)滤光阵列。如图6和图7所示，该滤光结构可以按照RGB Bayer进行排布，对应的像素点阵列用于感测RGB光信号。
一个像素点阵列可以对应一种颜色的滤光片，例如，在图6和图7中，像素点阵列1可用于接收红色光信号，像素点阵列2和像素点阵列3可用于接收绿色光信号，像素点阵列4可用于接收蓝色光信号，且图6和图7所示的4个像素点阵列可用于生成物体的彩色图像。
本申请实施例并不限于Bayer型图案，而是可以适用于任何适当的马赛克颜色图案，例如该马赛克颜色图案可以为RGB-IR、RGB-W、CYGM、CYYM、RGBE、RGBW等。
其中，R表示红色，G表示绿色，B表示蓝色，C表示青色，Y表示黄色，M表示品红，IR表示红外，E表示祖母绿(emerald)色，W表示白光。
一个像素点阵列上方的滤光片的颜色为同一种颜色，则该像素点阵列可用于采集一种颜色的光信号。在某种情况下，一个像素点阵列上方的滤光片的颜色也可以包括多种，该像素点阵列可用于接收多种颜色的光信号，即该像素点阵列中的两个像素点可以接收不同颜色的光信号。
由于相位调制结构能够探测的光信号的角度范围与入射光的波长有关，因此，对于滤光结构包括不同颜色的滤光片的情况，不同颜色的滤光片对应的凸台的几何尺寸可以不同，使得不同相位调制结构能够探测的光信号的入射角度的范围相同。
此外，作为一种实现方式，即使多个相位调制结构上方对应不同颜色的滤光片，但是也可以将该多个相位调制结构的凸台的几何尺寸设置为相同。这是由于虽然不同相位调制结构能够探测的光信号的入射角度的范围不同，但是本申请可以使用最小的入射角度范围进行深度信息探测。
此外，如图4所示，深度信息图像采集装置还可以包括微透镜阵列130，该微透镜阵列130可以设置在相位调制结构110和像素点阵列120之间，用于将经过相位调制结构110调制后的光信号汇聚至像素点阵列120。
该微透镜阵列中的微透镜与光学传感器中的像素点具有一一对应的关系，即一个像素点可以对应一个微透镜，一个微透镜可以将相位调制结构调制后的光信号汇聚至对应的像素点。如图4所示，微透镜131与像素点121对应。
此外，深度信息图像采集装置还可以包括减反膜(anti-reflection coating，ARC)，减反膜有利于提高光学传感器接收的光信号的强度。该减反膜可以设置在深度信息图像采集装置中任一器件的表面。例如，为了提高通过相位调制结构的光量，可以在相位调制结构的上表面设置一层减反膜。又如，为了提高像素点阵列接收的光量，可以在光学传感器的表面设置一层减反膜。
作为一种实现方式，微透镜阵列130可以设置在相位调制结构110的下方，滤光结构140可以设置在微透镜阵列130和像素点阵列120之间，该像 素点阵列120的上表面可以设置一层减反膜。
深度信息图像采集装置还可以包括主透镜，该主透镜也可以称为成像透镜，该主透镜用于对物体进行成像，该主透镜可以设置在所有相位调制结构的上方。多个相位调制结构用于接收经过该主透镜的光信号，并对该光信号进行相位调制，光学传感器用于接收该经过相位调制后的光信号。
下面结合图8，对本申请实施例的深度信息图像采集装置中的器件的位置关系进行描述。
该深度信息图像采集装置200可以包括相位调制结构210、微透镜阵列220、滤光结构230、隔板240、减反膜250以及像素点阵列260。其中，相位调制结构210包括基板211以及设置在基板211上方的凸台212；像素点阵列260可以包括感光单元261。
该相位调制结构210可以设置在像素点阵列260的上方，微透镜阵列220可以设置在相位调制结构210与像素点阵列260之间，滤光结构230可以设置在微透镜阵列220与像素点阵列260之间，减反膜250可以设置在像素点阵列260的上表面。
此外，如图14所示，本申请实施例还提供一种电子设备400，该电子设备400深度信息图像采集装置410和处理器420，该深度信息图像采集装置410可以为上文描述的任一种深度信息图像采集装置。
该处理器410可用于根据一个像素点阵列中的像素点接收的光信号的强度，确定入射至对应的相位调制结构的光信号的入射角度。
本申请实施例中的入射角度可以包括方位角和倾斜角，深度信息图像采集装置可以探测入射光信号的方位角和倾斜角。对于凹槽的相位调制结构，如果探测两个方向的角度信息，则需要设计不同轴线方向的凹槽，并且这种方式会牺牲分辨率，且需要通过插值法计算得到两个方向的角度信息。而本申请采用凸台的相位调制结构，通过一个相位调制结构即可同时探测出物体的方位角和倾斜角，结构简单，不需要进行复杂的光学设计。
方位角可以表示入射光信号在相位调制结构所在平面的投影与该平面上的坐标轴的夹角，倾斜角可以表示光信号与垂直于相位调制结构所在平面的方向的夹角。
如图9和图10所示，θ表示倾斜角，
Figure PCTCN2020103458-appb-000015
表示方位角。且在图10中，X轴和Y轴表示凸台所在平面上的任意两个相互垂直的方向，Z轴表示垂直于 凸台所在平面的方向。倾斜角θ表示光信号与Z轴之间的夹角，方位角
Figure PCTCN2020103458-appb-000016
表示光信号与X轴之间的夹角。
对于3×3像素点阵列，X轴和Y轴优选为沿着像素点阵列的排列方向延伸的方向，如图4所示。对于2×2像素点阵列，X轴和Y轴优选为为沿着像素点阵列的对角线延伸的方向，如图11所示。
下面对方位角和倾斜角的确定过程进行描述。
基于三角函数关系，处理器410可以根据如下公式计算方位角
Figure PCTCN2020103458-appb-000017
Figure PCTCN2020103458-appb-000018
其中，X G表示X方向的像素点接收的光信号的强度在X轴上的重心坐标，Y G表示Y方向的像素点接收的光信号的强度在Y轴上的重心坐标，X方向与Y方向为像素点阵列所在平面上相互垂直的两个方向。
以一个像素点阵列为3×3像素点阵列为例，可以将中间像素点的位置坐标设置为零点位置(0,0)，处理器410可以通过以下公式计算重心坐标(X G，Y G)：
Figure PCTCN2020103458-appb-000019
Figure PCTCN2020103458-appb-000020
其中，X i,j表示第i行第j列的像素点在X方向上的坐标，Y i,j表示第i行第j列的像素点在Y方向上的坐标，P i,j表示第i行第j列的像素点接收的光信号的强度，i和j均为正整数。
在该情况下，X轴和Y轴均为与像素点的排列方向一致的方向，如图4所示。
以上是以像素点(1,2)、(2,1)、(2,3)、(3,2)为例计算重心坐标的，这样计算的重心坐标比较准确，但本申请实施例并不限于此，还可以使用其他像素点计算重心坐标。
例如，本申请实施例还可以使用以下公式计算重心坐标：
Figure PCTCN2020103458-appb-000021
Figure PCTCN2020103458-appb-000022
如果一个像素点阵列为2×2像素点阵列，所述处理器410用于通过以下公式计算重心坐标(X G，Y G)：
Figure PCTCN2020103458-appb-000023
Figure PCTCN2020103458-appb-000024
其中，X i,j表示第i行第j列的像素点在X方向上的坐标，Y i,j表示第i行第j列的像素点在Y方向上的坐标，P i,j表示第i行第j列的像素点接收的光信号的强度，i和j均为正整数。
在该情况下，X轴和Y轴为与像素点的排列方向呈45°夹角的方向，如图11所示。
采用公式(4)和公式(5)计算的重心坐标比较准确，但是本申请实施例并不限于此，例如，还可以使用以下公式计算重心坐标：
Figure PCTCN2020103458-appb-000025
Figure PCTCN2020103458-appb-000026
此外，对于一个像素点阵列为3×3像素点阵列的情况，方位角
Figure PCTCN2020103458-appb-000027
还可以通过索贝尔算子进行计算。
例如，处理器410可以根据以下公式计算方位角
Figure PCTCN2020103458-appb-000028
Figure PCTCN2020103458-appb-000029
其中，G y表示根据索贝尔算子(sobel operator)法得到的Y方向的强度梯度，G x表示根据索贝尔算子得到的X方向的强度梯度。
具体地，索贝尔算子可以如下：
Figure PCTCN2020103458-appb-000030
Figure PCTCN2020103458-appb-000031
Figure PCTCN2020103458-appb-000032
公式(9)中，P表示一个像素点阵列的像素点强度。
将公式(7)、公式(8)与公式(9)进行卷积运算，如公式(10)和公式(11)，得到X方向的强度梯度G x和Y方向的强度梯度G y，*表示卷积运算。
G x＝sobel x*P                    (10)
G y＝sobel y*P                    (11)
并且通过强度梯度可以得到入射光的方位角
Figure PCTCN2020103458-appb-000033
Figure PCTCN2020103458-appb-000034
利用索贝尔算子法，可以充分利用上述3×3像素点阵列中的所有像素点，提升探测角度信息的精确度。
在通过上述方式计算得到方位角
Figure PCTCN2020103458-appb-000035
之后，处理器410可以根据预先建立的θ与f(θ)之间的对应关系确定倾斜角θ，且处理器410可以根据以下公式确定f(θ)：
Figure PCTCN2020103458-appb-000036
Figure PCTCN2020103458-appb-000037
表示方位角，ΔP x表示X方向的归一化功率差，ΔP y表示Y方向的归一化功率差。
当入射光方位角
Figure PCTCN2020103458-appb-000038
为0度时，Y方向像素点光功率差与X方向像素点光功率差之和可以表示为关于倾斜角θ的函数，如下：
f(θ)＝ΔP y(θ,0)+ΔP x(θ,0)                (14)
入射光方位角
Figure PCTCN2020103458-appb-000039
为0度时的Y方向像素点光功率差与X方向像素点光功率差之和ΔP y(θ,0)+ΔP x(θ,0)与方位角
Figure PCTCN2020103458-appb-000040
为其他角度时的功率差之和
Figure PCTCN2020103458-appb-000041
满足一定的光功率守恒关系。
如下，通过仿真可以得到以下守恒关系：
Figure PCTCN2020103458-appb-000042
通过图12可以看出，
Figure PCTCN2020103458-appb-000043
和
Figure PCTCN2020103458-appb-000044
时，f(θ)的曲线基本吻合，因此，
Figure PCTCN2020103458-appb-000045
为0度时的归一化功率差之和与
Figure PCTCN2020103458-appb-000046
为45度时的归一化功率差之和满足上述功率守恒关系。
结合公式(14)和公式(15)，f(θ)可以表示为：
Figure PCTCN2020103458-appb-000047
归一化功率差ΔP x、ΔP y可以根据像素点阵列中的像素点接收的光功率计算得到，方位角
Figure PCTCN2020103458-appb-000048
可以根据上文描述的方式计算得到，从而可以根据公式(16)计算得到f(θ)。
θ与f(θ)之间的对应关系可以是通过计算机仿真或通过实验探测得到，在通过上述公式计算得到f(θ)后，可以根据该θ与f(θ)之间的对应关系得到θ。
确定归一化功率差的方式有多种，本申请实施例对此不做具体限定。
对于一个像素点阵列为3×3像素点阵列的情况，处理器410可用于根据以下公式确定ΔP x和ΔP y：
Figure PCTCN2020103458-appb-000049
Figure PCTCN2020103458-appb-000050
公式(17)和公式(18)仅是确定ΔP x和ΔP y的一种实现方式，通过这种方式计算得到的ΔP x和ΔP y比较准确，但本申请并不限于此，还可以通过以下公式来确定ΔP x和ΔP y。
例如：
Figure PCTCN2020103458-appb-000051
Figure PCTCN2020103458-appb-000052
又例如：
Figure PCTCN2020103458-appb-000053
Figure PCTCN2020103458-appb-000054
对于一个像素点阵列为2×2像素点阵列的情况，处理器410可用于根据以下公式确定ΔP x和ΔP y：
Figure PCTCN2020103458-appb-000055
Figure PCTCN2020103458-appb-000056
公式(21)和公式(22)仅是确定ΔP x和ΔP y的一种实现方式，通过这种方式计算得到的ΔP x和ΔP y比较准确，但本申请并不限于此，还可以通过以下公式来确定ΔP x和ΔP y。
例如：
Figure PCTCN2020103458-appb-000057
Figure PCTCN2020103458-appb-000058
在上述计算过程中，2×2像素点阵列使得整个相位调制结构单元使用总像素点较少，同样面积的成像光学传感器上可以放置更多的该相位调制结构单元用于参与角度探测，可提升具有深度信息的图片分辨率。3×3像素点阵列具有较高的角度探测范围和探测精度，但会牺牲一定图片空间分辨率。具体采用哪种像素点阵列可根据实际需要确定。
当然，像素点阵列也不限于上述两种形式，还可以采用4×4像素点阵列或5×5像素点阵列等。
由图可以看出，在倾斜角θ为0或接近于0的情况下，光信号是以基本垂直于凸台所在平面的方向入射，在该情况下，方位角
Figure PCTCN2020103458-appb-000059
可以是任意值。因此，为了提高计算的角度的精度，本申请实施例还提供了一种计算入射光角度信息的方法，如图13所示，该方法可以是由处理器410执行的。
本申请实施例可以先确定f(θ)的绝对值，判断f(θ)的绝对值是否小于一定阈值。如果f(θ)的绝对值小于该阈值，则可直接确定θ＝0，
Figure PCTCN2020103458-appb-000060
如果f(θ)的绝对值不小于该阈值，则根据上述公式计算方位角
Figure PCTCN2020103458-appb-000061
然后根据方位角
Figure PCTCN2020103458-appb-000062
和公式计算出f(θ)，再根据θ与f(θ)之间的对应关系求出θ。
本申请实施例可以将一个相位调制结构与其对应的像素点阵列统称为一个角度敏感探测单元。
角度敏感探测单元选择不同颜色滤光片会形成不同颜色的角度敏感探测单元，虽然凸台高度在一定范围就能产生上述能量分布现象，但是相同的凸台高度对不同波长的入射光探测方位角和倾斜角的精度有不同的影响，优选地可以通过数值仿真对不同波长角度敏感探测单元的凸台的高度进行优化。
相位调制结构距离像素点感光单元的高度H与入射光倾斜角探测范围强相关，过大的H会使f(θ)出现随入射光倾斜角θ出现类似周期性的分布，即多个倾斜角对应相同的f(θ)数值，这样对于相同的f(θ)数值，会反演产生多个θ值。因此，在设计H的大小时，在设计探测范围内需要将f(θ)保持单调。H的具体取值与凸台的周期、入射光的波长、像素点尺寸等多个因素相关，可通过仿真优化得出，H一般小于10um。
举例来说，通过仿真可以得出，像素点尺寸为2um，凸台横向尺寸与像素点尺寸之间的比值为1:1，凸台高度为0.344um，H为2.644um，可以获得倾斜角θ探测范围为±23度，方位角的探测精度小于±5度。
可以理解的是，本申请实施例中的探测角度范围可以指在该范围内，f(θ)对应的θ值唯一。
单个角度敏感探测单元包含的所有像素点的探测强度之和作为该角度敏感度探测单元整体的探测强度，多个角度敏感探测单元周期性地排布形成的新图像探测像素点阵列的强度值通过算法生成物体的图像。单个角度敏感探测单元通过本申请的角度探测方法，可以得到每个角度敏感探测单元入射光角度信息，即可同时得到入射方位角和倾斜角，利用该角度信息形成图像深度探测和三维图像重构。
下面对根据光信号的入射角度确定物体图像的深度信息的方式进行描述。
直观理解为，当角度敏感探测单元收集的光信号的入射角已知后，根据光路可逆原理，以及根据成像光学系统的物理参数，可以反推出成像物点的位置信息。但实际三维图像成像或重构过程，可以使用散焦测距(depth from defocus，DFD)算法。
使用理想薄透镜成像的原理示意图如图15所示。
Figure PCTCN2020103458-appb-000063
其中，z f为在理想成像过程中，成像物体平面与光学系统物方主平面之间的距离；z s为在理想成像过程中，感光传感器与光学系统像方主平面之间的距离，f代表成像光学系统的焦距。
光学系统成像时，将物平面上的点理想地成像于像平面，此时将感光传感器放置于像平面即可成清晰二维图像。
考虑成像空间中的任意一点同样满足高斯成像公式。其中u代表物距，v代表像距。
Figure PCTCN2020103458-appb-000064
当u≠z f时，该光学系统成模糊像，根据上述两式可以得出模糊像点的大小b w，通过在实际探测到的图像中计算b w，并通过下式可以得出实际物体距光学系统物方主点的距离u即成像物体的图像的深度信息即可得出。
Figure PCTCN2020103458-appb-000065
其中D为光瞳孔径。
需要指出的是，当物点位于对准平面(对准平面与理想成像物像平面为共轭面)前或者后时，可以产生相同的b w值，会给计算u带来未知因素，仅通过上述公式无法判断。如图15所示，在u＞z f时，物点的成像过程可以如图15中的实线所示，当u＜z f时，物点的成像过程可以如图15中的虚线所示，在这两种情况下，b w的大小相同，因此，仅根据b w无法判断是欠焦还是过焦，从而无法判断物距。
但是，结合上文描述的判断感光面上的入射光方向，尤其是倾斜角θ，可以清晰地判断出是欠焦还是过焦，从而准确地确定出物距，然后通过物距可以确定物体图像的深度信息。以图15为例，实线和虚线光信号入射至感光面处的光信号的方向正好相反，因此，可以根据入射光信号的方向能够判断出是欠焦还是过焦。
位于对准平面前后的物点的成像光束入射到感光传感器上的角度敏感探测单元时的入射角度，尤其是倾斜角θ可以清楚地分辨。
需要说明的是，在本申请实施例和所附权利要求书中使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请实施例。
例如，在本申请实施例和所附权利要求书中所使用的单数形式的“一 种”、“所述”、“上述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。
所属领域的技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请实施例的范围。
如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实施例的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器、随机存取存储器、磁碟或者光盘等各种可以存储程序代码的介质。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的设备、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请提供的几个实施例中，应该理解到，所揭露的电子设备、装置和方法，可以通过其它的方式实现。
例如，以上所描述的装置实施例中单元或模块或组件的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如，多个单元或模块或组件可以结合或者可以集成到另一个系统，或一些单元或模块或组件可以忽略，或不执行。
又例如，上述作为分离/显示部件说明的单元/模块/组件可以是或者也可以不是物理上分开的，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元/模块/组件来实现本申请实施例的目的。
最后，需要说明的是，上文中显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
以上内容，仅为本申请实施例的具体实施方式，但本申请实施例的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请实施例揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请实施例的保护范围之内。因此，本申请实施例的保护范围应以权利要求的保护范围为准。

Claims (31)

  1. 一种深度信息图像采集装置，其特征在于，包括：
    多个二维光学相位调制结构，所述多个二维光学相位调制结构中的每个二维光学相位调制结构包括基板以及设置在所述基板上的凸台，所述二维光学相位调制结构用于对物体反射的光信号进行空间相位调制；
    光学传感器，设置在所述多个二维光学相位调制结构的下方，所述光学传感器包括多个像素点阵列，所述多个像素点阵列与所述多个二维光学相位调制结构具有一一对应关系，所述多个像素点阵列中的一个像素点阵列用于接收经过与其对应的二维光学相位调制结构调制后的光信号，所述一个像素点阵列包括多个像素点，所述多个像素点接收的光信号的强度分布用于确定入射至所述对应的二维光学相位调制结构的光信号的入射角度，所述入射角度用于确定所述物体图像的深度信息。
  2. 根据权利要求1所述的装置，其特征在于，所述凸台的横截面为中心对称图形。
  3. 根据权利要求1或2所述的装置，其特征在于，所述凸台为圆柱形结构。
  4. 根据权利要求1-3中任一项所述的装置，其特征在于，所述凸台的中心与与其对应的像素点阵列的中心重合。
  5. 根据权利要求1-3中任一项所述的装置，其特征在于，所述凸台的中心相对于与其对应的像素点阵列的中心具有偏移量，使得所述对应的像素点阵列确定的入射角度为光信号的实际入射角度-主光线入射角。
  6. 根据权利要求1-5中任一项所述的装置，其特征在于，所述凸台的横向尺寸与一个像素点的尺寸之间的比值介于0.5～1.5之间。
  7. 根据权利要求1-6中任一项所述的装置，其特征在于，所述凸台的高度位于λ/4～5λ之间，λ为入射至所述凸台的光信号在真空中的波长。
  8. 根据权利要求7所述的装置，其特征在于，所述多个二维光学相位调制结构与光学传感器之间的距离H小于10μm。
  9. 根据权利要求1-8中任一项所述的装置，其特征在于，一个二维光学相位调制结构中的基板与与其对应的像素点阵列在垂直于所述基板表面的方向上的投影重叠。
  10. 根据权利要求1-9中任一项所述的装置，其特征在于，所述一个像素点阵列为N×N像素点阵列，N为大于2的正整数。
  11. 根据权利要求1-9中任一项所述的装置，其特征在于，所述一个像素点阵列为3×3像素点阵列。
  12. 根据权利要求1-9中任一项所述的装置，其特征在于，所述一个像素点阵列为2×2像素点阵列。
  13. 根据权利要求1-12中任一项所述的装置，其特征在于，所述一个像素点阵列接收的光信号的总和用于生成所述物体的图像。
  14. 根据权利要求1-13中任一项所述的装置，其特征在于，所述光学传感器为背照式光学传感器。
  15. 根据权利要求1-14中任一项所述的装置，其特征在于，所述多个二维光学相位调制结构中的凸台的几何尺寸均相同。
  16. 根据权利要求1-14中任一项所述的装置，其特征在于，所述多个二维光学相位调制结构包括至少两种几何尺寸的凸台。
  17. 根据权利要求1-16中任一项所述的装置，其特征在于，还包括滤光结构，所述滤光结构设置在所述多个二维光学相位调制结构与所述光学传感器之间，所述滤光结构用于对所述物体反射的光信号进行过滤。
  18. 根据权利要求17所述的装置，其特征在于，一个二维光学相位调制结构对应一种颜色的滤光片。
  19. 根据权利要求17或18所述的装置，其特征在于，所述滤光结构包括马赛克滤光片。
  20. 根据权利要求17-19中任一项所述的装置，其特征在于，所述滤光结构包括拜耳滤光阵列。
  21. 根据权利要求17-20中任一项所述的装置，其特征在于，所述滤光结构包括不同颜色的滤光片，不同颜色的滤光片对应的二维光学相位调制结构的凸台的几何尺寸不同，使得不同二维光学相位调制结构能够探测的光信号入射角度的范围相同。
  22. 根据权利要求1-21中任一项所述的装置，其特征在于，还包括微透镜阵列，所述微透镜阵列设置在所述多个二维光学相位调制结构与所述光学传感器之间，所述微透镜阵列中的微透镜与所述光学传感器中的像素点具有一一对应的关系，所述微透镜阵列用于将经过所述多个二维光学相位调制 结构调制后的光信号汇聚至所述光学传感器。
  23. 根据权利要求1-22中任一项所述的装置，其特征在于，所述入射角度包括方位角和倾斜角，所述倾斜角表示光信号与垂直于所述多个二维光学相位调制结构所在平面的方向的夹角，所述方位角表示光信号在所述多个二维光学相位调制结构所在平面的投影与所述平面上的坐标轴的夹角。
  24. 一种电子设备，其特征在于，包括：
    如权利要求1-23所述的深度信息图像采集装置；以及
    处理器，连接至所述深度信息图像采集装置，所述处理器用于根据所述一个像素点阵列中的像素点接收的光信号的强度，确定入射至所述对应的二维光学相位调制结构的光信号的入射角度，所述入射角度包括方位角和倾斜角。
  25. 根据权利要求24所述的电子设备，其特征在于，所述处理器用于根据以下公式计算方位角
    Figure PCTCN2020103458-appb-100001
    Figure PCTCN2020103458-appb-100002
    其中，X G表示X方向的像素点接收的光信号的强度在X轴上的重心坐标，Y G表示Y方向的像素点接收的光信号的强度在X轴上的重心坐标，X方向与Y方向为二维光学相位调制结构所在平面上相互垂直的两个方向。
  26. 根据权利要求25所述的电子设备，其特征在于，所述一个像素点阵列为3×3像素点阵列，所述处理器用于通过以下公式计算重心坐标(X G，Y G)：
    Figure PCTCN2020103458-appb-100003
    Figure PCTCN2020103458-appb-100004
    其中，X i,j表示第i行第j列的像素点在X方向上的坐标，Y i,j表示第i行第j列的像素点在Y方向上的坐标，P i,j表示第i行第j列的像素点接收的光信号的强度，i和j均为正整数。
  27. 根据权利要求25所述的电子设备，其特征在于，所述一个像素点阵列为2×2像素点阵列，所述处理器用于通过以下公式计算重心坐标(X G，Y G)：
    Figure PCTCN2020103458-appb-100005
    Figure PCTCN2020103458-appb-100006
    其中，X i,j表示第i行第j列的像素点在X方向上的坐标，Y i,j表示第i行第j列的像素点在Y方向上的坐标，P i,j表示第i行第j列的像素点接收的光信号的强度，i和j均为正整数。
  28. 根据权利要求24所述的电子设备，其特征在于，所述一个像素点阵列为3×3像素点阵列，所述处理器用于根据以下公式计算方位角
    Figure PCTCN2020103458-appb-100007
    Figure PCTCN2020103458-appb-100008
    其中，G y表示根据索贝尔算子得到的Y方向的强度梯度，G x表示根据索贝尔算子得到的X方向的强度梯度。
  29. 根据权利要求25-28中任一项所述的电子设备，其特征在于，所述处理器用于根据预先建立的θ与f(θ)之间的对应关系确定倾斜角θ，且所述处理器用于根据以下公式确定f(θ)：
    Figure PCTCN2020103458-appb-100009
    Figure PCTCN2020103458-appb-100010
    表示方位角，ΔP x表示X方向的归一化功率差，ΔP y表示Y方向的归一化功率差。
  30. 根据权利要求29所述的电子设备，其特征在于，所述一个像素点阵列为3×3像素点阵列，所述处理器用于通过以下公式确定X方向和Y方向的像素点的归一化功率差：
    Figure PCTCN2020103458-appb-100011
    Figure PCTCN2020103458-appb-100012
    P i,j表示第i行第j列的像素点接收的光信号的强度。
  31. 根据权利要求29所述的电子设备，其特征在于，所述一个像素点阵列为2×2像素点阵列，所述处理器用于通过以下公式确定X方向和Y方向的像素点的归一化功率差：
    Figure PCTCN2020103458-appb-100013
    Figure PCTCN2020103458-appb-100014
    P i,j表示第i行第j列的像素点接收的光信号的强度。
PCT/CN2020/103458 2020-07-22 2020-07-22 深度信息图像采集装置和电子设备 Ceased WO2022016412A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202080034583.3A CN114365019B (zh) 2020-07-22 2020-07-22 深度信息图像采集装置和电子设备
PCT/CN2020/103458 WO2022016412A1 (zh) 2020-07-22 2020-07-22 深度信息图像采集装置和电子设备

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/103458 WO2022016412A1 (zh) 2020-07-22 2020-07-22 深度信息图像采集装置和电子设备

Publications (1)

Publication Number Publication Date
WO2022016412A1 true WO2022016412A1 (zh) 2022-01-27

Family

ID=79728387

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/103458 Ceased WO2022016412A1 (zh) 2020-07-22 2020-07-22 深度信息图像采集装置和电子设备

Country Status (2)

Country Link
CN (1) CN114365019B (zh)
WO (1) WO2022016412A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117392316A (zh) * 2023-10-13 2024-01-12 清华大学 基于系列欠焦图像的三维重构方法及装置
WO2024236137A3 (en) * 2023-05-17 2025-02-20 Ams-Osram Ag Optical camera sensor, camera system and method for multi-spectral light sensing

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116452433B (zh) * 2023-03-10 2025-10-03 浙江大学 用于光存储点阵读取的基于Sobel滤波的检测方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120091372A1 (en) * 2008-07-25 2012-04-19 Cornell University Light field image sensor, method and applications
CN109716176A (zh) * 2016-06-07 2019-05-03 艾瑞3D 有限公司 用于深度采集和三维成像的光场成像装置和方法
TW201925860A (zh) * 2017-12-05 2019-07-01 加拿大商艾瑞３Ｄ股份有限公司 用於深度獲取之光場影像處理方法
CN110612429A (zh) * 2018-04-10 2019-12-24 深圳市汇顶科技股份有限公司 三维影像测距系统及方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101086527B (zh) * 2007-07-11 2010-08-25 浙江大学 一种探测入射激光方向的方法及信号探测装置
CN109884742B (zh) * 2019-04-03 2025-02-11 深圳市灵明光子科技有限公司 光场成像装置和光场成像方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120091372A1 (en) * 2008-07-25 2012-04-19 Cornell University Light field image sensor, method and applications
CN109716176A (zh) * 2016-06-07 2019-05-03 艾瑞3D 有限公司 用于深度采集和三维成像的光场成像装置和方法
TW201925860A (zh) * 2017-12-05 2019-07-01 加拿大商艾瑞３Ｄ股份有限公司 用於深度獲取之光場影像處理方法
CN110612429A (zh) * 2018-04-10 2019-12-24 深圳市汇顶科技股份有限公司 三维影像测距系统及方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024236137A3 (en) * 2023-05-17 2025-02-20 Ams-Osram Ag Optical camera sensor, camera system and method for multi-spectral light sensing
CN117392316A (zh) * 2023-10-13 2024-01-12 清华大学 基于系列欠焦图像的三维重构方法及装置

Also Published As

Publication number Publication date
CN114365019B (zh) 2024-09-20
CN114365019A (zh) 2022-04-15

Similar Documents

Publication Publication Date Title
US12474509B2 (en) Light field imaging device and method for depth acquisition and three-dimensional imaging
CN206759600U (zh) 成像系统
EP3423858B1 (en) 3d imaging system and method
US12259563B2 (en) Light field imaging device and method for 3D sensing
CN116034480A (zh) 成像装置和电子装置
US12253414B2 (en) Image sensor and preparation method thereof, and electronic device
US20250040267A1 (en) Spectral Element Array, Image Sensor and Image Apparatus
EP2965134A1 (en) Phase gratings with odd symmetry for high-resoultion lensless optical sensing
CN115516283A (zh) 偏振成像摄像头
CN101842908A (zh) 透射检测器、包含相同透射检测器的系统和相关方法
KR102827127B1 (ko) 이미지 센서 및 이를 포함하는 전자 장치
CN114365019B (zh) 深度信息图像采集装置和电子设备
TWI676393B (zh) 影像感測器
WO2020233601A1 (zh) 成像层、成像装置、电子设备、波带片结构及感光像元
WO2022104629A1 (zh) 一种图像传感器、分光滤色器件及图像传感器的制备方法
WO2022027194A1 (zh) 图像传感器、图像处理方法及装置、成像系统
US20250097597A1 (en) Phase detection autofocus pixel
CA3026789C (en) Light field imaging device and method for depth acquisition and three-dimensional imaging

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20946346

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20946346

Country of ref document: EP

Kind code of ref document: A1