WO2019196001A1 - 三维影像测距系统及方法 - Google Patents

三维影像测距系统及方法 Download PDF

Info

Publication number
WO2019196001A1
WO2019196001A1 PCT/CN2018/082441 CN2018082441W WO2019196001A1 WO 2019196001 A1 WO2019196001 A1 WO 2019196001A1 CN 2018082441 W CN2018082441 W CN 2018082441W WO 2019196001 A1 WO2019196001 A1 WO 2019196001A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
phase
light
pixel
target object
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/082441
Other languages
English (en)
French (fr)
Inventor
杨孟达
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Goodix Technology Co Ltd
Original Assignee
Shenzhen Goodix Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Goodix Technology Co Ltd filed Critical Shenzhen Goodix Technology Co Ltd
Priority to PCT/CN2018/082441 priority Critical patent/WO2019196001A1/zh
Priority to CN201880000670.XA priority patent/CN110612429B/zh
Publication of WO2019196001A1 publication Critical patent/WO2019196001A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object

Definitions

  • the present application relates to a three-dimensional image ranging system and method, and more particularly to a three-dimensional image ranging system and method that can calculate the fine position of structured light.
  • the structured light three-dimensional measurement method as a non-contact three-dimensional information acquisition technology, has been widely used due to its advantages of simple implementation, high speed and high precision.
  • the basic idea of structured light 3D measurement is to use the geometric relationship of structured light projection to obtain three-dimensional information of objects. Firstly, the encoded structural optical template is projected onto the object to be tested by the projection device, and the projected image is recorded by the camera, and the captured image is matched with the projected structural optical template. After the matching point is found, the projection point and the matching point are utilized. And the triangular relationship of the object to solve the three-dimensional information of the target object.
  • the existing structured light three-dimensional measuring system is not precise enough to calculate the coordinate position of the object in the captured image, and limits the accuracy of its three-dimensional information. Therefore, there is a need for improvement in the prior art.
  • an embodiment of the present application provides a three-dimensional image ranging system, including a light-emitting module, including a diffraction unit, and a first light-emitting unit for emitting first light to the diffraction in a first time.
  • the diffraction unit generates a first structured light by diffracting the first light in the first time; and a second light emitting unit configured to emit the second light to the second time a diffraction unit that forms a second structured light by diffracting the second light during the second time, wherein between the first structured light and the second structured light
  • the phase difference of the structured light is an odd multiple of ( ⁇ /2);
  • the photosensitive pixel array is configured to receive the reflected light corresponding to the first structured light at the first time to generate a first image, and in the Receiving, by the second time, the reflected light corresponding to the second structured light to generate a second image;
  • the computing unit is coupled to the photosensitive pixel array for performing the following steps: according to the first image and the second An image associated with the first structured light In-phase and quadrature video image in relation to the second structure of the light; and a depth image according to the image and the quadrature-phase image is generated corresponding to the target object.
  • the diffraction unit includes a first diffrative subunit and a second diffractive subunit, and the first diffractive subunit generates a diffraction effect on the first light in the first time to generate a Depicting the first structured light, the second diffractive sub-unit forms a diffracting effect on the second light in the second time to generate the second structured light.
  • the first lighting unit receives the first pulse signal and emits the first light according to the first pulse signal
  • the second lighting unit receives the second pulse signal and emits according to the second pulse signal The second light.
  • the duty ratio of the first pulse signal and the second pulse signal is less than a specific duty ratio, and the luminous powers of the first light emitting unit and the second light emitting unit are greater than a specific power.
  • the first light emitted by the first light emitting unit has a first incident angle with respect to the diffraction unit
  • the second light emitted by the second light emitting unit is relative to the diffraction unit Having a second incident angle, the first incident angle being different from the second incident angle, such that the phase difference between the structured light between the first structured light and the second structured light is ( ⁇ /2) An odd multiple.
  • the photosensitive pixel array includes a plurality of photosensitive pixel circuits, a first photosensitive pixel circuit of the plurality of photosensitive pixel circuits includes a photosensitive element, and the first photoelectric reading circuit is coupled to the photosensitive element, including: the first transmission a gate coupled to the photosensitive element, wherein the first transmission gate is turned on at the first time; a first output transistor coupled to the first transfer gate; and a first read transistor coupled The first output transistor is configured to output a first output signal; and the second photoelectric reading circuit is coupled to the photosensitive element, and includes: a second transfer gate coupled to the photosensitive element, wherein the The second transmission gate is turned on at the second time; the second output transistor is coupled to the second transfer gate; and the second read transistor is coupled to the second output transistor for outputting the second An output signal, wherein a first pixel value corresponding to the first photosensitive pixel circuit in the first image is the first output signal, and the second image corresponds to the first photosensitive pixel circuit
  • One second The prime value is the second output signal
  • the first photosensitive pixel circuit further includes: a third photoelectric reading circuit coupled to the photosensitive element, comprising: a third transfer gate coupled to the photosensitive element, wherein the third transfer gate is The third time is turned on, in the third time, the first light emitting unit and the second light emitting unit are not illuminated; the third output transistor is coupled to the third transfer gate; and the third read a transistor, coupled to the third output transistor, for outputting a third output signal, wherein the first in-phase pixel value corresponding to the first photosensitive pixel circuit in the in-phase image is related to Subtracting the third output signal from a first output signal, wherein the first orthogonal pixel value corresponding to the first photosensitive pixel circuit in the orthogonal image is related to the second output signal minus the first a three output signal; wherein the plurality of third output signals output by the plurality of photosensitive pixel circuits form a third image.
  • a third photoelectric reading circuit coupled to the photosensitive element, comprising: a third transfer gate coupled to the photosensitive
  • the first photosensitive pixel circuit further includes a fourth photoelectric reading circuit coupled to the photosensitive element, including: a fourth transfer gate coupled to the photosensitive element, wherein the fourth transfer gate is The fourth time is turned on, the first time is separated from the fourth time by a time interval; the fourth output transistor is coupled to the fourth transfer gate; and the fourth read transistor is coupled to the fourth output a transistor for outputting a fourth output signal, wherein the first in-phase pixel value corresponding to the first photosensitive pixel circuit in the in-phase image is related to the first output signal and the fourth output signal; And the plurality of fourth output signals output by the plurality of photosensitive pixel circuits form a fourth image; wherein the operation unit acquires a fly corresponding to the target object according to the first image and the fourth image Time distance.
  • a fourth photoelectric reading circuit coupled to the photosensitive element, including: a fourth transfer gate coupled to the photosensitive element, wherein the fourth transfer gate is The fourth time is turned on, the first time is separated from the fourth time by a time interval; the
  • the operation unit is configured to perform the following steps to generate the depth image corresponding to the target object according to the in-phase image and the orthogonal image: according to the in-phase image and the orthogonal image, Generating a phase image, wherein the phase image represents an image phase between the in-phase image formed by the first structured light and the orthogonal image formed by the second structured light; according to the phase image, Generating a first light stripe image corresponding to the first phase angle image, wherein the first light stripe image records the image phase as a coordinate position of the first phase angle; and generating a corresponding image according to the first light stripe image The depth image of the target object.
  • the first phase angle is zero.
  • the operation unit is configured to perform the following steps to generate the first light stripe image corresponding to the first phase angle according to the phase image: obtaining a position of the phase image in a first pixel coordinate position a first phase image pixel value, and obtaining a second phase image pixel value in the second pixel coordinate position in the phase image, wherein the first pixel coordinate position is directly adjacent to the second pixel coordinate in the first dimension a position determining whether the first phase angle is between the first phase image pixel value and the second phase image pixel value; when the first phase angle is between the first phase image pixel value and Performing an interpolation operation to obtain an interpolation result according to the first phase angle, the first phase image pixel value, and the second phase image pixel value when the second phase image pixel value is between;
  • the interpolation result is stored in the first pixel coordinate position of the first light stripe image, and the interpolation result becomes the first pixel coordinate in the first light stripe image Streaks of light of the image pixel value is set.
  • the operation unit is configured to perform the following steps to perform the interpolation operation to obtain the inner phase according to the first phase angle, the first phase image pixel value, and the second phase image pixel value.
  • Insert result calculate the interpolation result as Wherein ⁇ represents the first phase angle, (n, m) represents the first pixel coordinate position, (n-1, m) represents the second pixel coordinate position, and PHI(n, m) represents the The first phase image pixel value, PHI(n-1, m), the second phase image pixel value.
  • the operation unit is further configured to perform the following steps to generate the first light stripe image corresponding to the first phase angle according to the phase image: when the first phase angle is not between the When the first phase image pixel value is between the phase image and the second phase image pixel value, the light stripe image pixel value corresponding to the first pixel coordinate position in the first light stripe image is 0.
  • the target object corresponds to the first three-dimensional image coordinate of the third pixel coordinate position; and the depth image pixel value corresponding to the third
  • the operation unit is further configured to perform the following steps to generate the depth image corresponding to the target object according to the in-phase image and the orthogonal image: corresponding to at least one according to the phase image At least one second stripe image of the second phase angle, wherein the at least one second phase angle is different from the first phase angle, and the at least one second stripe image is different from the first stripe image Phases; and integrating the first stripe image and the at least one second stripe image into an integrated image; and generating the depth image corresponding to the target object based on the integrated image.
  • the at least one second phase angle is an integer multiple of (2 ⁇ /L), and L is a positive integer greater than one.
  • the operation unit is configured to perform the following steps to integrate the first light stripe image and the at least one second light stripe image into the integrated image: generating the integrated image as the first light stripe And an addition result of the image and the at least one second light stripe image.
  • the photosensitive pixel array receives background light at a third time to generate a third image
  • the photosensitive pixel array receives reflected light corresponding to the first structured light to generate a fourth image in a fourth time
  • the computing unit is further configured to: generate the depth image corresponding to the target object according to the in-phase image and the orthogonal image: generating a correspondence according to the first image and the fourth image a time-of-flight distance of the target object; determining an angle according to the flying time distance and the first light stripe image, wherein the angle represents between the target object, the light emitting module, and the photosensitive pixel array An angle of the image; and the depth image corresponding to the target object is generated based on the integrated image and the angle.
  • the dimensional image coordinates are coordinate values of the first dimension
  • x 0 represents the coordinate value of the two-dimensional image coordinate in the second dimension
  • n represents the coordinate value of the fourth pixel coordinate position in the first dimension
  • m represents
  • the fourth pixel coordinate position is a coordinate value of the second dimension
  • MG(n, m) represents an integrated image pixel value of the integrated image at a fourth pixel coordinate position; according to the second two-dimensional image coordinate and Calculating a second three-dimensional image coordinate of the target object corresponding to the third pixel coordinate position; and calculating a depth of the target object
  • the embodiment of the present application further provides a three-dimensional image ranging method, which is applied to a three-dimensional image ranging system, wherein the three-dimensional image ranging system includes a light-emitting module and a photosensitive pixel array, The light emitting module emits first structured light in a first time and second structured light in a second time, the photosensitive pixel array receiving reflected light corresponding to the first structured light at the first time to generate a first image, and receiving reflected light corresponding to the second structured light at the second time to generate a second image, wherein a structural light phase difference between the first structured light and the second structured light
  • the three-dimensional image ranging method includes: generating an in-phase image related to the first structured light according to the first image and the second image, and related to the first An orthogonal image of the two structured lights; and generating a depth image corresponding to the target object based on the in-phase image and the orthogonal image.
  • the embodiment of the present application calculates the image phase by using the first structured light and the second structured light having a structured optical phase difference of ( ⁇ /2) or (3 ⁇ /2); and calculates the stripe structured light according to the image phase (
  • the stripe-like structure is light perpendicular to the fine position of the first direction/dimension. Compared with the prior art, the present application can accurately obtain the depth information of the target object.
  • FIG. 1 is a schematic diagram of the appearance of a three-dimensional image ranging system according to an embodiment of the present application
  • FIG. 2 is a functional block diagram of the 3D image ranging system of FIG. 1;
  • FIG. 3 is a schematic diagram of a diffraction unit according to an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a first structured light and a second structured light according to an embodiment of the present application
  • FIG. 5 is a schematic diagram of a process according to an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a photosensitive pixel circuit according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a photosensitive pixel circuit according to an embodiment of the present application.
  • FIG. 8 is a schematic diagram of a photosensitive pixel circuit according to an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a process according to an embodiment of the present application.
  • FIG. 11 is a schematic diagram of a phase image according to an embodiment of the present application.
  • FIG. 12 is a schematic diagram of a process according to an embodiment of the present application.
  • FIG. 13 is a schematic diagram of a light stripe image according to an embodiment of the present application.
  • FIG. 14 is a schematic diagram of a process according to an embodiment of the present application.
  • 15 is a schematic diagram of a process according to an embodiment of the present application.
  • 16 is a schematic diagram of a light stripe image according to an embodiment of the present application.
  • 17 is a schematic diagram of a light stripe image according to an embodiment of the present application.
  • FIG. 18 is a schematic diagram of a light stripe image according to an embodiment of the present application.
  • FIG. 19 is a schematic diagram of an integrated image according to an embodiment of the present application.
  • FIG. 20 is a schematic diagram of a process according to an embodiment of the present application.
  • 21 is a relative positional relationship diagram of a light emitting module and a photosensitive module according to an embodiment of the present application
  • 22 is a schematic diagram of a process according to an embodiment of the present application.
  • FIG. 23 is a schematic diagram of a process according to an embodiment of the present application.
  • 24 is a schematic diagram of a light emitting module according to an embodiment of the present application.
  • Fig. 25 is a schematic view showing the light of a stripe structure projected onto an object.
  • the addition, subtraction, multiplication and division between the image A and the image B represents the addition, subtraction, multiplication and division of pixels and pixels by the image A and the image B.
  • A(n, m), B(n, m), and C(n, m) represent the pixel values of the image A, the image B, and the image C at the pixel coordinate position (n, m), respectively, and the image C.
  • Performing the function f on the image A represents performing a function f operation on each element in the image A.
  • m) f(A(n, m)).
  • n is a row pointer (Row Index) in the image
  • m is a column pointer (Column Index) in the image.
  • the unit of angle or phase is radians (also known as ⁇ , ie, Radian).
  • the 3D image ranging system can measure the distance using stripe structured light and triangulation.
  • Figure 25 is a schematic diagram of the stripe structure light projected onto the object (palm).
  • the stripe structured light is perpendicular to the first direction D1 (or the first dimension D1) and parallel to the second direction D2 (or the second dimension D2).
  • the stripe-shaped structure light when projected on the object, it is distorted by the shape of the object, and the image corresponding to the stripe-shaped structure light has a large change in the first direction/dimension D1.
  • calculating the coordinate position of the stripe-shaped structured light image in the first direction/dimension D1 is limited to the pixel itself, and cannot be more fine, and affects the distance calculated by the triangulation method.
  • the present application can calculate the fine position of the stripe-shaped structured light in the first direction/dimension D1, thereby accurately obtaining the depth information of the object.
  • FIG. 1 is a schematic diagram of the appearance of a three-dimensional image ranging system 10 according to an embodiment of the present application
  • FIG. 2 is a functional block diagram of the three-dimensional image ranging system 10.
  • the 3D image ranging system 10 can be disposed in the electronic device 1.
  • the electronic device 1 can be a smart phone or a tablet computer.
  • the 10 comprises a three-dimensional video ranging system 12, a photosensitive unit 16 and a calculation module 14, the light emitting module 12 of the first configuration of the optical emitting module SL1 to emit a first time T 1, the structure and emitting a second light SL2 in the second time T 2
  • the structured optical phase difference is an odd multiple of ( ⁇ /2), for example, the structured optical phase difference may be ( ⁇ /2) or Yes (3 ⁇ /2).
  • the photosensitive module 14 may include a lens (Lens) 146 and a photosensitive pixel array 142.
  • the photosensitive pixel array 142 includes a plurality of photosensitive pixel circuits 144 arranged in an array.
  • the photosensitive pixel array 142 is configured to receive the first structure corresponding to the first structure at the first time T 1 .
  • the light of the light SL1 is reflected to generate a first image P1, and the reflected light corresponding to the second structured light SL2 is received at a second time T 2 to generate a second image P2.
  • the computing unit 16 is coupled to the photosensitive pixel array 142, which may include a processor or a differential operational amplifier.
  • the computing unit 16 is configured to receive the first image P1 and the second image P2 according to the first image P1 and the second image P2. A depth image corresponding to the target object is generated.
  • the light emitting module 12 includes a diffraction unit DE, a first light emitting unit LE1, and a second light emitting unit LE2.
  • the diffraction unit DE may include a single diffraction optical element (DOE), a first light emitting unit LE1, and a second light emitting unit.
  • the LE2 can be a light emitting diode (LED) or a laser (Laser) firing unit.
  • the first light emitting unit LE1 and the second light emitting unit LE2 can emit strong light instantaneously (similar to a flash of a general camera), and the first light L1 and the second light L1 can be Pulse Modulated light, so that the structured light SL1 is The optical signal of SL2 is not easily interfered by the background light.
  • the first lighting unit LE1 receives the first pulse signal pm1 and emits the first light L1 according to the first pulse signal pm1
  • the second lighting unit LE2 receives the second pulse signal pm2 and transmits the second pulse signal pm2 according to the second pulse signal pm2.
  • the duty ratios of the first pulse signal pm1 and the second pulse signal pm2 are smaller than a specific duty ratio, and the light-emitting powers of the first light-emitting unit LE1 and the second light-emitting unit LE2 are greater than a specific power.
  • the duty ratios of the first pulse signal pm1 and the second pulse signal pm2 may be less than 1/50 or less than 1/1000, and the illumination powers of the first illumination unit LE1 and the second illumination unit LE2 may be greater than 4 watts. .
  • FIG. 3 is a schematic diagram of a first structured light SL1 and a second structured light SL2 according to an embodiment of the present application.
  • the first structured light SL1 and the second structured light SL2 are respectively represented by black stripes and diagonally hatched stripes, and the light stripes of the first structured light SL1 and the light stripes of the second structured light SL2 are respectively used.
  • (Light Strip) is perpendicular to the first direction/dimension D1 and parallel to the second direction/dimension D2.
  • the black stripe and the hatched hatching strip represent the lightest stripe of the first structured light SL1 and the second structured light SL2, respectively, and the curves CV1 and CV2 in the sub-picture 3b represent the first structured light SL1 and the first
  • the intensity distribution curve of the two structured light SL2 along the first direction D1, and the curves CV1 and CV2 are all sinusoidal curves.
  • curve CV2 can be considered to be delayed by ( ⁇ /2) compared to curve CV1 (or curve CV1 can be advanced (3 ⁇ /2) compared to curve CV2), ie second
  • the structured optical phase of the structured light SL2 is delayed by ( ⁇ /2) compared to the structured optical phase of the first structured light SL1 (or the structured optical phase of the first structured light SL1 is earlier than the structured optical phase of the second structured light SL2 ( 3 ⁇ /2)), therefore, the curve CV1 can be regarded as a sine wave (Sine Wave), and the curve CV2 can be regarded as a cosine wave (Cosine Wave).
  • the light stripe (black stripe) corresponding to the first structured light SL1 is separated from each other by a distance PD, and the phase of the structured light of the second structured light SL2 is delayed relative to the phase of the structured light of the first structured light SL1 ( ⁇ / 2)
  • the light stripe representing the second structured light SL2 ie, the hatched hatching
  • PD may represent the period of the curve CV1.
  • FIG. 4 is a schematic diagram of a first light emitting unit LE1, a second light emitting unit LE2, and a diffraction unit DE according to an embodiment of the present application.
  • the second light L2 has a second incident angle ⁇ 2 with respect to the diffraction unit DE after passing through the collimator CM, wherein the first incident angle ⁇ 1 is different from the second incident angle ⁇ 2 , and the first incident angle ⁇ 1 and the second can be adjusted
  • the incident angle ⁇ 2 is such that the phase of the structured light of the second structured light SL2 is delayed ( ⁇ /2) from the phase of the structured light of the first structured light SL1, that is, between the first structured light SL1 and the second structured light SL2.
  • the structural light phase difference is ( ⁇ /2).
  • the operation unit 16 may be based on the first image P1 corresponding to the reflected light of the first structured light SL1 and the second image corresponding to the reflected light of the second structured light SL2.
  • P2 generating an in-phase image I related to the first structured light SL1 and a quadrature image Q related to the second structured light SL2, and calculating according to the in-phase image I and the orthogonal image Q
  • the image phase between the in-phase image I and the orthogonal image Q, and according to the image phase, a depth image corresponding to the target object is generated.
  • the pixel values of different rows in the image represent the pixel values of the different positions corresponding to the first direction/dimension D1 captured by the photosensitive module 14, and the columns in the image are different.
  • the pixel value represents a pixel value photographed by the photosensitive module 14 at a different position corresponding to the second direction/dimension D2.
  • FIG. 5 is a schematic diagram of a process A0 according to an embodiment of the present application.
  • the flow A0 can be executed by the operation unit 16, and the flow A0 includes the following steps:
  • Step A02 Generate an in-phase image I related to the first structured light SL1 and an orthogonal image Q related to the second structured light SL2 according to the first image P1 and the second image P2.
  • Step A04 Generate a depth image DP corresponding to the target object OBJ based on the in-phase image I and the orthogonal image Q.
  • step A02 the arithmetic unit 16 generates the in-phase image I according to the first image P1 and generates the orthogonal image Q according to the second image P2.
  • FIG. 6 , FIG. 7 and FIG. 8 are respectively schematic diagrams of a photosensitive pixel circuit 60 , a photosensitive pixel circuit 70 and a photosensitive pixel circuit 80 according to an embodiment of the present application.
  • the photosensitive pixel circuits 60, 70, 80 can be used to implement the photosensitive pixel circuit 144.
  • the photosensitive pixel circuits 60, 70, 80 can be photosensitive pixel circuits at the pixel coordinate position (n, m) in the photosensitive pixel array 142 (ie, It is the (n, m)th photosensitive pixel circuit).
  • the photosensitive pixel circuit 60 includes a photosensitive element PD and photoelectric reading circuits 61 and 62, and the photosensitive element PD may be a photodiode.
  • the photoelectric reading circuit 61 includes a transfer gate TG1, an output transistor DV1, and a read transistor RD1.
  • the photo-electric read circuit 62 includes a transfer gate TG2, an output transistor DV2, and a read transistor RD2.
  • the transmission gates TG1 and TG2 are coupled to the photosensitive element PD, the output transistors DV1 and DV2 are respectively coupled to the transmission gates TG1 and TG2, and the read transistors RD1 and RD2 are respectively coupled to the output transistors DV1 and DV2, and respectively output the first output signals.
  • the transfer gates TG1, TG2 receive the signals TX1, TX2, respectively, and the read transistors RD1, RD2 receive the signal ROW.
  • the photoelectric reading circuits 61, 62 further include reset transistors RT1, RT2, respectively, and the reset transistors RT1, RT2 receive the reset signal Rst.
  • the photosensitive pixel circuit 60 further includes an anti-Blooming transistor AB to extract the photoelectrons generated by the photosensitive element PD by receiving the background light, so as not to affect the normal operation of the circuit.
  • the anti-smudge transistor AB receives the signal TX5.
  • the photosensitive pixel circuit 70 is similar to the photosensitive pixel circuit 60, so the same elements follow the same symbols. Different from the photosensitive pixel circuit 60, the photosensitive pixel circuit 70 further includes an optical reading circuit 63.
  • the photoelectric reading circuit 63 has the same circuit structure as the photoelectric reading circuits 61, 62, and includes a transfer gate TG3, an output transistor DV3, and a read. The transistor RD3 is taken, wherein the transfer gate TG3 receives the signal TX3.
  • the photosensitive pixel circuit 80 is similar to the photosensitive pixel circuit 70, so the same elements follow the same symbols. Different from the photosensitive pixel circuit 70, the photosensitive pixel circuit 80 further includes a photoelectric reading circuit 64.
  • the photoelectric reading circuit 64 has the same circuit structure as the photoelectric reading circuits 61, 62, 63, and includes a transmission gate TG4 and an output transistor DV4. And a read transistor RD4, wherein the transfer gate TG4 receives the signal TX4.
  • the photosensitive pixel circuit 80 can be used to calculate the time of flight (ToF) of the light emitted by the light-emitting module 12, and the operation unit 16 can obtain the time-of-flight distance corresponding to the target object by using the time-of-flight ranging method according to the light flight time.
  • ToF time of flight
  • FIG. 9 is a timing diagram of the first pulse signal pm1, the second pulse signal pm2, the signals TX1, TX2, TX3, TX4, and TX5 and the reset signal Rst according to the embodiment of the present application.
  • the first pulse signal pm1 and the second pulse signal pm2 have pulses at time T 1 ' and time T 2 ', respectively, and the first light-emitting unit LE1 emits the first light L1 at time T 1 ', and the second light is emitted.
  • the unit LE2 emits the second light L2 at time T 2 ', wherein the time T 1 ' overlaps with the first time T 1 and is located in the first time T 1 , and the time T 2 ′ overlaps with the second time T 2 Two times in T 2 .
  • the transfer gate TG1 of each of the photosensitive pixel circuits 144 (which may be the photosensitive pixel circuits 60, 70, 80) in the photosensitive pixel array 142 is turned on, and each photosensitive pixel circuit 144 in the photosensitive pixel array 142 ( It can output a first output signal Pout1 for the read transistor RD1 of the photosensitive pixel circuit 60, 70, 80), and the photosensitive pixel array 142 outputs the first image P1 according to the first output signal Pout1 output by each photosensitive pixel circuit 144.
  • the transfer gate TG2 of each of the photosensitive pixel circuits 144 (which may be the photosensitive pixel circuits 60, 70, 80) in the photosensitive pixel array 142 is turned on, and each photosensitive pixel circuit 144 in the photosensitive pixel array 142 ( It can output a second output signal Pout2 for the read transistor RD2 of the photosensitive pixel circuit 60, 70, 80), and the photosensitive pixel array 142 outputs the second image P2 according to the second output signal Pout2 output by each photosensitive pixel circuit 144.
  • each of the photosensitive pixel circuits 144 (which may be the photosensitive pixel circuits 70, 80) of the photosensitive pixel array 142 is turned on, and each photosensitive pixel circuit 144 of the photosensitive pixel array 142 (which may The third output signal Pout3 is outputted to the read transistor RD3 of the photosensitive pixel circuits 70, 80), and the photosensitive pixel array 142 outputs the third image P3 according to the third output signal Pout3 outputted by each of the photosensitive pixel circuits 144.
  • the third time T 3 and the first time T 1 and the second time T 2 do not overlap each other, that is, in the third time T 3 , the first light emitting unit LE1 and the second light emitting unit LE2 do not emit light.
  • the photosensitive pixel array 142 receives the background light to generate the third image P3.
  • each photosensitive pixel circuit 144 (which may be the photosensitive pixel circuit 80) in the photosensitive pixel array 142 is turned on, and each photosensitive pixel circuit 144 in the photosensitive pixel array 142 (which may be photosensitive)
  • the read transistor RD4 of the pixel circuit 80 outputs a fourth output signal Pout4, and the photosensitive pixel array 142 outputs a fourth image P4 according to the fourth output signal Pout4 output by each of the photosensitive pixel circuits 144.
  • the fourth time T 4 is separated from the first time T 1 by a time interval T d .
  • the photosensitive pixel circuit 144 is implemented by the photosensitive pixel circuit 60.
  • the (n, m) in-phase image pixel values I(n, m) in the in-phase image I can be Pout1
  • the (n, m) in the orthogonal image Q The orthogonal image pixel values Q(n, m) may be Pout2.
  • the operation unit 16 can generate the in-phase image I and the orthogonal image Q according to the formulas 1.1 and 1.2.
  • the photosensitive pixel circuit 144 is implemented by the photosensitive pixel circuit 70.
  • the arithmetic unit 16 generates the in-phase image I as the first image P1 minus the third image P3, and generates the orthogonal image Q as the second image P2 minus
  • the arithmetic unit 16 generates the in-phase image I and the orthogonal image Q according to the formulas 2.1 and 2.2, the interference of the background light can be eliminated.
  • the photosensitive pixel circuit 144 is implemented by the photosensitive pixel circuit 80.
  • the computing unit 16 generates the in-phase image I as the first image P1 plus the fourth image P4, and then subtracts twice the third image P3, and generates
  • the (n, m) in-phase image pixel values I(n, m) may be Pout1+Pout4-2*Pout3, and the (n, m)th orthogonal image pixel values Q(n, m) in the orthogonal image Q Can be Pout2-Pout3.
  • the operation unit 16 may additionally calculate the fly corresponding to the target object according to the fourth image P4. Time distance.
  • the formulas 2.1, 2.2, and 3.1 in the step A02 can be performed by the differential operational amplifier in the arithmetic unit 16.
  • step A04 the arithmetic unit 16 generates a depth image DP corresponding to the target object OBJ based on the in-phase image I and the orthogonal image Q.
  • FIG. 10 is a schematic diagram of a process B0 according to an embodiment of the present application.
  • the process B0 is the operation details of the step A04, and includes the following steps:
  • Step B02 Generate a phase image PHI according to the in-phase image I and the orthogonal image Q.
  • Step B04 Generate a light stripe image LSP 1 corresponding to the first phase angle ⁇ 1 according to the phase image PHI.
  • Step B06 The optical fringe image LSP 1, DP generated depth image corresponding to the target object (OBJ).
  • (n, m) tan -1 (I(n, m) / Q(n, m)). Since the first structured light SL1 and the second structured light SL2 have a phase difference of ( ⁇ /2), the curve CV1 can be regarded as a sine wave and the curve CV2 can be regarded as a cosine wave. Therefore, the phase image PHI can be regarded as an in-phase image. The phase of the image between I and the orthogonal image Q.
  • FIG. 11 is a schematic diagram of a phase image PHI according to an embodiment of the present application.
  • the phase image PHI is a 20 ⁇ 10 image.
  • FIG. 11 illustrates the phase image PHI after the mask processing, and the non-effective area in the phase image PHI after the mask processing (or The image of the non-target object OBJ) is first excluded, that is, after the formula 4 is executed, the operation unit 16 can set the pixel value of the phase image PHI corresponding to the non-active area (or non-target object OBJ) image to 0.
  • a specific value other than 2 ⁇ in the embodiment of Fig. 11, the specific value is -1.
  • the pixel having a pixel value of 0 to 2 ⁇ in the phase image PHI of FIG. 11 is an image forming an effective area (or a target image corresponding to the target object OBJ).
  • the operation unit 16 may determine the mask area of the mask processing according to the signal intensity of the photosensitive pixel circuit output signal (eg, Pout1, Pout2, Pout3, or Pout4), that is, determine the effective area or the non-active area.
  • step B04 the arithmetic unit 16 generates a light stripe image LSP 1 corresponding to the first phase angle ⁇ 1 based on the phase image PHI.
  • the operation unit 16 may perform step B04 according to the mask processed phase image PHI (FIG. 11).
  • the first phase angle ⁇ 1 is zero.
  • FIG. 12 is a schematic diagram of a process C0 according to an embodiment of the present application.
  • the process C0 is used to generate a (general) light fringe image LSP corresponding to the (general) phase angle ⁇ ; however, for the step B04, the arithmetic unit 16 executes the flow C0 to generate a (specific) phase angle ⁇ (specific) image light strip 1 is the LSP 1, step B04 is 16 (specific) phase angle [theta] 1 is substituted into the arithmetic unit processes to produce C0 (specific) light stripe image LSP 1.
  • the process C0 includes the following steps:
  • Step C00 Start.
  • Step C02 Acquire the (n, m)th pixel value PHI(n, m) of the phase image PHI.
  • Step C04 Determine whether the pixel value PHI(n, m) is between 0 and 2 ⁇ ? If yes, go to step C06; if no, go to step C14.
  • Step C06 Acquire the (n-1, m)th pixel value PHI(n-1, m) of the phase image PHI.
  • Step C08 Determine whether the phase angle ⁇ is between the pixel value PHI(n, m) and the pixel value PHI(n-1, m)? If yes, go to step C10; if no, if yes, go to step C14.
  • Step C10 Perform an interpolation operation to obtain an interpolation result r.
  • Step C16 End.
  • the operation unit 16 can pre-configure all the light stripe image pixel values in the light stripe image LSP to be zero.
  • step C02 the arithmetic unit 16 obtains the (n, m)th pixel value PHI(n, m) of the phase image PHI (corresponding to the first phase image pixel value in the claims).
  • step C06 the arithmetic unit 16 obtains the (n-1, m)th pixel value PHI(n-1, m) of the phase image PHI (corresponding to the second phase image pixel value in the claims). It should be noted that the (n, m)th pixel of the phase image PHI and the (n-1, m)th pixel of the phase image PHI are directly adjacent to two pixels in the first direction/dimension D1.
  • step C08 the arithmetic unit 16 determines whether the phase angle ⁇ is between the pixel value PHI(n, m) and the pixel value PHI(n-1, m), and the operation unit 16 can determine PHI(n-1, m). ⁇ ⁇ ⁇ PHI (n, m) (or judge PHI (n-1, m) ⁇ ⁇ ⁇ PHI (n, m), PHI (n-1, m) ⁇ ⁇ ⁇ PHI (n, m), PHI ( Whether n-1,m) ⁇ PHI(n,m) is true (True).
  • Equation 5 is as follows) Is it true?
  • step C10 the arithmetic unit 16 calculates the interpolation result r according to the phase angle ⁇ , the phase image pixel values PHI(n, m), and PHI(n-1, m) as Equation 6 as follows.
  • the purpose of the operation unit 16 performing step C10 is to calculate the fine position of the phase difference between the structure light of the first structured light SL1 and the second structured light SL2 as the phase angle ⁇ . More precisely, the operation unit 16 can perform the step C10.
  • the interpolation operation is such that the phase difference of the structured light is indeed the fine position of the phase angle ⁇ in the first direction D1 (or the first dimension D1).
  • step C12 the arithmetic unit 16 stores the interpolation result r to the pixel coordinate position (n, m) of the light stripe image LSP 1 , and the interpolation result r becomes the pixel coordinate position (n, m) in the light stripe image LSP.
  • the light stripe image pixel value LSP(n,m), that is, LSP(n,m) r.
  • the non-zero optical stripe image pixel value LSP(n,m) obtained by the arithmetic unit 16 after the execution of the flow C0 is between n-1 and n, that is, n-1 ⁇ LSP(n,m) ⁇ n.
  • the representative corresponds to the pixel coordinate position (n-1, m) and the pixel coordinate position (n, m) has (the first structure)
  • the phase difference of the structural light of the light SL1 and the second structured light SL2 is the existence of the phase angle ⁇
  • LSP(n, m) represents the first direction/dimension D1 (the first structured light SL1 and the second structured light)
  • the structural light phase difference of SL2 is the fine position at which the phase angle ⁇ occurs.
  • FIG. 13 is further labeled with a first direction/dimension D1 and a second direction/dimension D2, wherein the first dimension D1 may correspond to the y-axis in the two-dimensional image coordinates, and the second dimension D2 may correspond to the two-dimensional image coordinates.
  • first structured light SL1 and SL2 is a second structured light
  • configuration optical phase difference is 0 in FIG. 13 occurring streaks of light shading dot image at LSP / 1.
  • the known structural light phase difference is 0 between the pixel coordinate position (n-1, m) and the pixel coordinate position (n, m), further, in the first direction D1 (or the first dimension D1)
  • the fine position at which the phase difference of the upper structure light is 0 is LSP 1 (n, m).
  • the phase difference of the structured light is 0 between the pixel coordinate position (8, 4) and the pixel coordinate position (9, 4), that is, the position where the structural light phase difference is 0 in the first direction/dimension D1.
  • the position at which the structural light phase difference is 0 occurs at 8.75 in the first direction/dimension D1.
  • step B06 the arithmetic unit 16 according to the light stripe image LSP 1, DP generated depth image corresponding to the target object (OBJ).
  • the arithmetic unit 16 only for the pixel values of the image light stripe stripes video LSP 1 is not 0, the pixel coordinate position / distance calculation.
  • FIG. 14 is a schematic diagram of a process D0 according to an embodiment of the present application. The process D0 is the operation details of the step B06, and includes the following steps:
  • Step D04 two-dimensional image coordinates (x 0, y 0), the target object OBJ to obtain the coordinate position corresponding to the pixel (n, m) of the three-dimensional image coordinates (X 0, Y 0, Z 0).
  • Step D06 Calculate the depth image pixel value DP(n, m) of the target object OBJ corresponding to the pixel coordinate position (n, m) according to the three-dimensional image coordinates (X 0 , Y 0 , Z 0 ).
  • step D04 the arithmetic unit 16 two-dimensional image coordinates (x 0, y 0), the target object OBJ to obtain the coordinate position corresponding to the pixel (n, m) of the three-dimensional image coordinates (X 0, Y 0, Z 0).
  • the operation unit 16 can calculate the three-dimensional image coordinates (X 0 , Y 0 , Z 0 ) according to the two-dimensional image coordinates (x 0 , y 0 ) by using Equation 7 (ie, triangulation), where b is The distance between the light-emitting module 12 and the light-sensing module 14, ⁇ is the elevation angle of the target object OBJ with respect to the light-emitting module 12, and ⁇ is the azimuth angle of the target object OBJ with respect to the light-emitting module 12, where f is The focal length of the lens.
  • Equation 7 ie, triangulation
  • step D06 the arithmetic unit 16 calculates the depth image pixel value DP(n, m) based on the three-dimensional image coordinates (X 0 , Y 0 , Z 0 ).
  • the operation unit 16 can calculate the depth image pixel value DP(n, m) as
  • the image phase of the structured light is at the coordinate position of the first direction/dimension D1.
  • the present application can finely express the image phase of the stripe structured light in the coordinate position of the first direction/dimension D1, thereby accurately obtaining the object. Depth information.
  • the three-dimensional image ranging system 10 uses the light-emitting module 12 including the first light-emitting unit LE1 and the second light-emitting unit LE2 to emit a first phase difference of ( ⁇ /2) or (3 ⁇ /2).
  • the present application can obtain the fine position where the structural light phase difference is the first phase angle ⁇ 1 in the first direction/dimension D1, which can more accurately calculate the depth value of the target object OBJ.
  • the three-dimensional image ranging system 10 calculates the fine position of the stripe-shaped structured light in the first direction/dimension D1 by using step B04/flow C0, and increases the fineness of the depth value of the target object OBJ.
  • the 3D image ranging system 10 calculates the depth value by using the image phase between the in-phase image I and the orthogonal image Q, and the image phase is independent of the reflected light intensity, so the depth value calculated by the 3D image ranging system 10 does not It is affected by the light reflectance of the target object OBJ.
  • the optical stripe image LSP 1 is a sparse (Sparse) image
  • the depth image DP is also sparse.
  • the sparse image represents that most of the pixels in the image have a value of 0, only a small portion.
  • the pixel value is not 0.
  • the present application may also generate at least one phase angle ⁇ ′ (corresponding to the claim)
  • At least one light stripe image LSP' of at least one second phase angle (corresponding to at least one second stripe image in the claims) to increase the density of the depth image DP.
  • the at least one phase angle ⁇ ′ is different from the first phase angle ⁇ 1
  • the at least one light stripe image LSP′ is different from the light stripe image LSP 1 .
  • FIG. 15 is a schematic diagram of a process E0 according to Embodiment 1 of the present application, and process E0 includes the following steps:
  • Step E02 Generate a phase image PHI according to the in-phase image I and the orthogonal image Q.
  • Step E04 generating a light stripe image LSP 1 corresponding to the first phase angle ⁇ 1 according to the phase image PHI.
  • Step E06 generating at least one light stripe image LSP′ corresponding to at least one phase angle ⁇ ′ according to the phase image PHI, wherein at least one phase angle ⁇ ′ is different from the first phase angle ⁇ 1 , and at least one light stripe image LSP′ is different from Light stripe image LSP 1 .
  • Step E08 Integrating the light stripe image LSP 1 and the at least one light stripe image LSP′ into the integrated image MG.
  • Step E10 According to the integrated image MG, a depth image DP corresponding to the target object OBJ is generated.
  • Steps E02 and E04 in the process E0 and steps B02 and B04 in the process B0 are not described here.
  • step E06 the arithmetic unit 16 generates at least one light stripe image LSP' corresponding to at least one phase angle ⁇ ' according to the phase image PHI.
  • the operation unit 16 generates three light stripe images LSP′ corresponding to three phase angles ⁇ ′ except the phase angle ⁇ 1 according to the phase image PHI.
  • the unit 16 can generate the light stripe images LSP 2 , LSP 3 , LSP 4 corresponding to the phase angles ⁇ 2 , ⁇ 3 , ⁇ 4 according to the phase image PHI, wherein the phase angles ⁇ 2 , ⁇ 3 , ⁇ 4 can be ( ⁇ /2), ( ⁇ ), (3 ⁇ /2).
  • Step E06 is that the arithmetic unit 16 substitutes the (specific) phase angles ⁇ 2 , ⁇ 3 , and ⁇ 4 into the flow C0 to generate the (specific) light stripe images LSP 2 , LSP 3 , and LSP 4 .
  • the characteristics of the optical stripe image LSP 2 , LSP 3 , and LSP 4 are similar to those of the optical stripe image LSP 1 and will not be described here.
  • step E08 the arithmetic unit 16 integrates the light stripe image LSP 1 and the at least one light stripe image LSP′ into the integrated image MG. Since the first phase angle ⁇ 1 is different from the at least one phase angle ⁇ ′, the pixel positions of the non-zero light stripe image in the light stripe image LSP 1 and the at least one light stripe image LSP′ are different, and thus the computing unit 16 can generate integration.
  • the image MG is the addition result of the light stripe image LSP 1 and the at least one light stripe image LSP′.
  • the arithmetic unit 16 can add the optical stripe images LSP 1 , LSP 2 , LSP 3 , and LSP 4 to generate the integrated image MG as the optical stripe images LSP 1 , LSP 2 , LSP 3 , and LSP 4 .
  • FIG. 19 is an integrated image MG integrated according to the optical stripe images LSP 1 , LSP 2 , LSP 3 and LSP 4 shown in FIGS. 13 , 16 , 17 and 18 .
  • the integrated image MG is denser than the light stripe image LSP 1 .
  • step E10 the arithmetic unit 16 generates a depth image DP corresponding to the target object OBJ based on the integrated image MG.
  • FIG. 20 is a schematic diagram of a process F0 according to an embodiment of the present application.
  • the process F0 is the operation details of the step E10, and includes the following steps:
  • Step F04 a two-dimensional image coordinates (x 0, y 0), the target object OBJ to obtain the coordinate position corresponding to the pixel (n, m) of the three-dimensional image coordinates (X 0, Y 0, Z 0).
  • Step F06 Calculate the depth image pixel value DP(n, m) of the target object OBJ corresponding to the pixel coordinate position (n, m) according to the three-dimensional image coordinates (X 0 , Y 0 , Z 0 ).
  • the process F0 is similar to the process D0, except that in step F02, the coordinate value y 0 is the integrated image pixel value MG(n, m), and the MG(n, m) is the integrated image MG at the pixel coordinate position (n, m).
  • the pixel value of the integrated image pixel value MG(n, m) is not zero.
  • the 3D image ranging system 10 generates the light stripe images LSP 1 to LSP 4 corresponding to the phase angles ⁇ 1 to ⁇ 4 by using the flow E0, and integrates the light stripe images LSP 1 to LSP 4 into a dense integration.
  • FIG. 21 is a relative positional relationship diagram of the light-emitting module 12, the photosensitive module 14, and the positions OA, OB, and OC.
  • the problem of the light plane blur is that the 3D image ranging system cannot distinguish whether the target object OBJ is located at the position OA or the position OB or the position OC (the positions OA, OB, OC are different with respect to the light-emitting module 12, respectively).
  • the 3D image ranging system 10 can first obtain the flying time distance corresponding to the target object by using the time-of-flight ranging method (the photosensitive pixel circuit 144 is realized by the photosensitive pixel circuit 80), and then according to the flying time distance.
  • the angle ⁇ required for the triangulation method is calculated, and finally the depth/distance of the corresponding target object OBJ is calculated by triangulation according to the angle ⁇ .
  • the angle ⁇ is the elevation angle of the target object OBJ relative to the light emitting module 12, that is, the angle between the target object OBJ, the light emitting module 12, and the photosensitive pixel array 142.
  • FIG. 22 is a schematic diagram of a process G0 according to an embodiment of the present application.
  • the process G0 can be executed by the operation unit 16, and the process G0 includes the following steps:
  • Step G02 According to the first image P1 and the fourth image P4, a flying time distance D ToF corresponding to the target object OBJ is generated.
  • Step G04 The angle ⁇ _opt is determined according to the flying time distance D ToF and the light stripe image LSP 1 , wherein the angle ⁇ _opt represents an angle between the target object OBJ, the light emitting module 12 and the photosensitive pixel array 142.
  • Step G06 A depth image DP corresponding to the target object OBJ is generated based on the integrated image MG and the angle ⁇ _opt.
  • the operation unit 16 may first acquire the time-of-flight image TF according to the first image P1 and the fourth image P4 (where the photosensitive pixel circuit 144 is implemented by the photosensitive pixel circuit 80), and then calculate the corresponding image according to the time-of-flight image TF.
  • Each time-of-flight image pixel value TF(n, m) in the time-of-flight image TF represents a time-of-flight distance between the pixel coordinate position (n, m) of the target object OBJ and the three-dimensional image ranging system 10.
  • the computing unit 16 can generate a flying time distance D ToF as a statistical value of the time-of-flight image TF, such as the arithmetic unit 16 can generate the flying time distance D ToF as the average value, the maximum value, the median or the public of the time-of-flight image TF Statistics such as statistics.
  • the details of the formula 8/time-of-flight ranging method are known to those skilled in the art, and details are not described herein again.
  • the arithmetic unit 16 can determine the angle ⁇ _opt according to the flying time distance D ToF and the light stripe image LSP 1 .
  • the arithmetic unit 16 can obtain a plurality of possible angles ⁇ (1) to ⁇ (K) in advance, and the angles ⁇ (1) to ⁇ (K) can be related to the characteristics of the diffraction unit DE.
  • the arithmetic unit 16 may be [alpha] in accordance with the angle obtained in advance (1) ⁇ ⁇ (K) , - performing K times flow D0, respectively, i.e.
  • the angle ⁇ (1) ⁇ ⁇ (K ) - are substituted into the step D04 in Equation 7, thereby generating K distances D ⁇ , (1) ⁇ D ⁇ , (K) generated by triangulation.
  • the arithmetic unit 16 can compare the distances D ⁇ , (1) to D ⁇ , (K) generated by the triangulation method with the flying time distance D ToF obtained in the step G02 to obtain D ⁇ , and (k_opt) is the distance D.
  • ⁇ , (1) to D ⁇ , (K) is the distance closest to the flying time distance D ToF .
  • step G06 the arithmetic unit 16 can generate the depth image DP corresponding to the target object OBJ according to the integrated image MG generated in step E08 and the angle ⁇ _opt generated in step G04.
  • FIG. 23 is a schematic diagram of a process H0 according to an embodiment of the present application.
  • the process H0 is the operation details of the step G06, and includes the following steps:
  • Step H04 Obtain three-dimensional image coordinates (X 0 , Y 0 , Z 0 ) of the target object OBJ corresponding to the pixel coordinate position (n, m) according to the two-dimensional image coordinates (x 0 , y 0 ) and the angle ⁇ _opt.
  • Step H06 Calculate the depth image pixel value DP(n, m) of the target object OBJ corresponding to the pixel coordinate position (n, m) according to the three-dimensional image coordinates (X 0 , Y 0 , Z 0 ).
  • the process H0 is similar to the processes F0 and D0.
  • the process H0 is different from the process F0 in that, in step H04, the operation unit 16 substitutes the angle ⁇ _opt with the (x 0 , y 0 ) obtained in step H02 into the formula 7 in step D04.
  • the arithmetic unit 16 can obtain the three-dimensional image coordinates (corresponding to the angle ⁇ _opt) according to step H04 (X 0 , Y 0 , Z 0 ), calculate the depth image pixel value DP(n, m).
  • the details of the operation of the process H0 please refer to the paragraphs of the process F0 and the process D0, which will not be repeated here.
  • the 3D image ranging system 10 obtains the flying time distance D ToF of the target object OBJ by using the flow G0, and compares the distance D ⁇ , (1) ⁇ D ⁇ , (K) and the flying time distance generated by the triangulation method.
  • D ToF to determine the angle ⁇ _opt, and generate a depth image DP according to the integrated image MG and the angle ⁇ _opt.
  • the present application utilizes the flying time distance D ToF to eliminate the possibility of blurring of the light plane while maintaining the advantages of the time-of-flight ranging method and the triangulation method.
  • the diffraction unit is not limited to include a single diffractive optical element.
  • FIG. 24 is a schematic diagram of a light emitting module DE' according to an embodiment of the present application.
  • the diffraction unit DE' comprises a first diffraction sub-unit DE1 and a second diffraction sub-unit DE2.
  • the first diffraction sub-unit DE1 can be a diffractive optical element
  • the second diffrative sub-unit DE2 can be another A diffractive optical element.
  • the first lighting unit LE1 may emit light toward the first diffrative sub-unit DE1, and the second lighting unit LE2 may emit light toward the second diffractive sub-unit DE2.
  • the first sub-unit DE1 diffraction structures generating a first light SL1 in the first time T 1 formed in the first optical diffraction effect of L1
  • the second sub-unit DE2 diffraction time T 2 of the second pair
  • the second light L2 forms a diffractive action to produce the second structured light SL2, which is also within the scope of the present application.
  • the present application calculates the phase of the image by using the first structured light and the second structured light having a phase difference of ( ⁇ /2) or (3 ⁇ /2), and calculates the stripe structured light according to the phase of the image.
  • the fine position of the first direction/dimension perpendicular to the strip-like structured light.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Measurement Of Optical Distance (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

一种三维影像测距系统（10），包括发光模块（12）、感光像素阵列（142）及运算单元（16）。所述发光模块于第一时间中发射第一结构光，并于第二时间中发射第二结构光，其中所述第一结构光与所述第二结构光之间的结构光相位差为π/2的奇数倍。所述感光像素阵列于所述第一时间产生第一影像（P1），并于所述第二时间产生第二影像（P2）。运算单元用来根据所述第一影像及所述第二影像，产生相关于所述第一结构光的同相影像（I）及相关于所述第二结构光的正交影像（Q），以及根据所述同相影像及所述正交影像，产生对应于目标物件的深度影像（DP）。

Description

三维影像测距系统及方法 技术领域
本申请涉及一种三维影像测距系统及方法，尤其涉及一种可计算结构光的精细位置的三维影像测距系统及方法。
背景技术
随着科学与技术的飞速发展，物体三维信息的获取在很多应用领域都有着广泛的应用前景，如生产自动化、人机交互、医学诊断、逆向工程、数字化建模等。其中，结构光三维测量法作为一种非接触式的三维信息获取技术，因其实现简单、速度快和精度高等优点得到了广泛应用。
结构光三维测量法的基本思想是利用结构光投影的几何关系来获得物体的三维信息。首先通过投影设备将编码的结构光模版投射到待测物体上，并使用摄像机记录下投影图像，将所拍图像与所投影的结构光模版进行匹配，找到匹配点后，利用投影点、匹配点及物体的三角关系求解目标物体的三维信息。
然而，现有结构光三维测量系统在计算物体于所拍摄图像的坐标位置不够精细，而限制其三维信息的精准度。因此，现有技术实有改进的必要。
发明内容
因此，本申请部分实施例的目的即在于提供一种可计算结构光的精细位置的三维影像测距系统及方法，以改善现有技术的缺点。
为了解决上述技术问题，本申请实施例提供了一种三维影像测距系统，包括发光模块，包括绕射单元；第一发光单元，用来于第一时间中发射第一光至所述绕射单元，所述绕射单元于所述第一时间中对所述第一光形成绕射作用而产生第一结构光；第二发光单元，用来于第二时间中发射第二光至所述绕射单元，所述绕射单元于所述第二时间中对所述第二光形成绕射作用而产生第二结构光，其中所述第一结构光与所述第二结构光之间的结构光相位差为(π/2)的奇数倍；感光像素阵列，用来于所述第一时间接收对应于所述第一结构光的反射光，以产生第一影像，并于所述第二时间接收对应于所述第二结构光的反射光，以产生第二影像；运算单元，耦接于所述感光像素阵列，用来执行以下步骤：根据所述第一影像及所述第二影像，产生相关于所述第一结构光的同相影像及相关于所述第二结构光的正交影像；以及根据所述同相影像及所述正交影像，产生对应于目标物件的深度影像。
例如，所述绕射单元包括第一绕射子单元及第二绕射子单元，所述第一绕射子单元于所述第一时间中对所述第一光形成绕射作用而产生所述第一结构光，所述第二绕射子单元于所述第二时间中对所述第二光形成绕射作用而产生所述第二结构光。
例如，所述第一发光单元接收第一脉冲信号并根据所述第一脉冲信号发射出所述第一光，所述第二发光单元接收第二脉冲信号并根据所述第二脉冲信号发射出所述第二光。
例如，所述第一脉冲信号及所述第二脉冲信号的占空比小于特定占空比，所述第一发光单元及所述第二发光单元的发光功率大于特定功率。
例如，所述第一发光单元所发射的所述第一光相对于所述绕射单元具有第一入射角，所述第二发光单元所发射的所述第二光相对于所述绕射单元具有第二入射角，所述第一入射角与所述第二入射角不同，而使得所述第一结构光与所述第二结构光之间的所述结构光相位差为(π/2)的奇数倍。
例如，所述感光像素阵列包括多个感光像素电路，多个感光像素电路中一第一感光像素电路包括感光元件；第一光电读取电路，耦接于所述感光元件，包括：第一传输闸，耦接于所述感光元件，其中所述第一传输闸于所述第一时间导通；第一输出晶体管，耦接于所述第一传输闸；以及第一读取晶体管，耦接于所述第一输出晶体管，用来输出第一输出信号；以及第二光电读取电路，耦接于所述感光元件，包括：第二传输闸，耦接于所述感光元件，其中所述第二传输闸于所述第二时间导通；第二输出晶体管，耦接于所述第二传输闸；以及第二读取晶体管，耦接于所述第二输出晶体管，用来输出第二输出信号；其中，所述第一影像中对应于所述第一感光像素电路的一第一像素值为所述第一输出信号，所述第二影像中对应于所述第一感光像素电路的一第二像素值为所 述第二输出信号；其中，所述同相影像中对应于所述第一感光像素电路的第一同相像素值相关于所述第一输出信号，所述正交影像中对应于所述第一感光像素电路的第一正交像素值相关于所述第二输出信号。
例如，所述第一感光像素电路还包括：第三光电读取电路，耦接于所述感光元件，包括：第三传输闸，耦接于所述感光元件，其中所述第三传输闸于第三时间导通，于所述第三时间中，所述第一发光单元及所述第二发光单元皆不发光；第三输出晶体管，耦接于所述第三传输闸；以及第三读取晶体管，耦接于所述第三输出晶体管，用来输出第三输出信号；其中，所述同相影像中对应于所述第一感光像素电路的所述第一同相像素值相关于所述第一输出信号减去所述第三输出信号，所述正交影像中对应于所述第一感光像素电路的所述第一正交像素值相关于所述第二输出信号减去所述第三输出信号；其中，所述多个感光像素电路所输出的多个第三输出信号形成第三影像。
例如，所述第一感光像素电路还包括第四光电读取电路，耦接于所述感光元件，包括：第四传输闸，耦接于所述感光元件，其中所述第四传输闸于第四时间导通，所述第一时间与所述第四时间相隔时间间隔；第四输出晶体管，耦接于所述第四传输闸；以及第四读取晶体管，耦接于所述第四输出晶体管，用来输出第四输出信号；其中，所述同相影像中对应于所述第一感光像素电路的所述第一同相像素值相关于所述第一输出信号及第四输出信号；其中，所述多个感光像素电路所输出的多个第四输出信号形成第四影像；其中，所述运算单元根据所述第一影像及所述第四影像，取得对应于所述目标物件的飞时距离。
例如，所述运算单元用来执行以下步骤，以根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像：根据所述同相影像及所述正交影像，产生相位影像，其中所述相位影像代表所述第一结构光所形成的所述同相影像与所述第二结构光所形成的所述正交影像之间的影像相位；根据所述相位影像，产生对应于第一相位角度的第一光条纹影像，其中所述第一光条纹影像纪录所述影像相位为所述第一相位角度的坐标位置；以及根据所述第一光条纹影像，产生对应于所述目标物件的所述深度影像。
例如，所述第一相位角度为0。
例如，所述运算单元用来执行以下步骤，以根据所述相位影像，产生对应于所述第一相位角度的所述第一光条纹影像：取得所述相位影像中位于第一像素坐标位置的第一相位影像像素值，并取得所述相位影像中位于第二像素坐标位置的第二相位影像像素值，其中所述第一像素坐标位置于第一维度直接相邻于所述第二像素坐标位置；判断所述第一相位角度是否介于所述第一相位影像像素值与所述第二相位影像像素值之间；当所述第一相位角度介于所述第一相位影像像素值与所述第二相位影像像素值之间时，根据所述第一相位角度、所述第一相位影像像素值及所述第二相位影像像素值，执行内插运算，以取得内插结果；将所述内插结果储存至所述第一光条纹影像的所述第一像素坐标位置，而所述内插结果成为所述第一光条纹影像中于所述第一像素坐标位置的光条纹影像像素值。
例如，所述运算单元用来执行以下步骤，以根据所述第一相位角度、所述第一相位影像像素值及所述第二相位影像像素值，执行所述内插运算以取得所述内插结果：计算所述内插结果为
Figure PCTCN2018082441-appb-000001
其中，θ代表所述第一相位角度，(n,m)代表所述第一像素坐标位置，(n-1,m)代表所述第二像素坐标位置，PHI(n,m)代表所述第一相位影像像素值，PHI(n-1,m)所述第二相位影像像素值。
例如，所述运算单元另用来执行以下步骤，以根据所述相位影像，产生对应于所述第一相位角度的所述第一光条纹影像：当所述第一相位角度不介于所述相位影像中所述第一相位影像像素值与所述第二相位影像像素值之间时，所述第一光条纹影像中对应于所述第一像素坐标位置的光条纹影像像素值为0。
例如，所述运算单元用来执行以下步骤，以根据所述第一光条纹影像，产生对应于所述目标物件的所述深度影像：取得所述目标物件对应于第三像素坐标位置的第一二维图像坐标，其中所述第一二维图像坐标为(x 0,y 0)＝(m,LSP 1(n,m))，y 0代表所述第一二维图像坐标于第一维度的坐标值，x 0代表所述第一二维图像坐标于第二维度的坐标值，n代表所述第三像素坐标位置于所述第一维度的坐标值，m代表所述第三像素坐标位置于所述第二维度的坐标值，LSP 1(n,m)代表所述第一光条纹影像于第三像素坐标位置的光条纹影像像素值；根据所述第一二维图像坐标，取得所述目标物件对应于所述第三像素坐标位置的第一三维图像坐标；以及根据所述第一三维图像坐标，计算所述目标物 件对应于所述第三像素坐标位置的深度影像像素值。
例如，所述运算单元另用来执行以下步骤，以根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像：根据所述相位影像，产生对应于至少一第二相位角度的至少一第二光条纹影像，其中所述至少一第二相位角度不同于所述第一相位角度，所述至少一第二光条纹影像不同于所述第一光条纹影像影像相位；以及将所述第一光条纹影像及所述至少一第二光条纹影像整合成为整合影像；以及根据所述整合影像，产生对应于所述目标物件的所述深度影像。
例如，所述至少一第二相位角度为(2π/L)的整数倍，L为大于1的正整数。
例如，所述运算单元用来执行以下步骤，以将所述第一光条纹影像及所述至少一第二光条纹影像整合成为所述整合影像：产生所述整合影像为所述第一光条纹影像及所述至少一第二光条纹影像的相加结果。
例如，所述运算单元用来执行以下步骤，以根据所述整合影像，产生对应于所述目标物件的所述深度影像：取得所述目标物件于所述整合影像中对应于第四像素坐标位置的第二二维图像坐标，其中所述第二二维图像坐标为(x 0,y 0)＝(m,MG(n,m))，y 0代表所述第二二维图像坐标于第一维度的坐标值，x 0代表所述二维图像坐标于第二维度的坐标值，n代表所述第四像素坐标位置于所述第一维度的坐标值，m代表所述第四像素坐标位置于所述第二维度的坐标值， MG(n,m)代表所述整合影像于第四像素坐标位置的整合影像像素值；根据所述第二二维图像坐标，取得所述目标物件对应于所述第三像素坐标位置的第二三维图像坐标；以及根据所述第二三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
例如，所述感光像素阵列于第三时间接收背景光以产生第三影像，所述感光像素阵列于第四时间中接收对应于所述第一结构光的反射光以产生第四影像，所述运算单元另用来执行以下步骤，以根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像：根据所述第一影像及所述第四影像，产生对应于所述目标物件的飞时距离；根据所述飞时距离及所述第一光条纹影像，决定角度，其中所述角度代表所述目标物件、所述发光模块与所述感光像素阵列之间的夹角；以及根据所述整合影像以及所述角度，产生对应于所述目标物件的所述深度影像。
例如，所述运算单元另用来执行以下步骤，以根据所述整合影像以及所述角度，产生对应于所述目标物件的所述深度影像：取得所述目标物件于所述整合影像中对应于第四像素坐标位置的第二二维图像坐标，其中所述第二二维图像坐标为(x 0,y 0)＝(m,MG(n,m))，y 0代表所述第二二维图像坐标于第一维度的坐标值，x 0代表所述二维图像坐标于第二维度的坐标值，n代表所述第四像素坐标位置于所述第一维度的坐标值，m代表所述第四像素坐标位置于所述第二维度的坐标值，MG(n,m)代表所述整合影像于第四像素坐标位置的整合影像像素值；根据所述第二二维图像坐标以及所述角度，取得所述目标物件对应于所述 第三像素坐标位置的第二三维图像坐标；以及根据所述第二三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
为了解决上述技术问题，本申请实施例另提供了一种三维影像测距方法，应用于三维影像测距系统，其特征在于，所述三维影像测距系统包括发光模块及感光像素阵列，所述发光模块于第一时间中发射第一结构光，并于第二时间中发射第二结构光，所述感光像素阵列于所述第一时间接收对应于所述第一结构光的反射光以产生第一影像，并于所述第二时间接收对应于所述第二结构光的反射光以产生第二影像，其中所述第一结构光与所述第二结构光之间的结构光相位差为(π/2)的奇数倍，所述三维影像测距方法包括：根据所述第一影像及所述第二影像，产生相关于所述第一结构光的同相影像及相关于所述第二结构光的正交影像；以及根据所述同相影像及所述正交影像，产生对应于目标物件的深度影像。
本申请实施例利用包括结构光相位差为(π/2)或(3π/2)的第一结构光及第二结构光，计算影像相位；并根据影像相位，计算条纹状结构光在(与条纹状结构光垂直的)第一方向/维度的精细位置。相较于现有技术，本申请可精确地取得目标物件的深度信息。
附图说明
图1为本申请实施例一三维影像测距系统的外观示意图；
图2为图1的三维影像测距系统的功能方块示意图；
图3为本申请实施例一绕射单元的示意图；
图4为本申请实施例一第一结构光及一第二结构光的示意图；
图5为本申请实施例一流程的示意图；
图6为本申请实施例一感光像素电路的示意图；
图7为本申请实施例一感光像素电路的示意图；
图8为本申请实施例一感光像素电路的示意图；
图9为本申请实施例多个信号的时序示意图；
图10为本申请实施例一流程的示意图；
图11为本申请实施例一相位影像的示意图；
图12为本申请实施例一流程的示意图；
图13为本申请实施例一光条纹影像的示意图；
图14为本申请实施例一流程的示意图；
图15为本申请实施例一流程的示意图；
图16为本申请实施例一光条纹影像的示意图；
图17为本申请实施例一光条纹影像的示意图；
图18为本申请实施例一光条纹影像的示意图；
图19为本申请实施例一整合影像的示意图；
图20为本申请实施例一流程的示意图；
图21为本申请实施例的发光模块、感光模块的相对位置关系图；
图22为本申请实施例一流程的示意图；
图23为本申请实施例一流程的示意图；
图24为本申请实施例一发光模块的示意图；
图25为条纹状结构光投射于物件的示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
在本申请说明书以及权利要求中，影像A与影像B之间进行加减乘除运算代表影像A及影像B进行像素与像素之间的加减乘除运算。详细来说，假设A(n,m)、B(n,m)及C(n,m)分别代表影像A、影像B及影像C于像素坐标位置(n,m)的像素值，影像C等于影像A加影像B(记为C＝A+B)代表影像C的第(n,m)个像素值C(n,m)为C(n,m)＝A(n,m)+B(n,m)，影像C等于影像A减影像B(记为C＝A-B)代表影像C的第(n,m)个像素值C(n,m)为C(n,m)＝A(n,m)-B(n,m)，影像C等于影像A乘以影像B(记为C＝A*B)代表影像C的第(n,m)个像素值C(n,m)为C(n,m)＝A(n,m)*B(n,m)，影像C等于影像A除以影像B(记为C＝A/B)代表影像C的第(n,m)个像素值C(n,m)为C(n,m)＝A(n,m)/B(n,m)。对影像A进行函数f运算代表对影像A中每一元素进行函数f运算，例如C＝f(A)代表影像C的第(n,m)个像素值C(n,m)为C(n,m)＝f(A(n,m))。另外，n为影像中的行指针(Row Index)，m为影像中的列指针(Column Index)。另外，角度或相位的单位为弧度(又称弪度，即Radian)。
三维影像测距系统可利用条纹状结构光以及三角测量法量测距离，请参考 图25，图25为条纹状结构光投射于物件(手掌)的示意图。条纹状结构光垂直于第一方向D1(或第一维度D1)而平行于第二方向D2(或第二维度D2)。如图25所示，条纹状结构光投射于物件时会因物件的形状而扭曲，而对应条纹状结构光的影像在第一方向/维度D1会有较大的变化。现有技术中，计算条纹状结构光影像在第一方向/维度D1的坐标位置受限于像素本身，而无法更加精细，而影响三角测量法所计算出的距离。本申请可计算条纹状结构光在第一方向/维度D1的精细位置，进而精确地取得物件的深度信息。
具体来说，图1为本申请实施例一三维影像测距系统10的外观示意图，图2为三维影像测距系统10的功能方块示意图。三维影像测距系统10可设置于电子装置1中，电子装置1可为智能手机、或平板计算机。三维影像测距系统10包括发光模块12、感光模块14以及运算单元16，发光模块12于第一时间T 1中发射第一结构光SL1，并于第二时间T 2中发射第二结构光SL2，其中第一结构光SL1与第二结构光SL2之间具有结构光相位差，而结构光相位差为(π/2)的奇数倍，例如，结构光相位差可为(π/2)或是(3π/2)。感光模块14可包括镜头(Lens)146以及感光像素阵列142，感光像素阵列142包括排列成阵列的多个感光像素电路144，感光像素阵列142用来于第一时间T 1接收对应于第一结构光SL1的反射光，以产生第一影像P1，并于第二时间T 2接收对应于第二结构光SL2的反射光，以产生第二影像P2。运算单元16耦接于感光像素阵列142，其可包含处理器或差分运算放大器，运算单元16用来以接收第一影像P1及第二影像P2，并根据第一影像P1及第二影像P2，产生对应于目标物件的深度影像。
发光模块12包括绕射单元DE、第一发光单元LE1以及第二发光单元LE2，绕射单元DE可包括单一绕射光学元件(Diffraction Optical Element，DOE)，第一发光单元LE1以及第二发光单元LE2可为发光二极管(LED)或激光(Laser)发射单元。第一发光单元LE1用来于第一时间T 1中发射第一光L1至绕射单元DE，绕射单元DE于第一时间T 1中对第一光L1形成绕射作用而产生第一结构光SL1；第二发光单元LE2用来于第二时间T 2中发射第二光L1至绕射单元DE，绕射单元DE于第二时间T 2中对第二光L2形成绕射作用而产生第二结构光SL2。
第一发光单元LE1以及第二发光单元LE2可瞬间发出强光(类似一般照相机的闪光灯)，而第一光L1及第二光L1可为脉冲调变(Pulse Modulated)光，使得关于结构光SL1、SL2的光信号不易受到背景光干扰。换句话说，第一发光单元LE1接收第一脉冲信号pm1并根据第一脉冲信号pm1发射出第一光L1，第二发光单元LE2接收第二脉冲信号pm2并根据第二脉冲信号pm2发射出第二光L2。其中，第一脉冲信号pm1及第二脉冲信号pm2的占空比小于特定占空比，而第一发光单元LE1以及第二发光单元LE2的发光功率大于特定功率。于一实施例中，第一脉冲信号pm1及第二脉冲信号pm2的占空比可小于1/50或小于1/1000，第一发光单元LE1以及第二发光单元LE2的发光功率可大于4瓦特。其余关于发光模块12根据脉冲信号pm1、pm2产生结构光SL1、SL2的技术细节，可参考现有的文献所揭露的内容，于此不再赘述。
图3为本申请实施例第一结构光SL1及第二结构光SL2的示意图。为了方便说明，于子图3a中，第一结构光SL1及第二结构光SL2分别用黑色条纹及斜线阴影条纹来表示，第一结构光SL1的光条纹及第二结构光SL2的光条纹(Light Strip)皆垂直于第一方向/维度D1而平行于第二方向/维度D2。实际上，黑色条纹及斜线阴影条纹分别代表第一结构光SL1及第二结构光SL2最亮的光条纹，另外，子图3b中的曲线CV1及曲线CV2分别代表第一结构光SL1及第二结构光SL2沿第一方向D1的强度分布曲线(Intensity Profile)，曲线CV1、CV2皆为正弦式(Sinusoidal)曲线。于子图3b所绘示的实例中，曲线CV2可视为相较于曲线CV1延迟了(π/2)(或曲线CV1可相较于曲线CV2领先了(3π/2))，即第二结构光SL2的结构光相位相较于第一结构光SL1的结构光相位延迟(π/2)(或第一结构光SL1的结构光相位相较于第二结构光SL2的结构光相位领先(3π/2))，因此，曲线CV1可视为正弦波(Sine Wave)，曲线CV2可视为余弦波(Cosine Wave)。
另一方面，对应第一结构光SL1的光条纹(黑色条纹)彼此之间相隔一距离PD，第二结构光SL2的结构光相位相较于第一结构光SL1的结构光相位延迟(π/2)代表对应第二结构光SL2的光条纹(即斜线阴影条纹)位于对应第一结构光SL1的光条纹(即黑色条纹)下方PD/4处，其中PD可代表曲线CV1的周期。
为了达到第二结构光SL2的结构光相位相较于第一结构光SL1的结构光相位延迟(π/2)，或达到对应第二结构光SL2的光条纹(斜线阴影条纹)位于对应 第一结构光SL1的光条纹(黑色条纹)下方PD/4处，可调整第一发光单元LE1及第二发光单元LE2相对于绕射单元DE的入射角。请参考图4，图4为本申请实施例第一发光单元LE1、第二发光单元LE2及绕射单元DE的示意图。如图4所示，第一发光单元LE1所发射的第一光L1通过准直器(Collimator)CM之后相对于绕射单元DE具有第一入射角μ 1，第二发光单元LE2所发射的第二光L2通过准直器CM之后相对于绕射单元DE具有第二入射角μ 2，其中第一入射角μ 1与第二入射角μ 2不同，可调整第一入射角μ 1及第二入射角μ 2，而使得第二结构光SL2的结构光相位相较于第一结构光SL1的结构光相位延迟(π/2)，即第一结构光SL1与第二结构光SL2之间的所述结构光相位差为(π/2)。
另外，由于曲线CV1为正弦波而曲线CV2为余弦波，运算单元16可根据对应于第一结构光SL1的反射光的第一影像P1及对应于第二结构光SL2的反射光的第二影像P2，产生相关于第一结构光SL1的一同相(In-Phase)影像I以及相关于第二结构光SL2的一正交(Quadrature)影像Q，并根据同相影像I及正交影像Q，计算同相影像I与正交影像Q之间的影像相位，并根据影像相位，产生对应于目标物件的深度影像。
另外，于本申请说明书以及权利要求中，影像中不同行(Row)的像素值代表感光模块14所拍摄到对应第一方向/维度D1上不同位置的像素值，影像中不同列(Column)的像素值代表感光模块14所拍摄到对应第二方向/维度D2上不同位置的像素值。
关于运算单元16的运作，可归纳成为一流程A0。图5为本申请实施例流程A0的示意图。流程A0可由运算单元16来执行，流程A0包含以下步骤：
步骤A02：根据第一影像P1及第二影像P2，产生相关于第一结构光SL1的同相影像I以及相关于第二结构光SL2的正交影像Q。
步骤A04：根据同相影像I及正交影像Q，产生对应于目标物件OBJ的深度影像DP。
于步骤A02中，运算单元16根据第一影像P1产生同相影像I并根据第二影像P2产生正交影像Q。详细来说，请参考图6、图7及图8，图6、图7及图8分别为本申请实施例一感光像素电路60、一感光像素电路70及一感光像素电路80的示意图。感光像素电路60、70、80可用来实现感光像素电路144，为了方便说明，感光像素电路60、70、80可为感光像素阵列142中位于像素坐标位置(n,m)的感光像素电路(即为第(n,m)个感光像素电路)。
感光像素电路60包括感光元件PD以及光电读取电路61、62，感光元件PD可为感光二极管(Photo Diode)。光电读取电路61包括传输闸TG1、输出晶体管DV1以及读取晶体管RD1，光电读取电路62包括传输闸TG2、输出晶体管DV2以及读取晶体管RD2。传输闸TG1、TG2耦接于感光元件PD，输出晶体管DV1、DV2分别耦接于传输闸TG1、TG2，读取晶体管RD1、RD2分别耦接于输出晶体管DV1、DV2，并分别输出第一输出信号Pout1、第二输出信号Pout2。传输闸TG1、TG2分别接收信号TX1、TX2，读取晶体管RD1、RD2接收信号ROW。光电读取电路61、62另分别包括重置晶体管RT1、RT2， 重置晶体管RT1、RT2接收重置信号Rst。感光像素电路60另包括一防晕染(Anti-Blooming)晶体管AB，以将感光元件PD因接收背景光而产生的光电子汲取出来，以免影响电路的正常运作。防晕染晶体管AB接收信号TX5。
感光像素电路70与感光像素电路60类似，故相同元件沿用相同符号。与感光像素电路60不同的是，感光像素电路70另包括光电读取电路63，光电读取电路63与光电读取电路61、62的电路结构相同，其包括传输闸TG3、输出晶体管DV3以及读取晶体管RD3，其中传输闸TG3接收信号TX3。
感光像素电路80与感光像素电路70类似，故相同元件沿用相同符号。与感光像素电路70不同的是，感光像素电路80另包括光电读取电路64，光电读取电路64与光电读取电路61、62、63的电路结构相同，其包括传输闸TG4、输出晶体管DV4以及读取晶体管RD4，其中传输闸TG4接收信号TX4。感光像素电路80可用于计算发光模块12所发出光线的光线飞行时间(Time of Flight，ToF)，运算单元16可根据光线飞行时间利用飞时测距法，取得对应于目标物件的飞时距离，其细节详述于后。
请一并参考图9，图9为本申请实施例第一脉冲信号pm1、第二脉冲信号pm2、信号TX1、TX2、TX3、TX4、TX5以及重置信号Rst的时序示意图。如图9所示，第一脉冲信号pm1及第二脉冲信号pm2分别于时间T 1’及时间T 2’具有脉冲，第一发光单元LE1于时间T 1’发射第一光L1，第二发光单元LE2于时间T 2’发射第二光L2，其中时间T 1’与第一时间T 1重迭而位于第一时间 T 1中，时间T 2’与第二时间T 2重迭而位于第二时间T 2中。
于第一时间T 1，感光像素阵列142中每个感光像素电路144(其可为感光像素电路60、70、80)的传输闸TG1导通，感光像素阵列142中每个感光像素电路144(其可为感光像素电路60、70、80)的读取晶体管RD1输出第一输出信号Pout1，感光像素阵列142根据每个感光像素电路144所输出的第一输出信号Pout1，输出第一影像P1。
于第二时间T 2，感光像素阵列142中每个感光像素电路144(其可为感光像素电路60、70、80)的传输闸TG2导通，感光像素阵列142中每个感光像素电路144(其可为感光像素电路60、70、80)的读取晶体管RD2输出第二输出信号Pout2，感光像素阵列142根据每个感光像素电路144所输出的第二输出信号Pout2，输出第二影像P2。
于第三时间T 3，感光像素阵列142中每个感光像素电路144(其可为感光像素电路70、80)的传输闸TG3导通，感光像素阵列142中每个感光像素电路144(其可为感光像素电路70、80)的读取晶体管RD3输出第三输出信号Pout3，感光像素阵列142根据每个感光像素电路144所输出的第三输出信号Pout3，输出第三影像P3。其中，第三时间T 3与第一时间T 1及第二时间T 2不相互重迭，即于第三时间T 3中，第一发光单元LE1以及第二发光单元LE2皆不发光。换句话说，于第三时间T 3，感光像素阵列142接收背景光而产生第三影像P3。
于第四时间T 4，感光像素阵列142中每个感光像素电路144(其可为感光像素电路80)的传输闸TG4导通，感光像素阵列142中每个感光像素电路144(其可为感光像素电路80)的读取晶体管RD4输出第四输出信号Pout4，感光像素阵列142根据每个感光像素电路144所输出的第四输出信号Pout4，输出第四影像P4。其中，第四时间T 4与第一时间T 1相隔时间间隔T d。
于一实施例中，感光像素电路144以感光像素电路60来实现，运算单元16产生同相影像I为第一影像P1，并产生正交影像Q为第二影像P2，即I＝P1(公式1.1)，Q＝P2(公式1.2)，换句话说，同相影像I中第(n,m)个同相影像像素值I(n,m)可为Pout1，正交影像Q中第(n,m)个正交影像像素值Q(n,m)可为Pout2。当第一发光单元LE1以及第二发光单元LE2的发光功率够强以至于背景光可忽略不计时，运算单元16可根据公式1.1及1.2产生同相影像I及正交影像Q。
于一实施例中，感光像素电路144以感光像素电路70来实现，运算单元16产生同相影像I为第一影像P1减去第三影像P3，并产生正交影像Q为第二影像P2减去第三影像P3，即I＝P1-P3(公式2.1)，Q＝P2-P3(公式2.2)，换句话说，同相影像I中第(n,m)个同相影像像素值I(n,m)可为Pout1-Pout3，正交影像Q中第(n,m)个正交影像像素值Q(n,m)可为Pout2-Pout3。当运算单元16根据公式2.1及2.2产生同相影像I及正交影像Q时，可消除背景光的干扰。
于一实施例中，感光像素电路144以感光像素电路80来实现，运算单元16产生同相影像I为第一影像P1加上第四影像P4后再减去第三影像P3的2倍，并产生正交影像Q为第二影像P2减去第三影像P3，即I＝P1+P4-2*P3(公式3.1)，Q＝P2-P3(即公式2.2)，换句话说，同相影像I中第(n,m)个同相影像像素值I(n,m)可为Pout1+Pout4-2*Pout3，正交影像Q中第(n,m)个正交影像像素值Q(n,m)可为Pout2-Pout3。当感光像素电路144以感光像素电路80来实现且运算单元16根据公式3.1及2.2产生同相影像I及正交影像Q时，运算单元16可根据第四影像P4另外计算出对应于目标物件的飞时距离。其中，步骤A02中的公式2.1、2.2、3.1可通过运算单元16内的差分运算放大器来执行。
于步骤A04中，运算单元16根据同相影像I及正交影像Q，产生对应于目标物件OBJ的深度影像DP。请参考图10，图10为本申请实施例一流程B0的示意图。流程B0为步骤A04的操作细节，其包含以下步骤：
步骤B02：根据同相影像I及正交影像Q，产生相位影像PHI。
步骤B04：根据相位影像PHI，产生对应于第一相位角度θ 1的光条纹影像LSP 1。
步骤B06：根据光条纹影像LSP 1，产生对应于目标物件OBJ的深度影像DP。
于步骤B02中，运算单元16产生相位影像PHI为PHI＝tan -1(I/Q)(公式4)，即相位影像PHI的第(n,m)个像素值PHI(n,m)为PHI(n,m)＝tan -1(I(n,m)/Q(n,m))。由于第一结构光SL1与第二结构光SL2具有(π/2)的结构光相 位差，曲线CV1可视为正弦波而曲线CV2可视为余弦波，因此，相位影像PHI可视为同相影像I与正交影像Q之间的影像相位。
请参考图11，图11为本申请实施例相位影像PHI的示意图。于图11中，相位影像PHI为20×10的影像，另外，图11绘示经过遮罩(Mask)处理过后的相位影像PHI，于遮罩处理后的相位影像PHI中，非有效区域(或非目标物件OBJ)的影像已先排除，也就是说，执行完公式4后，运算单元16可将相位影像PHI中对应于非有效区域(或非目标物件OBJ)影像的像素值设为0到2π以外的特定数值，于图11的实施例中，该特定数值为-1。换句话说，图11的相位影像PHI中像素值介于0到2π的像素即形成有效区域的影像(或对应于目标物件OBJ的目标影像)。另外，于一实施例中，运算单元16可根据感光像素电路输出信号(如Pout1、Pout2、Pout3或Pout4)的信号强度决定遮罩处理的遮罩区域，即决定有效区域或非有效区域。
于步骤B04中，运算单元16根据相位影像PHI，产生对应于第一相位角度θ 1的光条纹影像LSP 1。于一实施例中，运算单元16可根据遮罩处理后的相位影像PHI(如图11)执行步骤B04。于一实施例中，第一相位角度θ 1为0。请参考图12，图12为本申请实施例一流程C0的示意图。一般来说，流程C0用来产生对应于(一般)相位角度θ的(一般)光条纹影像LSP；然而，对步骤B04而言，运算单元16执行流程C0以产生对应于(特定)相位角度θ 1的(特定)光条纹影像LSP 1，即步骤B04为运算单元16将(特定)相位角度θ 1代入流程C0中以产生(特定)光条纹影像LSP 1。于后续关于流程C0的说明中， 将对(一般)相位角度θ以及(一般)光条纹影像LSP进行说明。如图12所示流程C0包含以下步骤：
步骤C00：开始。
步骤C02：取得相位影像PHI的第(n,m)个像素值PHI(n,m)。
步骤C04：判断像素值PHI(n,m)是否介于0到2π？若是，执行步骤C06；若否，执行步骤C14。
步骤C06：取得相位影像PHI的第(n-1,m)个像素值PHI(n-1,m)。
步骤C08：判断相位角度θ是否介于像素值PHI(n,m)与像素值PHI(n-1,m)之间？若是，执行步骤C10；若否，若是，执行步骤C14。
步骤C10：执行内插运算，以取得内插结果r。
步骤C12：令LSP(n,m)＝r。
步骤C14：令LSP(n,m)＝0。
步骤C16：结束。
于步骤C00中，运算单元16可跳过行指标n为1(n＝1)而先从行指标n大于或等于2(n≥2)开始计算。另外，于一实施例中，运算单元16可预先配置光条纹影像LSP中所有的光条纹影像像素值为0。
于步骤C02中，运算单元16取得相位影像PHI的第(n,m)个像素值PHI(n,m)(可对应权利要求中的第一相位影像像素值)。于步骤C06中，运算单元16取得相位影像PHI的第(n-1,m)个像素值PHI(n-1,m)(可对应权利要求中的第二相位影像像素值)。需注意的是，相位影像PHI的第(n,m)个像素与相位影 像PHI的第(n-1,m)个像素为于第一方向/维度D1上直接相邻两个像素。
于步骤C08中，运算单元16判断相位角度θ是否介于像素值PHI(n,m)与像素值PHI(n-1,m)之间，运算单元16可判断PHI(n-1,m)≤θ<PHI(n,m)(或判断PHI(n-1,m)<θ≤PHI(n,m)、PHI(n-1,m)≤θ≤PHI(n,m)、PHI(n-1,m)<θ<PHI(n,m)其中之一)是否为真(True)。另外，当相位角度θ为0(θ＝0)时，运算单元16于步骤C08等同于判断PHI(n-1,m)+π≤θ+π<PHI(n,m)+π(或判断公式5如下)是否为真。
mod(PHI(n-1,m)+π,2π)≤mod(θ+π,2π)<mod(PHI(n,m)+π,2π)(公式5)
于步骤C10中，运算单元16可根据相位角度θ、相位影像像素值PHI(n,m)及PHI(n-1,m)，计算内插结果r为公式6如下。运算单元16执行步骤C10的目的在于计算出第一结构光SL1与第二结构光SL2之间的结构光相位差为相位角度θ的精细位置，更精确的说，运算单元16可利用步骤C10执行内插运算，以结构光相位差确实为相位角度θ于第一方向D1(或第一维度D1)上的精细位置。
Figure PCTCN2018082441-appb-000002
于步骤C12中，运算单元16将内插结果r储存至光条纹影像LSP 1的像素坐标位置(n,m)，而内插结果r成为光条纹影像LSP中于像素坐标位置(n,m)的光条纹影像像素值LSP(n,m)，即LSP(n,m)＝r。
另外，运算单元16执行流程C0后所得到非0的光条纹影像像素值LSP(n,m)会介于n-1与n之间，即n-1<LSP(n,m)<n。更进一步地，当光条纹影像像素值LSP(n,m)不为0时，代表对应于像素坐标位置(n-1,m)与像素坐标位置(n,m)之间具有(第一结构光SL1与第二结构光SL2的)结构光相位差为相位角度θ的存在事实，且LSP(n,m)代表于第一方向/维度D1上，(第一结构光SL1与第二结构光SL2的)结构光相位差为相位角度θ发生的精细位置。
请参考图13，图13为运算单元16执行步骤B04后所产生的光条纹影像LSP 1(其中相位角度θ为第一相位角度θ 1，且θ＝θ 1＝0)。为了方便说明，图13另标示有第一方向/维度D1以及第二方向/维度D2，其中第一维度D1可对应二维图像坐标中的y轴，而第二维度D2可对应二维图像坐标中的x轴。由图13可知，(第一结构光SL1与第二结构光SL2的)结构光相位差为0发生于图13/光条纹影像LSP 1的点状底纹处。在已知结构光相位差为0发生像素坐标位置(n-1,m)与像素坐标位置(n,m)之间的情况下，更进一步地，于第一方向D1(或第一维度D1)上结构光相位差为0发生的精细位置为LSP 1(n,m)。举例来说，结构光相位差为0发生像素坐标位置(8,4)与像素坐标位置(9,4)之间，即于第一方向/维度D1上结构光相位差为0发生的位置介于8与9之间，更进一步地，结构光相位差为0发生的位置于第一方向/维度D1上为8.75。
于步骤B06中，运算单元16根据光条纹影像LSP 1，产生对应于目标物件OBJ的深度影像DP。于一实施例中，运算单元16仅针对光条纹影像LSP 1中光条纹影像像素值不为0的像素坐标位置进行深度/距离计算。请参考图14，图 14为本申请实施例一流程D0的示意图。流程D0为步骤B06的操作细节，其包含以下步骤：
步骤D02：取得目标物件OBJ对应于像素坐标位置(n,m)的二维图像坐标(x 0,y 0)为(x 0,y 0)＝(m,LSP 1(n,m))。
步骤D04：根据二维图像坐标(x 0,y 0)，取得目标物件OBJ对应于像素坐标位置(n,m)的三维图像坐标(X 0,Y 0,Z 0)。
步骤D06：根据三维图像坐标(X 0,Y 0,Z 0)，计算目标物件OBJ对应于像素坐标位置(n,m)的深度影像像素值DP(n,m)。
于步骤D02中，运算单元16取得目标物件OBJ对应于像素坐标位置(n,m)的二维图像坐标(x 0,y 0)为(x 0,y 0)＝(m,LSP 1(n,m))，其中x 0代表二维图像坐标(x 0,y 0)于第二维度D2的坐标值，y 0代表二维图像坐标(x 0,y 0)于第一维度D1的坐标值，n代表像素坐标位置(n,m)于第一维度D1的坐标值，m代表像素坐标位置(n,m)于第二维度D2的坐标值，条纹影像像素值LSP 1(n,m)不为0。
于步骤D04中，运算单元16根据二维图像坐标(x 0,y 0)，取得目标物件OBJ对应于像素坐标位置(n,m)的三维图像坐标(X 0,Y 0,Z 0)。于一实施例中，运算单元16可利用公式7(即三角测量法)，根据二维图像坐标(x 0,y 0)计算三维图像坐标(X 0,Y 0,Z 0)，其中b为发光模块12与感光模块14之间的距离，α为目标物件OBJ相对于发光模块12的俯仰角(Elevation Angle)，ρ为目标物件OBJ相对于发光模块12的方位角(Azimuth Angle)，f为镜头的焦距。另外，公式7/三角测量法的细节为本领域技术人员所知，可参考现有文献所揭露的内容，于 此不再赘述。
Figure PCTCN2018082441-appb-000003
于步骤D06中，运算单元16根据三维图像坐标(X 0,Y 0,Z 0)，计算深度影像像素值DP(n,m)。于一实施例中，运算单元16可计算深度影像像素值DP(n,m)为
Figure PCTCN2018082441-appb-000004
另外，当LSP 1(n,m)为0时，于一实施例中，深度影像像素值DP(n,m)可为0(即DP(n,m)＝0)。
另外，现有技术中利用三角测量法计算距离的二维图像坐标(x 0,y 0)为(x 0,y 0)＝(m,n)，其中n为整数，无法精细地表达条纹状结构光的影像相位在第一方向/维度D1的坐标位置。相较之下，本申请利用三角测量法计算距离的二维图像坐标(x 0,y 0)为(x 0,y 0)＝(m,LSP 1(n,m))，其中LSP 1(n,m)介于n-1与n之间的有理数，换句话说，本申请可精细地表达条纹状结构光的影像相位在第一方向/维度D1的坐标位置，进而精确地取得物件的深度信息。
简言之，三维影像测距系统10利用包括第一发光单元LE1及第二发光单元LE2的发光模块12，发射出结构光相位差为(π/2)或是(3π/2)的第一结构光SL1 及第二结构光SL2；利用流程A0，产生相关于第一结构光SL1的同相影像I以及相关于第二结构光SL2的正交影像Q；利用流程B0，产生代表同相影像I与正交影像Q之间的影像相位的相位影像PHI；利用流程C0，产生对应于第一相位角度θ 1的光条纹影像LSP 1；利用流程D0，根据光条纹影像LSP 1，产生对应于目标物件OBJ的深度影像DP。相较于现有技术，本申请可在第一方向/维度D1上，取得结构光相位差为第一相位角度θ 1发生的精细位置，其可更精确地计算目标物件OBJ的深度值。更进一步地，三维影像测距系统10利用步骤B04/流程C0计算条纹状结构光在第一方向/维度D1的精细位置，增加目标物件OBJ深度值的精细度。另外，三维影像测距系统10利用同相影像I与正交影像Q之间的影像相位来计算深度值，影像相位与反射光强度无关，因此三维影像测距系统10所计算出的深度值不会受到目标物件OBJ的光反射率的影响。
然而，由图13可知，光条纹影像LSP 1为较为稀疏(Sparse)的影像，而导致深度影像DP也较为稀疏，其中稀疏影像代表该影像中大部份的像素值为0，仅少部份的像素值不为0。为了使深度影像DP更加稠密(Dense)，除了对应于第一相位角度θ 1的光条纹影像LSP 1之外，本申请亦可产生对应于至少一相位角度θ’(可对应至权利要求中的至少一第二相位角度)的至少一光条纹影像LSP’(可对应至权利要求中的至少一第二光条纹影像)，以增加深度影像DP的稠密性。其中，至少一相位角度θ’不同于第一相位角度θ 1，至少一光条纹影像LSP’不同于光条纹影像LSP 1。
具体来说，图15为本申请实施例一流程E0的示意图，流程E0包含以下 步骤：
步骤E02：根据同相影像I及正交影像Q，产生相位影像PHI。
步骤E04：根据相位影像PHI，产生对应于第一相位角度θ 1的光条纹影像LSP 1。
步骤E06：根据相位影像PHI，产生对应于至少一相位角度θ’的至少一光条纹影像LSP’，至少一相位角度θ’不同于第一相位角度θ 1，至少一光条纹影像LSP’不同于光条纹影像LSP 1。
步骤E08：将光条纹影像LSP 1及至少一光条纹影像LSP’整合成为整合影像MG。
步骤E10：根据整合影像MG，产生对应于目标物件OBJ的深度影像DP。
流程E0中的步骤E02、E04与流程B0中的步骤B02、B04，于此不再赘述。
于步骤E06中，运算单元16根据相位影像PHI，产生对应于至少一相位角度θ’的至少一光条纹影像LSP’。于后续的实施例中，将以运算单元16根据相位影像PHI产生分别对应于除了相位角度θ 1以外3个相位角度θ’的3个光条纹影像LSP’为例进行说明，换句话说，运算单元16可根据相位影像PHI，产生分别对应于相位角度θ 2、θ 3、θ 4的光条纹影像LSP 2、LSP 3、LSP 4，其中相位角度θ 2、θ 3、θ 4可分别为(π/2)、(π)、(3π/2)。步骤E06为运算单元16将(特定)相位角度θ 2、θ 3、θ 4代入流程C0中，以产生(特定)光条纹影像LSP 2、LSP 3、LSP 4。关于步骤E06的操作细节，请参考关于流程C0的段落，于此不再赘述。 请参考图16～18，图16为对应于相位角度θ 2(θ 2＝π/2)的光条纹影像LSP 2，图17为对应于相位角度θ 3(θ 3＝π)的光条纹影像LSP 3，图18为对应于相位角度θ 4(θ 4＝3π/2)的光条纹影像LSP 4。光条纹影像LSP 2、LSP  3、LSP  4的特性与光条纹影像LSP 1类似，而于此不再赘述。
于步骤E08中，运算单元16将光条纹影像LSP 1及至少一光条纹影像LSP’整合(Merge)成为整合影像MG。由于第一相位角度θ 1与至少一相位角度θ’不同，光条纹影像LSP 1及至少一光条纹影像LSP’中非0的光条纹影像像素位置皆不相同，因此，运算单元16可产生整合影像MG为光条纹影像LSP 1及至少一光条纹影像LSP’的相加结果。也就是说，承上述实施例，因相位角度θ 1、θ 2、θ 3、θ 4互不相同，光条纹影像LSP 1、LSP 2、LSP 3、LSP 4中非0的光条纹影像像素位置皆不相同，因此，运算单元16可将光条纹影像LSP 1、LSP 2、LSP 3、LSP 4相加，以产生整合影像MG为光条纹影像LSP 1、LSP 2、LSP 3、LSP 4的相加结果，即MG＝LSP 1+LSP 2+LSP 3+LSP 4。
请参考图19，图19为根据绘示于图13、16、17、18的光条纹影像LSP 1、LSP 2、LSP 3、LSP 4而整合成的整合影像MG。由图19可知，整合影像MG较光条纹影像LSP 1稠密。
步骤E10中，运算单元16根据整合影像MG，产生对应于目标物件OBJ的深度影像DP。请参考图20，图20为本申请实施例一流程F0的示意图。流程F0为步骤E10的操作细节，其包含以下步骤：
步骤F02：取得目标物件OBJ对应于像素坐标位置(n,m)的二维图像坐标(x 0,y 0)为(x 0,y 0)＝(m,MG(n,m))。
步骤F04：根据二维图像坐标(x 0,y 0)，取得目标物件OBJ对应于像素坐标位置(n,m)的三维图像坐标(X 0,Y 0,Z 0)。
步骤F06：根据三维图像坐标(X 0,Y 0,Z 0)，计算目标物件OBJ对应于像素坐标位置(n,m)的深度影像像素值DP(n,m)。
流程F0与流程D0类似，不同之处在于步骤F02中，坐标值y 0为整合影像像素值MG(n,m)，MG(n,m)为整合影像MG于像素坐标位置(n,m)的像素值，整合影像像素值MG(n,m)不为0。其余关于流程F0的操作细节，请参考关于流程D0的段落，于此不再赘述。
简言之，三维影像测距系统10利用流程E0，产生对应于相位角度θ 1～θ 4的光条纹影像LSP 1～LSP 4，并将光条纹影像LSP 1～LSP 4整合成为较为稠密的整合影像MG。因此，运算单元16执行流程E0、F0所产生的深度影像DP亦较为稠密，可增加深度影像DP的非0深度影像像素的个数。
需注意的是，流程D0、F0皆是单纯利用三角测量法计算目标物件OBJ的深度/距离。然而，三角测量法具有光平面模糊(Light Plane Ambiguity)的问题。请参考图21，图21为发光模块12、感光模块14以及位置OA、OB、OC的相对位置关系图。简单来说，光平面模糊的问题为三维影像测距系统会无法分辨目标物件OBJ是位于位置OA或是位置OB亦或是位置OC(而位置OA、OB、 OC相对于发光模块12分别具有不同的角度α A、α B、α C)，反而影响测距精准度。
为了解决光平面模糊的问题，三维影像测距系统10可利用飞时测距法先取得对应于目标物件的飞时距离(感光像素电路144由感光像素电路80来实现)，再根据飞时距离计算出三角测量法所需的角度α，最后根据该角度α利用三角测量法计算对应目标物件OBJ的深度/距离。其中，角度α为目标物件OBJ相对于发光模块12的俯仰角，即目标物件OBJ、发光模块12与感光像素阵列142之间的夹角。
请参考图22，图22为本申请实施例一流程G0的示意图。流程G0可由运算单元16来执行，流程G0包含以下步骤：
步骤G02：根据第一影像P1及第四影像P4，产生对应于目标物件OBJ的飞时距离D ToF。
步骤G04：根据飞时距离D ToF及光条纹影像LSP 1，决定角度α_opt，其中角度α_opt代表目标物件OBJ、发光模块12与感光像素阵列142之间的夹角。
步骤G06：根据整合影像MG以及角度α_opt，产生对应于目标物件OBJ的深度影像DP。
于步骤G02中，运算单元16可根据第一影像P1及第四影像P4先取得飞时影像TF(其中感光像素电路144以感光像素电路80来实现)，再根据飞时 影像TF计算对应于目标物件OBJ的飞时距离D ToF。于一实施例中，运算单元16可计算飞时影像TF为TF＝(P4-P3)/(P1+P4-2*P3)*(c*T)(公式8)，其中c代表光速，T代表传输闸的导通区间的时间长度，换句话说，运算单元16可产生飞时影像TF中第(n,m)个同相影像像素值TF(n,m)为(Pout4-Pout3)/(Pout1+Pout4-2*Pout3)*(c*T)。飞时影像TF中每一飞时影像像素值TF(n,m)代表于像素坐标位置(n,m)目标物件OBJ与三维影像测距系统10之间的飞时距离。另外，运算单元16可产生飞时距离D ToF为飞时影像TF的一统计值，如运算单元16可产生飞时距离D ToF为飞时影像TF的平均值、最大值、中位数或众数等统计值。另外，公式8/飞时测距法的细节为本领域技术人员所知，于此不再赘述。
于步骤G04中，运算单元16可根据飞时距离D ToF及光条纹影像LSP 1，决定角度α_opt。运算单元16可事先取得可能的多个角度α (1)～α (K)，角度α (1)～α (K)可相关于绕射单元DE的特性。运算单元16可根据事先取得的角度α (1)～α (K)， -分别执行K次流程D0，即将角度α (1)～α (K)-分别代入步骤D04中的公式7，进而产生K个利用三角测量法产生的距离D Δ,(1)～D Δ,(K)。运算单元16可将利用三角测量法产生的距离D Δ,(1)～D Δ,(K)与步骤G02所得的飞时距离D ToF进行比对，以取得D Δ,(k_opt)为距离D Δ,(1)～D Δ,(K)最接近飞时距离D ToF的距离。运算单元16决定角度α_opt为角度α (1)～α (K)中对应于D Δ,(k_opt)的角度α (k_opt)，即α_opt＝α (k_opt)。
于步骤G06中，运算单元16可根据步骤E08所产生的整合影像MG以及步骤G04所产生的角度α_opt，产生对应于目标物件OBJ的深度影像DP。请 参考图23，图23为本申请实施例一流程H0的示意图。流程H0为步骤G06的操作细节，其包含以下步骤：
步骤H02：取得目标物件OBJ对应于像素坐标位置(n,m)的二维图像坐标(x 0,y 0)为(x 0,y 0)＝(m,MG(n,m))。
步骤H04：根据二维图像坐标(x 0,y 0)以及角度α_opt，取得目标物件OBJ对应于像素坐标位置(n,m)的三维图像坐标(X 0,Y 0,Z 0)。
步骤H06：根据三维图像坐标(X 0,Y 0,Z 0)，计算目标物件OBJ对应于像素坐标位置(n,m)的深度影像像素值DP(n,m)。
流程H0与流程F0、D0类似，流程H0与流程F0不同之处在于，于步骤H04中，运算单元16将角度α_opt连同步骤H02所取得的(x 0,y 0)代入步骤D04中的公式7，以产生对应于角度α_opt的三维图像坐标(X 0,Y 0,Z 0)，于步骤H06中，运算单元16即可根据步骤H04所取得(对应于角度α_opt)的三维图像坐标(X 0,Y 0,Z 0)，计算深度影像像素值DP(n,m)。其余关于流程H0的操作细节，请参考关于流程F0及流程D0的段落，于此不再赘述。
简言之，三维影像测距系统10利用流程G0取得目标物件OBJ的飞时距离D ToF，并比对三角测量法产生的距离D Δ,(1)～D Δ,(K)与飞时距离D ToF，以决定角度α_opt，并根据整合影像MG以及角度α_opt，产生深度影像DP。换句话说，本申请利用飞时距离D ToF来排除光平面模糊的可能性，同时保有飞时测距法以及三角测量法的优点。
需注意的是，前述实施例用以说明本发明之概念，本领域具通常知识者当可据以做不同的修饰，而不限于此。举例来说，绕射单元不限于包括单一绕射光学元件，请参考图24，图24为本申请实施例一发光模块DE’的示意图。所述绕射单元DE’包括第一绕射子单元DE1及第二绕射子单元DE2，第一绕射子单元DE1可为一绕射光学元件，而第二绕射子单元DE2可为另一绕射光学元件。第一发光单元LE1可朝向第一绕射子单元DE1发光，而第二发光单元LE2可朝向第二绕射子单元DE2发光。换句话说，第一绕射子单元DE1于第一时间T 1中对第一光L1形成绕射作用而产生第一结构光SL1，第二绕射子单元DE2于第二时间T 2中对第二光L2形成绕射作用而产生第二结构光SL2，亦属于本申请的范畴。
综上所述，本申请利用包括结构光相位差为(π/2)或(3π/2)的第一结构光及第二结构光，计算影像相位；并根据影像相位，计算条纹状结构光在(与条纹状结构光垂直的)第一方向/维度的精细位置。相较于现有技术，本申请可精确地取得目标物件的深度信息。
以上所述仅为本申请的部分实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本申请的保护范围之内。

Claims (33)

  1. 一种三维影像测距系统，其特征在于，包括：
    发光模块，包括：
    绕射单元；
    第一发光单元，用来于第一时间中发射第一光至所述绕射单元，所述绕射单元于所述第一时间中对所述第一光形成绕射作用而产生第一结构光；
    第二发光单元，用来于第二时间中发射第二光至所述绕射单元，所述绕射单元于所述第二时间中对所述第二光形成绕射作用而产生第二结构光，其中所述第一结构光与所述第二结构光之间的结构光相位差为(π/2)的奇数倍；
    感光像素阵列，用来于所述第一时间接收对应于所述第一结构光的反射光，以产生第一影像，并于所述第二时间接收对应于所述第二结构光的反射光，以产生第二影像；
    运算单元，耦接于所述感光像素阵列，用来执行以下步骤：
    根据所述第一影像及所述第二影像，产生相关于所述第一结构光的同相影像及相关于所述第二结构光的正交影像；以及
    根据所述同相影像及所述正交影像，产生对应于目标物件的深度影像。
  2. 如权利要求1所述的三维影像测距系统，其特征在于，所述绕射单元包括第一绕射子单元及第二绕射子单元，所述第一绕射子单元于所述第一时间中 对所述第一光形成绕射作用而产生所述第一结构光，所述第二绕射子单元于所述第二时间中对所述第二光形成绕射作用而产生所述第二结构光。
  3. 如权利要求1所述的三维影像测距系统，其特征在于，所述第一发光单元接收第一脉冲信号并根据所述第一脉冲信号发射出所述第一光，所述第二发光单元接收第二脉冲信号并根据所述第二脉冲信号发射出所述第二光。
  4. 如权利要求3所述的三维影像测距系统，其特征在于，所述第一脉冲信号及所述第二脉冲信号的占空比小于特定占空比，所述第一发光单元及所述第二发光单元的发光功率大于特定功率。
  5. 如权利要求1所述的三维影像测距系统，其特征在于，所述第一发光单元所发射的所述第一光相对于所述绕射单元具有第一入射角，所述第二发光单元所发射的所述第二光相对于所述绕射单元具有第二入射角，所述第一入射角与所述第二入射角不同，而使得所述第一结构光与所述第二结构光之间的所述结构光相位差为(π/2)的奇数倍。
  6. 如权利要求1所述的三维影像测距系统，其特征在于，所述感光像素阵列包括多个感光像素电路，多个感光像素电路中一第一感光像素电路包括：
    感光元件；
    第一光电读取电路，耦接于所述感光元件，包括：
    第一传输闸，耦接于所述感光元件，其中所述第一传输闸于所述第一时间导通；
    第一输出晶体管，耦接于所述第一传输闸；以及
    第一读取晶体管，耦接于所述第一输出晶体管，用来输出第一输出信号；以及
    第二光电读取电路，耦接于所述感光元件，包括：
    第二传输闸，耦接于所述感光元件，其中所述第二传输闸于所述第二时间导通；
    第二输出晶体管，耦接于所述第二传输闸；以及
    第二读取晶体管，耦接于所述第二输出晶体管，用来输出第二输出信号；
    其中，所述第一影像中对应于所述第一感光像素电路的一第一像素值为所述第一输出信号，所述第二影像中对应于所述第一感光像素电路的一第二像素值为所述第二输出信号；
    其中，所述同相影像中对应于所述第一感光像素电路的第一同相像素值相关于所述第一输出信号，所述正交影像中对应于所述第一感光像素电路的第一正交像素值相关于所述第二输出信号。
  7. 如权利要求6所述的三维影像测距系统，其特征在于，所述第一感光像素电路还包括：
    第三光电读取电路，耦接于所述感光元件，包括：
    第三传输闸，耦接于所述感光元件，其中所述第三传输闸于第三时间导通，于所述第三时间中，所述第一发光单元及所述第二发光单元皆不发光；
    第三输出晶体管，耦接于所述第三传输闸；以及
    第三读取晶体管，耦接于所述第三输出晶体管，用来输出第三输出信号；
    其中，所述同相影像中对应于所述第一感光像素电路的所述第一同相像素值相关于所述第一输出信号减去所述第三输出信号，所述正交影像中对应于所述第一感光像素电路的所述第一正交像素值相关于所述第二输出信号减去所述第三输出信号；
    其中，所述多个感光像素电路所输出的多个第三输出信号形成第三影像。
  8. 如权利要求6所述的三维影像测距系统，其特征在于，所述第一感光像素电路还包括：
    第四光电读取电路，耦接于所述感光元件，包括：
    第四传输闸，耦接于所述感光元件，其中所述第四传输闸于第四时间导通，所述第一时间与所述第四时间相隔时间间隔；
    第四输出晶体管，耦接于所述第四传输闸；以及
    第四读取晶体管，耦接于所述第四输出晶体管，用来输出第四输出信号；
    其中，所述同相影像中对应于所述第一感光像素电路的所述第一同相像素值相关于所述第一输出信号及第四输出信号；
    其中，所述多个感光像素电路所输出的多个第四输出信号形成第四影像；
    其中，所述运算单元根据所述第一影像及所述第四影像，取得对应于所述目标物件的飞时距离。
  9. 如权利要求1所述的三维影像测距系统，其特征在于，所述运算单元用来执行以下步骤，以根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像：
    根据所述同相影像及所述正交影像，产生相位影像，其中所述相位影像代表所述第一结构光所形成的所述同相影像与所述第二结构光所形成的所述正交影像之间的影像相位；
    根据所述相位影像，产生对应于第一相位角度的第一光条纹影像，其中所述第一光条纹影像纪录所述影像相位为所述第一相位角度的坐标位置；以及
    根据所述第一光条纹影像，产生对应于所述目标物件的所述深度影像。
  10. 如权利要求9所述的三维影像测距系统，其特征在于，所述第一相位角度为0。
  11. 如权利要求9所述的三维影像测距系统，其特征在于，所述运算单元用来执行以下步骤，以根据所述相位影像，产生对应于所述第一相位角度的所述第一光条纹影像：
    取得所述相位影像中位于第一像素坐标位置的第一相位影像像素值，并取得所述相位影像中位于第二像素坐标位置的第二相位影像像素值，其中所述第一像素坐标位置于第一维度直接相邻于所述第二像素坐标位置；
    判断所述第一相位角度是否介于所述第一相位影像像素值与所述第二相位影像像素值之间；
    当所述第一相位角度介于所述第一相位影像像素值与所述第二相位影像像素值之间时，根据所述第一相位角度、所述第一相位影像像素值及所述第二相位影像像素值，执行内插运算，以取得内插结果；
    将所述内插结果储存至所述第一光条纹影像的所述第一像素坐标位置，而所述内插结果成为所述第一光条纹影像中于所述第一像素坐标位置的光条纹影像像素值。
  12. 如权利要求11所述的三维影像测距系统，其特征在于，所述运算单元用来执行以下步骤，以根据所述第一相位角度、所述第一相位影像像素值及所述第二相位影像像素值，执行所述内插运算以取得所述内插结果：
    计算所述内插结果为
    Figure PCTCN2018082441-appb-100001
    其中，θ代表所述第一相位角度，(n,m)代表所述第一像素坐标位置，(n-1,m)代表所述第二像素坐标位置，PHI(n,m)代表所述第一相位影像像素值，PHI(n-1,m)所述第二相位影像像素值。
  13. 如权利要求11所述的三维影像测距系统，其特征在于，所述运算单元另用来执行以下步骤，以根据所述相位影像，产生对应于所述第一相位角度的所述第一光条纹影像：
    当所述第一相位角度不介于所述相位影像中所述第一相位影像像素值与所述第二相位影像像素值之间时，所述第一光条纹影像中对应于所述第一像素坐标位置的光条纹影像像素值为0。
  14. 如权利要求9所述的三维影像测距系统，其特征在于，所述运算单元用来执行以下步骤，以根据所述第一光条纹影像，产生对应于所述目标物件的所述深度影像：
    取得所述目标物件对应于第三像素坐标位置的第一二维图像坐标，其中所述第一二维图像坐标为(x 0,y 0)＝(m,LSP 1(n,m))，y 0代表所述第一二维图像坐标于第一维度的坐标值，x 0代表所述第一二维图像坐标于第二维度的坐标值，n代表所述第三像素坐标位置于所述第一维度的坐标值，m代表所述第三像素坐标位置于所述第二维度的坐标值，LSP 1(n,m)代表所述第一光条纹影像于第三像素坐标位置的光条纹影像像素值；
    根据所述第一二维图像坐标，取得所述目标物件对应于所述第三像素坐标位置的第一三维图像坐标；以及
    根据所述第一三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
  15. 如权利要求9所述的三维影像测距系统，其特征在于，所述运算单元另用来执行以下步骤，以根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像：
    根据所述相位影像，产生对应于至少一第二相位角度的至少一第二光条纹影像，其中所述至少一第二相位角度不同于所述第一相位角度，所述至少一第二光条纹影像不同于所述第一光条纹影像影像相位；以及
    将所述第一光条纹影像及所述至少一第二光条纹影像整合成为整合影像；以及
    根据所述整合影像，产生对应于所述目标物件的所述深度影像。
  16. 如权利要求15所述的三维影像测距系统，其特征在于，所述至少一第二相位角度为(2π/L)的整数倍，L为大于1的正整数。
  17. 如权利要求15所述的三维影像测距系统，其特征在于，所述运算单元用来执行以下步骤，以将所述第一光条纹影像及所述至少一第二光条纹影像整合成为所述整合影像：
    产生所述整合影像为所述第一光条纹影像及所述至少一第二光条纹影像的相加结果。
  18. 如权利要求15所述的三维影像测距系统，其特征在于，所述运算单元用来执行以下步骤，以根据所述整合影像，产生对应于所述目标物件的所述深度影像：
    取得所述目标物件于所述整合影像中对应于第四像素坐标位置的第二二维图像坐标，其中所述第二二维图像坐标为(x 0,y 0)＝(m,MG(n,m))，y 0代表所述第二二维图像坐标于第一维度的坐标值，x 0代表所述二维图像坐标于第二维度的坐标值，n代表所述第四像素坐标位置于所述第一维度的坐标值，m代表所述第四像素坐标位置于所述第二维度的坐标值，MG(n,m)代表所述整合影像于第四像素坐标位置的整合影像像素值；
    根据所述第二二维图像坐标，取得所述目标物件对应于所述第三像素坐标位置的第二三维图像坐标；以及
    根据所述第二三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
  19. 如权利要求15所述的三维影像测距系统，其特征在于，所述感光像素阵列于第三时间接收背景光以产生第三影像，所述感光像素阵列于第四时间中接收对应于所述第一结构光的反射光以产生第四影像，所述运算单元另用来执行以下步骤，以根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像：
    根据所述第一影像及所述第四影像，产生对应于所述目标物件的飞时距离；
    根据所述飞时距离及所述第一光条纹影像，决定角度，其中所述角度代表所述目标物件、所述发光模块与所述感光像素阵列之间的夹角；以及
    根据所述整合影像以及所述角度，产生对应于所述目标物件的所述深度影像。
  20. 如权利要求19所述的三维影像测距系统，其特征在于，所述运算单元另用来执行以下步骤，以根据所述整合影像以及所述角度，产生对应于所述目标物件的所述深度影像：
    取得所述目标物件于所述整合影像中对应于第四像素坐标位置的第二二维图像坐标，其中所述第二二维图像坐标为(x 0,y 0)＝(m,MG(n,m))，y 0代表所述第二二维图像坐标于第一维度的坐标值，x 0代表所述二维图像坐标于第二维度的坐标值，n代表所述第四像素坐标位置于所述第一维度的坐标值，m代表所述第四像素坐标位置于所述第二维度的坐标值，MG(n,m)代表所述整合影像于第四像素坐标位置的整合影像像素值；
    根据所述第二二维图像坐标以及所述角度，取得所述目标物件对应于所述第三像素坐标位置的第二三维图像坐标；以及
    根据所述第二三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
  21. 一种三维影像测距方法，应用于三维影像测距系统，其特征在于，所述三维影像测距系统包括发光模块及感光像素阵列，所述发光模块于第一时间中发射第一结构光，并于第二时间中发射第二结构光，所述感光像素阵列于所述第一时间接收对应于所述第一结构光的反射光以产生第一影像，并于所述第二时间接收对应于所述第二结构光的反射光以产生第二影像，其中所述第一结 构光与所述第二结构光之间的结构光相位差为(π/2)的奇数倍，所述三维影像测距方法包括：
    根据所述第一影像及所述第二影像，产生相关于所述第一结构光的同相影像及相关于所述第二结构光的正交影像；以及
    根据所述同相影像及所述正交影像，产生对应于目标物件的深度影像。
  22. 如权利要求21所述的三维影像测距方法，其特征在于，根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像的步骤包括：
    根据所述同相影像及所述正交影像，产生相位影像，其中所述相位影像代表所述第一结构光所形成的所述同相影像与所述第二结构光所形成的所述正交影像之间的影像相位；
    根据所述相位影像，产生对应于第一相位角度的第一光条纹影像，其中所述第一光条纹影像纪录所述影像相位为所述第一相位角度的坐标位置；以及
    根据所述第一光条纹影像，产生对应于所述目标物件的所述深度影像。
  23. 如权利要求22所述的三维影像测距方法，其特征在于，所述第一相位角度为0。
  24. 如权利要求22所述的三维影像测距方法，其特征在于，根据所述相位影像，产生对应于所述第一相位角度的所述第一光条纹影像的步骤包括：
    取得所述相位影像中位于第一像素坐标位置的第一相位影像像素值，并取得所述相位影像中位于第二像素坐标位置的第二相位影像像素值，其中所述第一像素坐标位置于第一维度直接相邻于所述第二像素坐标位置；
    判断所述第一相位角度是否介于所述第一相位影像像素值与所述第二相位影像像素值之间；
    当所述第一相位角度介于所述第一相位影像像素值与所述第二相位影像像素值之间时，根据所述第一相位角度、所述第一相位影像像素值及所述第二相位影像像素值，执行内插运算，以取得内插结果；
    将所述内插结果储存至所述第一光条纹影像的所述第一像素坐标位置，而所述内插结果成为所述第一光条纹影像中于所述第一像素坐标位置的光条纹影像像素值。
  25. 如权利要求24所述的三维影像测距方法，其特征在于，根据所述第一相位角度、所述第一相位影像像素值及所述第二相位影像像素值，执行所述内插运算以取得所述内插结果的步骤包括：
    计算所述内插结果为
    Figure PCTCN2018082441-appb-100002
    其中，θ代表所述第一相位角度，(n,m)代表所述第一像素坐标位置，(n-1,m)代表所述第二像素坐标位置，PHI(n,m)代表所述第一相位影像像素值，PHI(n-1,m)所述第二相位影像像素值。
  26. 如权利要求24所述的三维影像测距方法，其特征在于，根据所述相位影像，产生对应于所述第一相位角度的所述第一光条纹影像的步骤另包括：
    当所述第一相位角度不介于所述相位影像中所述第一相位影像像素值与所述第二相位影像像素值之间时，所述第一光条纹影像中对应于所述第一像素坐标位置的光条纹影像像素值为0。
  27. 如权利要求22所述的三维影像测距方法，其特征在于，根据所述第一光条纹影像，产生对应于所述目标物件的所述深度影像的步骤包括：
    取得所述目标物件对应于第三像素坐标位置的第一二维图像坐标，其中所述第一二维图像坐标为(x 0,y 0)＝(m,LSP 1(n,m))，y 0代表所述第一二维图像坐标于第一维度的坐标值，x 0代表所述第一二维图像坐标于第二维度的坐标值，n代表所述第三像素坐标位置于所述第一维度的坐标值，m代表所述第三像素坐标位置于所述第二维度的坐标值，LSP 1(n,m)代表所述第一光条纹影像于第三像素坐标位置的光条纹影像像素值；
    根据所述第一二维图像坐标，取得所述目标物件对应于所述第三像素坐标位置的第一三维图像坐标；以及
    根据所述第一三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
  28. 如权利要求22所述的三维影像测距方法，其特征在于，根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像的步骤另包括：
    根据所述相位影像，产生对应于至少一第二相位角度的至少一第二光条纹影像，其中所述至少一第二相位角度不同于所述第一相位角度，所述至少一第二光条纹影像不同于所述第一光条纹影像影像相位；以及
    将所述第一光条纹影像及所述至少一第二光条纹影像整合成为整合影像；以及
    根据所述整合影像，产生对应于所述目标物件的所述深度影像。
  29. 如权利要求28所述的三维影像测距方法，其特征在于，所述至少一第二相位角度为(2π/L)的整数倍，L为大于1的正整数。
  30. 如权利要求28所述的三维影像测距方法，其特征在于，将所述第一光条纹影像及所述至少一第二光条纹影像整合成为所述整合影像的步骤包括：
    产生所述整合影像为所述第一光条纹影像及所述至少一第二光条纹影像的相加结果。
  31. 如权利要求28所述的三维影像测距方法，其特征在于，根据所述整合影像，产生对应于所述目标物件的所述深度影像的步骤包括：
    取得所述目标物件于所述整合影像中对应于第四像素坐标位置的第二二维图像坐标，其中所述第二二维图像坐标为(x 0,y 0)＝(m,MG(n,m))，y 0代表所述第二二维图像坐标于第一维度的坐标值，x 0代表所述二维图像坐标于第二维度的坐标值，n代表所述第四像素坐标位置于所述第一维度的坐标值，m代表所述第四像素坐标位置于所述第二维度的坐 标值，MG(n,m)代表所述整合影像于第四像素坐标位置的整合影像像素值；
    根据所述第二二维图像坐标，取得所述目标物件对应于所述第三像素坐标位置的第二三维图像坐标；以及
    根据所述第二三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
  32. 如权利要求28所述的三维影像测距方法，其特征在于，所述感光像素阵列于第三时间接收背景光以产生第三影像，所述感光像素阵列于第四时间中接收对应于所述第一结构光的反射光以产生第四影像，根据所述同相影像及所述正交影像，产生对应于所述目标物件的所述深度影像的步骤另包括：
    根据所述第一影像及所述第四影像，产生对应于所述目标物件的飞时距离；
    根据所述飞时距离及所述第一光条纹影像，决定角度，其中所述角度代表所述目标物件、所述发光模块与所述感光像素阵列之间的夹角；以及
    根据所述整合影像以及所述角度，产生对应于所述目标物件的所述深度影像。
  33. 如权利要求32所述的三维影像测距方法，其特征在于，根据所述整合影像以及所述角度，产生对应于所述目标物件的所述深度影像的步骤另包括：
    取得所述目标物件于所述整合影像中对应于第四像素坐标位置的第二二维图像坐标，其中所述第二二维图像坐标为(x 0,y 0)＝(m,MG(n,m))，y 0代表所述第二二维图像坐标于第一维度的坐标值，x 0代表所述二维图 像坐标于第二维度的坐标值，n代表所述第四像素坐标位置于所述第一维度的坐标值，m代表所述第四像素坐标位置于所述第二维度的坐标值，MG(n,m)代表所述整合影像于第四像素坐标位置的整合影像像素值；
    根据所述第二二维图像坐标以及所述角度，取得所述目标物件对应于所述第三像素坐标位置的第二三维图像坐标；以及
    根据所述第二三维图像坐标，计算所述目标物件对应于所述第三像素坐标位置的深度影像像素值。
PCT/CN2018/082441 2018-04-10 2018-04-10 三维影像测距系统及方法 Ceased WO2019196001A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2018/082441 WO2019196001A1 (zh) 2018-04-10 2018-04-10 三维影像测距系统及方法
CN201880000670.XA CN110612429B (zh) 2018-04-10 2018-04-10 三维影像测距系统及方法

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/082441 WO2019196001A1 (zh) 2018-04-10 2018-04-10 三维影像测距系统及方法

Publications (1)

Publication Number Publication Date
WO2019196001A1 true WO2019196001A1 (zh) 2019-10-17

Family

ID=68163405

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/082441 Ceased WO2019196001A1 (zh) 2018-04-10 2018-04-10 三维影像测距系统及方法

Country Status (2)

Country Link
CN (1) CN110612429B (zh)
WO (1) WO2019196001A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111366945A (zh) * 2020-05-27 2020-07-03 深圳市汇顶科技股份有限公司 基于飞行时间的测距方法和相关测距系统
US12535580B2 (en) 2020-03-25 2026-01-27 Shenzhen GOODIX Technology Co., Ltd. Time-of-flight based distance measuring method and related distance measuring system

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2022016412A1 (zh) * 2020-07-22 2022-01-27 深圳市汇顶科技股份有限公司 深度信息图像采集装置和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1786658A (zh) * 2005-12-29 2006-06-14 清华紫光股份有限公司 一种采用双波长结构光测量物体轮廓的方法及装置
US20160063309A1 (en) * 2014-08-29 2016-03-03 Google Inc. Combination of Stereo and Structured-Light Processing
CN106289092A (zh) * 2015-05-15 2017-01-04 高准精密工业股份有限公司 光学装置及其发光装置
US20170236014A1 (en) * 2014-11-05 2017-08-17 Trw Automotive U.S. Llc Augmented object detection using structured light
CN107690565A (zh) * 2017-08-14 2018-02-13 深圳市汇顶科技股份有限公司 三维影像系统及电子装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1786658A (zh) * 2005-12-29 2006-06-14 清华紫光股份有限公司 一种采用双波长结构光测量物体轮廓的方法及装置
US20160063309A1 (en) * 2014-08-29 2016-03-03 Google Inc. Combination of Stereo and Structured-Light Processing
US20170236014A1 (en) * 2014-11-05 2017-08-17 Trw Automotive U.S. Llc Augmented object detection using structured light
CN106289092A (zh) * 2015-05-15 2017-01-04 高准精密工业股份有限公司 光学装置及其发光装置
CN107690565A (zh) * 2017-08-14 2018-02-13 深圳市汇顶科技股份有限公司 三维影像系统及电子装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12535580B2 (en) 2020-03-25 2026-01-27 Shenzhen GOODIX Technology Co., Ltd. Time-of-flight based distance measuring method and related distance measuring system
CN111366945A (zh) * 2020-05-27 2020-07-03 深圳市汇顶科技股份有限公司 基于飞行时间的测距方法和相关测距系统
CN111366945B (zh) * 2020-05-27 2020-10-16 深圳市汇顶科技股份有限公司 基于飞行时间的测距方法和相关测距系统

Also Published As

Publication number Publication date
CN110612429A (zh) 2019-12-24
CN110612429B (zh) 2021-03-26

Similar Documents

Publication Publication Date Title
WO2021008209A1 (zh) 深度测量装置及距离测量方法
WO2021120403A1 (zh) 一种深度测量装置及测量方法
US10677923B2 (en) Optoelectronic modules for distance measurements and/or multi-dimensional imaging
WO2021120402A1 (zh) 一种融合的深度测量装置及测量方法
CN106256124B (zh) 结构化立体
WO2021051478A1 (zh) 一种双重共享tdc电路的飞行时间距离测量系统及测量方法
JP5633058B1 (ja) ３次元計測装置及び３次元計測方法
CN107917701A (zh) 基于主动式双目立体视觉的测量方法及rgbd相机系统
CN111308482B (zh) 基于编码调制图像的滤波连续波飞行时间测量
WO2021051479A1 (zh) 一种基于插值的飞行时间测量方法及测量系统
WO2021051481A1 (zh) 一种动态直方图绘制飞行时间距离测量方法及测量系统
KR20130037152A (ko) 패턴 광을 이용한 깊이 정보 획득 장치 및 방법
TWI898971B (zh) 飛行時間感測系統中使用的圖像感測器
CN107860337B (zh) 基于阵列相机的结构光三维重建方法与装置
WO2021051480A1 (zh) 一种动态直方图绘制飞行时间距离测量方法及测量系统
WO2019153626A1 (zh) 深度图像引擎及深度图像计算方法
CN110378971A (zh) 一种图像对齐精度的检测方法及装置、设备、存储介质
WO2022241942A1 (zh) 一种深度相机及深度计算方法
CN110612429B (zh) 三维影像测距系统及方法
CN105681687A (zh) 图像处理设备以及包括图像处理设备的移动相机
CN110441785A (zh) 时间飞行距离测量系统
CN111815695B (zh) 深度图像获取方法、装置、移动终端及存储介质
CN105303572B (zh) 基于主被动结合的深度信息获取方法
WO2023071650A1 (zh) 深度相机、制造光发射模组的方法和终端
CN108036742B (zh) 线条结构光三维传感方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18914724

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18914724

Country of ref document: EP

Kind code of ref document: A1