CN109951625A - Color depth imaging sensor, imaging device, forming method and color depth image acquiring method - Google Patents
Color depth imaging sensor, imaging device, forming method and color depth image acquiring method Download PDFInfo
- Publication number
- CN109951625A CN109951625A CN201910295258.1A CN201910295258A CN109951625A CN 109951625 A CN109951625 A CN 109951625A CN 201910295258 A CN201910295258 A CN 201910295258A CN 109951625 A CN109951625 A CN 109951625A
- Authority
- CN
- China
- Prior art keywords
- pixel
- tof
- color depth
- imaging sensor
- group
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 106
- 238000000034 method Methods 0.000 title claims abstract description 42
- 230000005540 biological transmission Effects 0.000 claims description 43
- 238000006243 chemical reaction Methods 0.000 claims description 20
- 238000009792 diffusion process Methods 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 12
- 230000011664 signaling Effects 0.000 claims description 12
- 239000000758 substrate Substances 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 6
- 230000005611 electricity Effects 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 238000004519 manufacturing process Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 6
- 239000003990 capacitor Substances 0.000 description 5
- 238000010587 phase diagram Methods 0.000 description 4
- 230000003321 amplification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000010363 phase shift Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 1
- 239000013078 crystal Substances 0.000 description 1
- 229910021419 crystalline silicon Inorganic materials 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Landscapes
- Color Television Image Signal Generators (AREA)
- Solid State Image Pick-Up Elements (AREA)
Abstract
The application provides a kind of color depth imaging sensor, imaging device, the forming method of color depth imaging sensor and color depth image acquiring method, color depth imaging sensor includes multiple along capable and arranged along the column pixel group, pixel group includes the pixel of N row M column, pixel includes: R pixel, G pixel, B pixel and TOF pixel, wherein, pixel group includes that an at least R pixel, G pixel, B pixel and TOF pixel, R pixel, G pixel, B pixel and TOF pixel is the pixel group of N row M column according to preset order permutation and combination.The application can get cromogram and depth map by one piece of imaging sensor, the problem of eliminating while obtaining using color image sensor imaging device and depth image sensor imaging device bulky brought by cromogram and depth map, power hungry and the equipment required to volume and power consumption can not be adapted to, manufacturing cost is reduced, so that the application range of product is more extensive.
Description
Technical field
This application involves image domains more particularly to color depth imaging sensor, imaging device, color depth image to pass
The forming method and color depth image acquiring method of sensor.
Background technique
In the prior art, RGB color sensor is using one piece of CCD or CMOS as color sensor, and dot structure is such as
Shown in Fig. 1, each pixel only has a kind of color, other two color can be multiplexed adjacent pixel, while in sensor photosensitive
Surface covers one layer of color filter array, estimates missing by the intensity dependence using each pixel and surrounding pixel point
Other two kinds of colors carry out colour reproduction, this pixel format is known as Bayer format.It is noted that in the neighbour of a 2*2
In domain, R pixel and B pixel each one, but there are two G pixels.
TOF sensor workflow as shown in Fig. 2, its can to target actively launch the optical signal through ovennodulation, then
Receive the optical signal that is reflected back by depth transducer, convert charge accumulated for effect of light intensity, further according to transmitting light with
The phase difference of light modulation is solved, operation is carried out and is converted, the distance of target, i.e. depth map can be obtained.
One imaging device cannot obtain cromogram and depth map simultaneously, if cromogram and depth map need to be obtained, need
There are two imaging devices, that is, need a color image sensor imaging device and a depth image sensor imaging device.Such as
Fruit uses color image sensor imaging device and depth image sensor imaging device simultaneously, then in some pairs of volumes, power consumptions
It is required that in very harsh equipment, undoubtedly a very big challenge, also, use two pieces of imaging sensor imaging device costs of manufacture
It is relatively high to be unfavorable for volume and power consumption integrated and occupy for height.
Summary of the invention
The application provides the formation side of a kind of color depth imaging sensor, imaging device, color depth imaging sensor
Method and color depth image acquiring method can need if obtaining cromogram and depth map simultaneously when solving imaging now
It wants color image sensor imaging device and depth image sensor imaging device and leads to that equipment volume is excessive, integrated difficulty
It is high, power consumption is big, and can not be using the equipment required to volume and power consumption the problem of.
According to a first aspect of the present application, the application provides a kind of color depth imaging sensor, and color depth image passes
Sensor includes multiple along capable and arranged along the column pixel group, and pixel group includes the pixel of N row M column, and pixel includes: R pixel, G picture
Element, B pixel and TOF pixel, wherein pixel group includes at least a R pixel, G pixel, B pixel and TOF pixel, R pixel, G
Pixel, B pixel and TOF pixel are according to the pixel group that preset order permutation and combination is that N row M is arranged.
Preferably, pixel group includes the pixel of two rows two column, and pixel group includes a R pixel, G pixel, B pixel and TOF
Pixel, pixel group is from upper left clockwise by G pixel, R pixel, TOF pixel, the sequence permutation and combination of B pixel.
Preferably, pixel group includes the pixel of two rows four column, and pixel group is from upper left clockwise by G pixel, R pixel, G picture
The sequence permutation and combination of element, R pixel, TOF pixel, B pixel, G pixel, B pixel.
Preferably, pixel group includes an at least pixel group and a TOF pixel, and the area of the TOF pixel in pixel group is pressed
The plain area setting ratio that takes pictures is configured.
Preferably, pixel group includes three pixel groups and a TOF pixel, wherein pixel group includes the picture of two rows two column
The area of element, TOF pixel is identical as the area of pixel group, in pixel according to from upper left clockwise by pixel group, pixel group,
TOF pixel, the sequence permutation and combination of pixel group, each pixel group include a R pixel, two G pixels and a B pixel, pixel group
It is arranged clockwise by the sequence of G pixel, R pixel, G pixel, B pixel from upper left.
Preferably, color depth imaging sensor includes at least two pixel groups, and the ranks quantity of pixel group is different, includes
At least a kind of difference in the putting in order of the type difference of pixel and pixel.
According to a second aspect of the present application, the application provides a kind of imaging device comprising: pixel array comprising more
A color depth imaging sensor as described above, color depth image sensor output voltage signal；Double sampled unit is used
In the voltage signal of acquisition color depth imaging sensor output；AD conversion unit, for voltage signal to be converted to number
Signal connects double sampled unit；Scanning element, for gating the digital signal of simultaneously output pixel array, connection modulus turns
Change unit；Control unit connects and controls pixel array, double sampled unit, AD conversion unit and scanning element to obtain
Take simultaneously output digit signals；Depth information computing unit is used for the digital signal exported according to color depth imaging sensor
It carries out operation and obtains the depth information of testee to form depth image, connect scanning element；Image processing unit is used
In the RGB structure formation color image that will be formed according to R pixel, G pixel, B pixel in color depth imaging sensor and to coloured silk
Chromatic graph picture is handled, and scanning element is connected；Integrated unit is used to carry out after depth image and Color Image Fusion defeated
Out, depth information computing unit and image processing unit are connected.
Preferably, the TOF pixel in color depth imaging sensor includes: substrate；Photodiode is set to substrate
It is interior, to the stored charge in response to incident reflected light on the photodiode；First voltage signal output module is used
It is voltage signal in the charge conversion in photodiode will be accumulated comprising the first transfer tube and the first floating diffusion region, the
The control terminal of one transmission transistor receives control signal, and the input terminal of the first transmission transistor connects photodiode, and first passes
The output end of defeated transistor output first voltage signal when the control terminal of the first transmission transistor receives control signal, first
The first voltage signal of photodiode is transferred to the first floating diffusion region and saved by transmission transistor；And second voltage
Signal output module, be used to accumulate photodiode charge conversion be voltage signal comprising second transmission crystal
Pipe and the second floating diffusion region, the control terminal of the second transmission transistor receive control signal, the input terminal of the second transmission transistor
Photodiode is connected, the output end of the second transmission transistor is when the control terminal of the second transmission transistor receives control signal
Second voltage signal is exported, the second voltage signal of photodiode is transferred to the second floating diffusion region by the second transmission transistor
It is saved；Wherein, when measuring TOF depth map, the control signal and modulation light for being sent to the first transmission transistor are set
Phase is identical, and setting is sent to the control signal of the second transmission transistor and the control signal for being sent to the first transmission transistor
Phase complements, transmitting modulation light to object under test；When being measured using two tap two phase methods, color depth image sensing
Device receive object under test receive modulation light back reflection return reflected light after, obtain first voltage signaling module output electricity
Pressure is the first voltage signal PS0 of first voltage signal output module output of the reflected light when phase is 0 °, obtains the second electricity
The voltage signal of pressure signaling module output is the of second voltage signal output module output of the reflected light when phase is 180 °
Two voltage signal PS1；According to formula:The flight time of modulation light is calculated, wherein TonTo adjust
Level is the high time to light processed in one cycle；When being measured using two taps, four phase method, color depth image is passed
After the photodiode of sensor receives reflected light again, the voltage for obtaining the output of first voltage signaling module is reflected light in phase
The first voltage signal PS2 of first voltage signal output module output when position is 90 °, obtains the output of second voltage signaling module
Voltage be the reflected light of modulation light when phase is 270 ° second voltage signal output module output second voltage signal
PS3；According to formula:The flight time of modulation light is calculated, wherein TtofFor frequency modulation
Light two-way time, TpluseFor the period of modulated optical signal.
According to the third aspect of the application, the application provides a kind of forming method of color depth imaging sensor, method
It include: that color depth imaging sensor is arranged includes multiple along capable and arranged along the column pixel group；It includes N row M that pixel group, which is arranged,
The pixel of column, pixel include: R pixel, G pixel, B pixel and TOF pixel；Pixel group is set and includes at least a R pixel, G picture
Element, B pixel and TOF pixel；It is N row that R pixel, G pixel, B pixel and TOF pixel, which is arranged, according to preset order permutation and combination
The pixel group of M column.
According to the fourth aspect of the application, the application provides a kind of color depth image acquiring method, and method includes: setting
Color depth imaging sensor includes multiple along capable and arranged along the column pixel group, and pixel group includes the pixel of N row M column, pixel
It include: R pixel, G pixel, B pixel and TOF pixel；Be arranged pixel group include an at least R pixel, G pixel, B pixel and
TOF pixel；The pixel group of R pixel, G pixel, B pixel and TOF pixel according to preset order permutation and combination for N row M column is set；
It controls R pixel, G pixel, the rgb pixel structure of B pixel composition and obtains color image；It controls TOF pixel and obtains depth image；It will
Color image merges to form color depth image with depth image.
The beneficial effects of the present application are as follows: it include multiple along capable and arranged along the column by the way that color depth imaging sensor is arranged
Pixel group, it is also both to pass in one block of image that setting pixel group, which includes at least a R pixel, G pixel, B pixel and TOF pixel,
Colour element and depth pixel are arranged on sensor, can be exempted from by being that one piece of imaging sensor can get cromogram and depth map
It has gone to obtain cromogram and depth using color image sensor imaging device and depth image sensor imaging device simultaneously
The problem of scheming brought bulky, power hungry and can not adapting to the equipment required to volume and power consumption, reduces
Manufacturing cost, so that the application range of product is more extensive.
Detailed description of the invention
Fig. 1 is the structural schematic diagram of RGB color sensor；
Fig. 2 is TOF sensor operation principle schematic diagram；
Fig. 3 is the structural schematic diagram of color depth imaging sensor in the application second embodiment；
Fig. 4 is the structural schematic diagram of color depth imaging sensor in the application 3rd embodiment；
Fig. 5 is the structural schematic diagram of color depth imaging sensor in the application fourth embodiment；
Fig. 6 is another structural schematic diagram of color depth imaging sensor in the application fourth embodiment；
Fig. 7 is the structural schematic diagram of color depth imaging sensor in the 5th embodiment of the application；
Fig. 8 is another structural schematic diagram of color depth imaging sensor in the 5th embodiment of the application；
Fig. 9 is the schematic diagram of imaging device in the application sixth embodiment；
Figure 10 is the structural schematic diagram of the TOF pixel of the color depth imaging sensor of the application；
Figure 11 is the flow chart of the forming method of color depth imaging sensor in the 7th embodiment of the application；
Figure 12 is the flow chart of color depth image acquiring method in the 8th embodiment of the application.
Specific embodiment
The application is described in further detail below by specific embodiment combination attached drawing.
The design of the application is: being simutaneously arranged rgb pixel and TOF pixel on the image sensor, can obtain coloured silk simultaneously
Chromatic graph and the accurate depth map of measurement.
Embodiment one:
Fig. 3 to Fig. 9 is please referred to, the application proposes a kind of color depth imaging sensor, color depth imaging sensor packet
Include multiple along capable and arranged along the column pixel group, pixel group includes the pixel of N row M column, and pixel includes: R pixel, G pixel, B picture
Element and TOF pixel.
Wherein, pixel group includes at least a R pixel, G pixel, B pixel and TOF pixel, R pixel, G pixel, B pixel with
And TOF pixel is according to the pixel group that preset order permutation and combination is that N row M is arranged.
Therefore, color depth imaging sensor can be composed of one or more pixel groups, specific as follows:
1, color depth imaging sensor includes same pixel group, those pixel groups are combined along row and along column repeated arrangement
For the color depth imaging sensor, pixel group includes the pixel of N row M column, wherein pixel group includes at least a R pixel, G picture
Element, B pixel and TOF pixel, R pixel, G pixel, B pixel and TOF pixel are N row M column according to preset order permutation and combination
Pixel group.
Alternatively, including an at least pixel group and with TOF pixel in pixel group, the area of the TOF pixel in pixel group is pressed
The plain area setting ratio that takes pictures is configured.It is made of for example, pixel group can be set the pixel that two rows two arrange, TOF pixel
Area it is identical as the area of pixel group, then the area of TOF pixel is four times of area of single pixel.
2, color depth imaging sensor includes a variety of pixel groups, those pixel groups are combined into the coloured silk along row and arranged along the column group
Color depth imaging sensor, color depth imaging sensor include at least two pixel groups, between pixel group there are ranks quantity,
At least a kind of difference in the putting in order of type and/or pixel comprising pixel.
For example, color depth imaging sensor may include the pixel group being composed of the pixel arrangement of N1 row M1 column
A1, the pixel group A2 being composed of the pixel arrangement that N2 row M2 is arranged, wherein N1 and N2, M1 and M2 can be same or different.
In another example pixel group A1 includes a R pixel, a G pixel, a B pixel and a TOF pixel, pixel group A2 includes by two R pictures
Element, three G pixels, two B pixels and a TOF pixel.
The process for obtaining color image and depth image is as follows:
TOF pixel obtains depth image.TOF pixel is collected using two taps, four phase method or two tap two phase methods
Charge needs the capacitor of two collection charges, i.e., double floating diffusion regions (FD), in this way reaction are fast, and ranging is more accurate.By infrared
The phase relation of laser signal and transfer tube (TG), such as 0 °, 90 °, 180 °, 270 °, the quantity of electric charge warp that four phases add up respectively
Cross related double adopt to be calculated, then be converted to digital signal through digital-to-analogue conversion (A/D), find out phase shift convert again photon flight when
Between so obtain depth value.If reading when needs point 2 frames using two phase, reads two phase diagrams and generate a Zhang Shen
Degree figure；If reading when needs point 4 frames using four phases, reads four phase diagrams and generate a depth maps.
Rgb pixel obtains color image.R pixel, G pixel, the rgb pixel structure of B pixel composition only need a collection
The capacitor of charge, by optical lens by objective Scenery Imaging on a sensor, under the action of ambient light, by effect of light
Intensity is converted into charge accumulated, is digital signal by digital-to-analogue conversion (A/D) cell translation after correlated-double-sampling unit sampling,
It is transferred to image processing unit.Image processing unit carries out interpretation to digital signal, digital picture is generated, by image processing unit
Gain control, correction, power amplification, storage and output are carried out, a frame is read.
Embodiment two:
Referring to Fig. 3, color depth imaging sensor includes multiple along row and along the pixel group of column repeated arrangement, pixel group
Pixel including two rows two column.Pixel group includes a R pixel, G pixel, B pixel and TOF pixel, and pixel group is suitable from upper left
Hour hands press G pixel, R pixel, TOF pixel, the sequence permutation and combination of B pixel.
In other embodiments, putting in order for R pixel, G pixel, B pixel and TOF pixel is also possible to from pixel group
Upper left, which is risen, presses TOF pixel, R pixel, G pixel, the sequence permutation and combination of B pixel.R pixel, G pixel, B pixel and TOF pixel
Put in order here with no restriction, can be configured according to specific circumstances.
Color depth sensor in the present embodiment can acquire RGB color figure and TOF depth map simultaneously.It is collected
The resolution ratio of RGB color figure is 2*2=4 times of TOF depth map.
TOF pixel generates depth map and the detailed process of rgb pixel structural generation cromogram please refers to embodiment one
Description, is not repeated herein.
Embodiment three:
Referring to Fig. 4, color depth imaging sensor includes multiple along row and along the pixel group of column repeated arrangement, pixel group
Pixel including two rows four column.Pixel group is from upper left clockwise by G pixel, R pixel, G pixel, R pixel, TOF pixel, B picture
Element, G pixel, the sequence permutation and combination of B pixel.
In other embodiments, putting in order for pixel is also possible to from pixel group upper left by TOF pixel, R pixel, G
Pixel, R pixel, G pixel, B pixel, G pixel, the sequence permutation and combination of B pixel.Pixel puts in order here with no restriction,
It can be configured according to specific circumstances.
Color depth sensor in the present embodiment can acquire RGB color figure and TOF depth map simultaneously.It is collected
The resolution ratio of RGB color figure is 4*2=8 times of TOF depth map.
TOF pixel generates depth map and the detailed process of rgb pixel structural generation cromogram please refers to embodiment one
Description, is not repeated herein.
Example IV:
Referring to Fig. 5, color depth imaging sensor includes multiple along row and along the pixel group of column repeated arrangement, pixel group
Including an at least pixel group and a TOF pixel, the area of the TOF pixel in pixel group according to pixel area setting ratio into
Row setting.
In the present embodiment, pixel group includes three pixel groups and a TOF pixel, wherein the area and pixel of TOF pixel
The area of group is identical.The size of pixel group is the pixel of four rows four column, and pixel group includes the pixel of two rows two column, that is,
It is that the area of TOF pixel is as the area of pixel group, that is, four times of area of pixel.According to from upper left in pixel group
It rises and presses pixel group, pixel group, TOF pixel, the sequence permutation and combination of pixel group clockwise.Each pixel group includes a R pixel, two
G pixel and a B pixel, pixel group are arranged by the sequence of G pixel, R pixel, G pixel, B pixel clockwise from upper left.This reality
RGB color figure and TOF depth map can be acquired simultaneously by applying the color depth sensor in example.Point of collected RGB color figure
Resolution is 4*4=16 times of TOF depth map.Meanwhile because TOF area is bigger, photosensitive effect is more preferable, and depth map effect also can
More preferably, measurement distance is more accurate.
Further, referring to Fig. 6, the pixel group in pixel group also may include TOF pixel, specifically, pixel group packet
R pixel, G pixel, B pixel and TOF pixel are included, pixel group is from upper left clockwise by G pixel, R pixel, TOF pixel, B picture
The sequence arrangement of element.
TOF pixel generates depth map and the detailed process of rgb pixel structural generation cromogram please refers to embodiment one
Description, is not repeated herein.
Embodiment five:
Referring to Fig. 7, color depth imaging sensor includes at least two pixel groups, there are ranks quantity between pixel group
The difference that puts in order of type difference and/or pixel different, comprising pixel.
For example, color depth imaging sensor may include the first pixel being composed of the pixel arrangement of N1 row M1 column
Group A1, the second pixel group A2 that the pixel arrangement including being arranged by N2 row M2 is composed, wherein N1 and N2, M1 and M2 can phases
With or it is different, for example the first pixel group A1 is the pixel group that is composed of pixel arrangement of two rows two column, the second pixel group A2
It is the pixel group that the pixel arrangement of three rows three column is composed.
The type for being also possible to the pixel that pixel group includes is different, for example, pixel group is the combination of pixels of two rows four column
Made of pixel group, the first pixel group includes two R pixels, two B pixels, two G pixels and two TOF pixels, and the second pixel group
Including two R pixels, two B pixels, three G pixels and a TOF pixel.
It is also possible to made of the combination of pixels that pixel group is two rows two column, but the pixel of pixel group and pixel group
The difference that puts in order, for example, being wherein from upper left clockwise by G pixel, R pixel, TOF pixel, B pixel with pixel group
Sequence be arranged in a combination, and another pixel group be from upper left clockwise by TOF pixel, G pixel, R pixel, B pixel it is suitable
Sequence is arranged in a combination.
For example, one of pixel group includes two referring to Fig. 7, color depth imaging sensor includes two kinds of pixel groups
The pixel that row four arranges, pixel group is from upper left clockwise by G pixel, R pixel, G pixel, R pixel, TOF pixel, B pixel, G picture
The sequence permutation and combination of element, B pixel.Another pixel group includes the pixel of two rows three column, and pixel group is clockwise from upper left
By G pixel, R pixel, TOF pixel, B pixel, G pixel, the sequence permutation and combination of B pixel.
Certainly, in other embodiments, the pixel group in color depth imaging sensor may include in embodiment five
The combination of pixel group and other type pixel groups.Referring to Fig. 8, one of pixel group includes three pixels in the present embodiment
Group and a TOF pixel, wherein the area of TOF pixel is identical as the area of pixel group.The size of pixel group is four rows four
The pixel of column, pixel group include the pixel of two rows two column, in pixel group according to from upper left clockwise by pixel group, pixel group,
TOF pixel, the sequence permutation and combination of pixel group.Each pixel group includes a R pixel, two G pixels and a B pixel, pixel group
It is arranged clockwise by the sequence of G pixel, R pixel, G pixel, B pixel from upper left.The picture of the area of TOF pixel and two rows two column
The area of element is identical.Another pixel group includes a R pixel, G pixel, B pixel and TOF pixel, which is from a left side
It has gone up clockwise by G pixel, R pixel, TOF pixel, the sequence permutation and combination of B pixel.
Therefore, pixel group can be a variety of pixels, a variety of ranks, a variety of pixel arrangement sequences various combination obtain.
Here to the pixel group type of color depth imaging sensor, restriction, the actual demand with specific reference to user are not determined.
TOF pixel generates depth map and the detailed process of rgb pixel structural generation cromogram please refers to embodiment one
Description, is not repeated herein.
Embodiment six:
Referring to Fig. 9, the imaging device of the application comprising: pixel array comprising multiple as described above colored deep
Spend imaging sensor, color depth image sensor output voltage signal；Double sampled unit is used to acquire color depth image
The voltage signal of sensor output；AD conversion unit connects double sampled list for voltage signal to be converted to digital signal
Member；Scanning element connects AD conversion unit for gating the digital signal of simultaneously output pixel array；Control unit connects
Pixel array, double sampled unit, AD conversion unit and scanning element are connect and controlled to obtain simultaneously output digit signals；Depth
Information calculating unit is used to carry out operation according to the digital signal that color depth imaging sensor exports to obtain testee
Depth information connects scanning element to form depth image；Image processing unit is used to be passed according to color depth image
R pixel, G pixel, the RGB structure of B pixel composition are formed by color image and handle color image in sensor, connect
Connect scanning element；Integrated unit is used to export after depth image and Color Image Fusion, connects depth information meter
Calculate unit and image processing unit.
The process of TOF pixel sampling depth figure is as follows.TOF pixel uses two taps, four phase method or two tap two phase methods
It is collected charge, needs the capacitor of two collection charges, i.e., the capacitor of double floating diffusion regions (FD), in this way reaction are fast, ranging
It is more accurate.By controlling the phase relation of infrared laser signal and the first transmission transistor TG1, the second transmission transistor TG2, such as
0 °, 90 °, 180 °, 270 °, the quantity of electric charge that four phases add up respectively is calculated by related double adopt, then through digital-to-analogue conversion (A/
D digital signal) is converted to, phase shift is found out and converts again and photon flight time and then obtain depth value.If read using two phase
Point 2 frames are needed when out, a depth map could be generated by reading two phase diagrams；If using four phases, when reading
Point 4 frames are needed, a depth map could be generated by reading four phase diagrams.
The process that rgb pixel structure acquires cromogram is as follows.R pixel, G pixel, the rgb pixel structure of B pixel composition are only
Need one collection charge capacitor, by optical lens by objective Scenery Imaging on a sensor, in the effect of ambient light
Under, charge accumulated is converted by effect of light intensity, after correlated-double-sampling unit sampling, is turned by digital-to-analogue conversion (A/D) chip
It is changed to digital signal, is transferred to image processing unit.Image processing unit carries out interpretation to digital signal, generates digital picture,
Gain control, correction, power amplification, storage and output are carried out by processing circuit, read a frame.
Referring to Fig. 10, the TOF pixel in color depth imaging sensor includes: substrate, photodiode PPD, first
Voltage signal output module 101 and second voltage signal output module 102.
In the present embodiment, substrate 203 includes: p-type single crystalline silicon substrate (p-type substrate) and p-type epitaxial layer
(p-epitaxial layer)。
Photodiode PPD, is set in substrate, to the reflection in response to being incident on photodiode PPD
Light and stored charge.
First voltage signal output module 101, be used to accumulate photodiode PPD charge conversion be voltage letter
Number comprising the control terminal of the first transmission transistor TG1 and the first floating diffusion region FD1, the first transmission transistor TG1 receive control
The input terminal of signal processed, the first transmission transistor TG1 connects photodiode PPD, and the output end of the first transmission transistor TG1 exists
The control terminal of first transmission transistor TG1 exports first voltage signal when receiving control signal, the first transmission transistor TG1 will
The first voltage signal of photodiode PPD is transferred to the first floating diffusion region FD1 and is saved.
Second voltage signal output module 102, be used to accumulate photodiode PPD charge conversion be voltage letter
Number comprising the control terminal of the second transmission transistor TG2 and the second floating diffusion region FD2, the second transmission transistor TG2 receive control
The input terminal of signal processed, the second transmission transistor TG2 connects photodiode PPD, and the output end of the second transmission transistor TG2 exists
The control terminal of second transmission transistor TG2 exports second voltage signal when receiving control signal, the second transmission transistor TG2 will
The second voltage signal of photodiode PPD is transferred to the second floating diffusion region FD2 and is saved.
The process of the TOF pixel sampling depth figure in color depth imaging sensor is said below with reference to Figure 10
It is bright.
Wherein, when measuring TOF depth image, setting is sent to control signal and the modulation of the first transmission transistor TG1
The phase of light is identical, and setting is sent to the control signal of the second transmission transistor TG2 and is sent to the first transmission transistor TG1's
Control the phase complements of signal, transmitting modulation light to object under test；
When being measured using two tap two phase methods, color depth imaging sensor receives object under test and receives
After the reflected light that modulation light back reflection is returned, the voltage for obtaining the output of first voltage signaling module is reflected light when phase is 0 °
First voltage signal output module 101 export first voltage signal PS0, obtain second voltage signaling module output voltage
Signal is the second voltage signal PS1 that second voltage signal output module 102 of the reflected light when phase is 180 ° exports；
According to formula:The flight time of modulation light is calculated, wherein TonExist for modulation light
Level is the high time in a cycle.According to the flight time of modulation light and the light velocity, the depth of TOF depth image can be obtained
Information.
When being measured using two taps, four phase method, the photodiode PPD of color depth imaging sensor is again
After receiving reflected light, the voltage for obtaining the output of first voltage signaling module is first voltage of the reflected light when phase is 90 °
The first voltage signal PS2 that signal output module 101 exports, the voltage for obtaining the output of second voltage signaling module is modulation light
The second voltage signal PS3 that second voltage signal output module 102 of the reflected light when phase is 270 ° exports；
According to formula:The flight time of modulation light is calculated, wherein TtofFor
Modulating light two-way time, TpluseFor the period of modulated optical signal.According to the flight time of modulation light and the light velocity, TOF can be obtained
The depth information of depth image.
Embodiment seven:
Please refer to Figure 11, the forming method of the color depth imaging sensor of the application, comprising:
Step S111: setting color depth imaging sensor includes multiple along capable and arranged along the column pixel group.
Step S112: setting pixel group includes the pixel of N row M column, and pixel includes: R pixel, G pixel, B pixel and TOF
Pixel.
Step S113: setting pixel group includes at least a R pixel, G pixel, B pixel and TOF pixel.
Step S114: setting R pixel, G pixel, B pixel and TOF pixel are N row M column according to preset order permutation and combination
Pixel group.
Embodiment eight:
Figure 12 is please referred to, the application proposes color depth image acquiring method, comprising:
Step S121: setting color depth imaging sensor includes multiple along capable and arranged along the column pixel group, pixel group
Pixel including N row M column, pixel includes: R pixel, G pixel, B pixel and TOF pixel.
Step S122: setting pixel group includes at least a R pixel, G pixel, B pixel and TOF pixel.
Step S123: setting R pixel, G pixel, B pixel and TOF pixel are N row M column according to preset order permutation and combination
Pixel group.
Step S124: control R pixel, G pixel, the rgb pixel structure of B pixel composition obtain color image.
Step S125: control TOF pixel obtains depth image.
Step S126: it merges color image to form color depth image with depth image.
Color depth imaging sensor in the present embodiment please refer to above example two, embodiment three, example IV or
Color depth imaging sensor, no longer describes in this way described in person's embodiment five.
The beneficial effects of the present application are as follows: it include multiple along capable and arranged along the column by the way that color depth imaging sensor is arranged
Pixel group, it is also both to pass in one block of image that setting pixel group, which includes at least a R pixel, G pixel, B pixel and TOF pixel,
Colour element and depth pixel are arranged on sensor, this makes it possible to obtain cromogram and depth simultaneously by one piece of imaging sensor
Figure, eliminate and meanwhile obtained using color image sensor imaging device and depth image sensor imaging device cromogram with
Bulky, power hungry brought by depth map and the problem of the equipment required to volume and power consumption can not be adapted to, drop
Low manufacturing cost, so that the application range of product is more extensive.
It will be understood by those skilled in the art that all or part of the steps of various methods can pass through in above embodiment
Program instructs related hardware to complete, which can be stored in a computer readable storage medium, storage medium can wrap
It includes: read-only memory, random access memory, disk or CD etc..
The foregoing is a further detailed description of the present application in conjunction with specific implementation manners, and it cannot be said that this Shen
Specific implementation please is only limited to these instructions.For those of ordinary skill in the art to which this application belongs, it is not taking off
Under the premise of from the present application design, a number of simple deductions or replacements can also be made.
Claims (10)
1. a kind of color depth imaging sensor, which is characterized in that the color depth imaging sensor include it is multiple along row and
Pixel group arranged along the column, the pixel group include N row M column pixel, the pixel include: R pixel, G pixel, B pixel with
And TOF pixel, wherein the pixel group includes at least one R pixel, G pixel, B pixel and TOF pixel, the R picture
Element, G pixel, B pixel and TOF pixel are according to the pixel group that preset order permutation and combination is that N row M is arranged.
2. color depth imaging sensor as described in claim 1, which is characterized in that the pixel group includes what two rows two arranged
The pixel, the pixel group include a R pixel, G pixel, B pixel and TOF pixel, pixel group up time from upper left
Needle presses G pixel, R pixel, TOF pixel, the sequence permutation and combination of B pixel.
3. color depth imaging sensor as described in claim 1, which is characterized in that the pixel group includes what two rows four arranged
The pixel, the pixel group is from upper left clockwise by G pixel, R pixel, G pixel, R pixel, TOF pixel, B pixel, G picture
The sequence permutation and combination of element, B pixel.
4. color depth imaging sensor as described in claim 1, which is characterized in that the pixel group includes an at least pixel
The area of group and a TOF pixel, the TOF pixel in the pixel group is set according to the area setting ratio of the pixel
It sets.
5. color depth imaging sensor as claimed in claim 4, which is characterized in that the pixel group includes three pixel groups
An and TOF pixel, wherein the pixel group includes the pixel of two rows two column, the area and the picture of the TOF pixel
Element group area it is identical, in the pixel according to from upper left clockwise by the pixel group, pixel group, TOF pixel, pixel group
Sequence permutation and combination, each pixel group includes a R pixel, two G pixels and a B pixel, and the pixel group is from upper left
It rises clockwise by G pixel, R pixel, G pixel, the arrangement of the sequence of B pixel.
6. color depth imaging sensor as described in claim 1, which is characterized in that the color depth imaging sensor packet
Include at least two pixel groups, the ranks quantity of the pixel group is different, type comprising the pixel different and described pixels
At least a kind of difference in putting in order.
7. a kind of imaging device, characterized in that it comprises:
Pixel array comprising multiple color depth imaging sensors as described in claim 1 to 6 any one, the coloured silk
Color depth image sensor output voltage signal；
Double sampled unit is used to acquire the voltage signal of the color depth imaging sensor output；
AD conversion unit connects the double sampled unit for the voltage signal to be converted to digital signal；
Scanning element connects the AD conversion unit for gating and exporting the digital signal of the pixel array；
Control unit connects and controls the pixel array, double sampled unit, AD conversion unit and scanning element to obtain
It takes and exports the digital signal；
Depth information computing unit, the digital signal for being used to be exported according to the color depth imaging sensor carry out operation and obtain
The depth information of testee is obtained to form depth image, connects the scanning element；
Image processing unit is used to be formed according to R pixel, G pixel, B pixel in the color depth imaging sensor
RGB structure forms color image and handles color image, connects the scanning element；
Integrated unit is used to export after the depth image and Color Image Fusion, connects the depth information
Computing unit and image processing unit.
8. imaging device as claimed in claim 7, which is characterized in that the TOF pixel in the color depth imaging sensor
Include:
Substrate；
Photodiode is set in the substrate, in response to incident reflected light on the photodiode
And stored charge；
First voltage signal output module, be used to accumulate the photodiode charge conversion be voltage signal,
Including the first transfer tube and the first floating diffusion region, the control terminal of first transmission transistor receives control signal, and described the
The input terminal of one transmission transistor connects the photodiode, and the output end of first transmission transistor is passed described first
The control terminal of defeated transistor exports first voltage signal when receiving the control signal, first transmission transistor will be described
The first voltage signal of photodiode is transferred to first floating diffusion region and is saved；And
Second voltage signal output module, be used to accumulate the photodiode charge conversion be voltage signal,
Including the second transmission transistor and the second floating diffusion region, the control terminal of second transmission transistor receives control signal, institute
The input terminal for stating the second transmission transistor connects the photodiode, and the output end of second transmission transistor is described the
The control terminal of two transmission transistors exports second voltage signal when receiving the control signal, second transmission transistor will
The second voltage signal of the photodiode is transferred to second floating diffusion region and is saved；
Wherein, when measuring TOF depth map, setting is sent to the phase of the control signal and modulation light of first transmission transistor
Position is identical, and setting is sent to the control signal of second transmission transistor and is sent to the control of first transmission transistor
The phase complements of signal emit the modulation light to object under test；
When being measured using two tap two phase methods, the color depth imaging sensor receives object under test and receives
After the reflected light that the modulation light back reflection is returned, the voltage for obtaining the first voltage signaling module output is the reflected light
The first voltage signal PS0 of first voltage signal output module output when phase is 0 °, obtains the second voltage
The voltage signal of signaling module output is that the second voltage signal output module of the reflected light when phase is 180 ° is defeated
Second voltage signal PS1 out；
According to formula:The flight time of the modulation light is calculated, wherein TonFor the modulation light
Level is the high time in one cycle；
When being measured using two taps, four phase method, the photodiode of the color depth imaging sensor receives again
To after the reflected light, the voltage for obtaining the first voltage signaling module output is the reflected light when phase is 90 °
The first voltage signal PS2 of the first voltage signal output module output obtains the second voltage signaling module output
Voltage is the second electricity of second voltage signal output module output of the reflected light of the modulation light when phase is 270 °
Press signal PS3；
According to formula:The flight time of the modulation light is calculated, wherein TtofFor
The modulating light two-way time, TpluseFor the period of modulated optical signal.
9. a kind of forming method of color depth imaging sensor, which is characterized in that the described method includes:
It includes multiple along capable and arranged along the column pixel group that the color depth imaging sensor, which is arranged,；
The pixel that the pixel group includes N row M column is set, and the pixel includes: R pixel, G pixel, B pixel and TOF pixel；
It includes at least one R pixel, G pixel, B pixel and TOF pixel that the pixel group, which is arranged,；
The picture of the R pixel, G pixel, B pixel and TOF pixel according to preset order permutation and combination for N row M column is set
Plain group.
10. a kind of color depth image acquiring method, which is characterized in that the described method includes:
It includes multiple along capable and arranged along the column pixel group that the color depth imaging sensor, which is arranged, and the pixel group includes N
The pixel of row M column, the pixel includes: R pixel, G pixel, B pixel and TOF pixel；
It includes at least one R pixel, G pixel, B pixel and TOF pixel that the pixel group, which is arranged,；
The picture of the R pixel, G pixel, B pixel and TOF pixel according to preset order permutation and combination for N row M column is set
Plain group；
It controls the R pixel, G pixel, the rgb pixel structure of B pixel composition and obtains color image；
It controls the TOF pixel and obtains depth image；
The color image is merged to form the color depth image with depth image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910295258.1A CN109951625B (en) | 2019-04-12 | 2019-04-12 | Color depth image sensor and imaging device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910295258.1A CN109951625B (en) | 2019-04-12 | 2019-04-12 | Color depth image sensor and imaging device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109951625A true CN109951625A (en) | 2019-06-28 |
| CN109951625B CN109951625B (en) | 2023-10-31 |
Family
ID=67014980
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910295258.1A Active CN109951625B (en) | 2019-04-12 | 2019-04-12 | Color depth image sensor and imaging device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109951625B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112505661A (en) * | 2020-11-23 | 2021-03-16 | Oppo（重庆）智能科技有限公司 | Pixel control method, pixel module, device, terminal and storage medium |
| WO2021248427A1 (en) * | 2020-06-12 | 2021-12-16 | 深圳市汇顶科技股份有限公司 | Depth sensing device and related electronic device, and method for operating depth sensing device |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120133737A1 (en) * | 2010-11-30 | 2012-05-31 | Min Dong-Ki | Image sensor for simultaneously obtaining color image and depth image, method of operating the image sensor, and image processing sytem including the image sensor |
| US20130010072A1 (en) * | 2011-07-08 | 2013-01-10 | Samsung Electronics Co., Ltd. | Sensor, data processing system, and operating method |
| US20130176426A1 (en) * | 2012-01-10 | 2013-07-11 | Ilia Ovsiannikov | Image sensor, method of sensing image, and image capturing apparatus including the image sensor |
| KR20140009774A (en) * | 2012-07-13 | 2014-01-23 | 삼성전자주식회사 | 3d image sensor and system including the same |
| US20140240692A1 (en) * | 2013-02-26 | 2014-08-28 | An-chun Tien | Time-of-flight (tof) system, sensor pixel, and method |
| US20140252437A1 (en) * | 2013-03-11 | 2014-09-11 | Min-Seok Oh | Depth pixel included in three-dimensional image sensor and three-dimensional image sensor including the same |
| US20140347442A1 (en) * | 2013-05-23 | 2014-11-27 | Yibing M. WANG | Rgbz pixel arrays, imaging devices, controllers & methods |
| US20150144790A1 (en) * | 2013-11-27 | 2015-05-28 | Aptina Imaging Corporation | Video and 3d time-of-flight image sensors |
| CN108566524A (en) * | 2018-01-31 | 2018-09-21 | 深圳市光微科技有限公司 | Pixel unit, image sensor chip, imaging system, the forming method of pixel unit and depth information measuring method |
| US20180275255A1 (en) * | 2015-11-26 | 2018-09-27 | Odos Imaging Ltd. | Imaging system, distance measuring device, and method for operating the imaging system and the distance measuring device |
| CN209710192U (en) * | 2019-04-12 | 2019-11-29 | 深圳市光微科技有限公司 | Color depth imaging sensor, imaging device, mobile terminal and display system |
-
2019
- 2019-04-12 CN CN201910295258.1A patent/CN109951625B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120133737A1 (en) * | 2010-11-30 | 2012-05-31 | Min Dong-Ki | Image sensor for simultaneously obtaining color image and depth image, method of operating the image sensor, and image processing sytem including the image sensor |
| US20130010072A1 (en) * | 2011-07-08 | 2013-01-10 | Samsung Electronics Co., Ltd. | Sensor, data processing system, and operating method |
| US20130176426A1 (en) * | 2012-01-10 | 2013-07-11 | Ilia Ovsiannikov | Image sensor, method of sensing image, and image capturing apparatus including the image sensor |
| KR20140009774A (en) * | 2012-07-13 | 2014-01-23 | 삼성전자주식회사 | 3d image sensor and system including the same |
| US20140240692A1 (en) * | 2013-02-26 | 2014-08-28 | An-chun Tien | Time-of-flight (tof) system, sensor pixel, and method |
| US20140252437A1 (en) * | 2013-03-11 | 2014-09-11 | Min-Seok Oh | Depth pixel included in three-dimensional image sensor and three-dimensional image sensor including the same |
| US20140347442A1 (en) * | 2013-05-23 | 2014-11-27 | Yibing M. WANG | Rgbz pixel arrays, imaging devices, controllers & methods |
| US20150144790A1 (en) * | 2013-11-27 | 2015-05-28 | Aptina Imaging Corporation | Video and 3d time-of-flight image sensors |
| US20180275255A1 (en) * | 2015-11-26 | 2018-09-27 | Odos Imaging Ltd. | Imaging system, distance measuring device, and method for operating the imaging system and the distance measuring device |
| CN108566524A (en) * | 2018-01-31 | 2018-09-21 | 深圳市光微科技有限公司 | Pixel unit, image sensor chip, imaging system, the forming method of pixel unit and depth information measuring method |
| CN209710192U (en) * | 2019-04-12 | 2019-11-29 | 深圳市光微科技有限公司 | Color depth imaging sensor, imaging device, mobile terminal and display system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021248427A1 (en) * | 2020-06-12 | 2021-12-16 | 深圳市汇顶科技股份有限公司 | Depth sensing device and related electronic device, and method for operating depth sensing device |
| US11943551B2 (en) | 2020-06-12 | 2024-03-26 | Shenzhen GOODIX Technology Co., Ltd. | Depth-sensing device and related electronic device and method for operating depth-sensing device |
| CN112505661A (en) * | 2020-11-23 | 2021-03-16 | Oppo（重庆）智能科技有限公司 | Pixel control method, pixel module, device, terminal and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109951625B (en) | 2023-10-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11201186B2 (en) | Solid-state imaging device, driving method therefor, and electronic apparatus | |
| KR101977711B1 (en) | Depth sensor, image capturing method thereof and image processing system having the depth sensor | |
| US9491421B2 (en) | Image pickup unit and electronic apparatus | |
| CN102891969B (en) | Image sensing apparatus and its method of operation | |
| CN108462841A (en) | Pel array and imaging sensor | |
| US20140263951A1 (en) | Image sensor with flexible pixel summing | |
| CN106358026A (en) | Image sensor including auto-focusing pixel and image processing system including the same | |
| CN109951625A (en) | Color depth imaging sensor, imaging device, forming method and color depth image acquiring method | |
| US20140104397A1 (en) | Image sensor having pixel architecture for capturing depth image and color image | |
| CN109429024A (en) | Solid state image sensor and picture pick-up device | |
| KR102003322B1 (en) | Method of operating depth pixel included in three-dimensional image sensor | |
| CN209710192U (en) | Color depth imaging sensor, imaging device, mobile terminal and display system | |
| CN109817655A (en) | Pixel unit, image sensor chip, imaging system, forming method with double charge storage organization and the method that fathoms | |
| CN110445998B (en) | Color polarization CIS, image processing method and storage medium | |
| US20180048836A1 (en) | Solid-state imaging device, driving method, and electronic apparatus | |
| Gershon et al. | LARGE FORMAT InSb INFRARED DETECTOR WITH 10 µm PIXELS | |
| TW202114400A (en) | Calibrated photo-detecting apparatus and calibration method thereof | |
| CN101873442A (en) | Solid-state imaging device, camera head and the method that drives this solid-state imaging device | |
| KR20180031288A (en) | Image sensor and imaging device including the same | |
| WO2020072134A9 (en) | Multi-resolution uncooled microbolometer focal plane array | |
| Knipp et al. | Vertically integrated amorphous silicon color sensor arrays | |
| JP6821967B2 (en) | Imaging device | |
| CN101771800A (en) | CMOS (Complementary Metal Oxide Semiconductor) image sensor and imaging method thereof | |
| KR102476968B1 (en) | Image sensor, camera module, mobile terminal and image collection method | |
| Gruev et al. | High resolution CCD polarization imaging sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 230000 8th floor, phase IV, intelligent technology park, No. 3963 Susong Road, Hefei Economic and Technological Development Zone, Anhui Province Applicant after: Optical micro information technology (Hefei) Co.,Ltd. Address before: 518000 Room 102, 1st floor, building 10, tongfuyu industrial city, Taoyuan Street, Nanshan District, Shenzhen City, Guangdong Province Applicant before: SHENZHEN GUANGWEI TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |