CN106643642B - Based on Dual-Aperture range-measurement system - Google Patents
Based on Dual-Aperture range-measurement system Download PDFInfo
- Publication number
- CN106643642B CN106643642B CN201510715859.5A CN201510715859A CN106643642B CN 106643642 B CN106643642 B CN 106643642B CN 201510715859 A CN201510715859 A CN 201510715859A CN 106643642 B CN106643642 B CN 106643642B
- Authority
- CN
- China
- Prior art keywords
- frame
- subframe
- color sub
- aperture diaphragm
- light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C3/00—Measuring distances in line of sight; Optical rangefinders
Abstract
A kind of range-measurement system includes the first aperture diaphragm, the second aperture diaphragm, imaging sensor and processing unit.First aperture diaphragm includes the thin slice of infrared cut of light optical filtering material.Second aperture diaphragm includes the thin slice of opaque material.Described image sensor receives the light for penetrating first aperture diaphragm and second aperture diaphragm and exports picture frame.The processing unit calculates separately the first color sub-frame of described image frame and the convolution of multiple first fuzzy cores to generate the first subframe of multiple blurrings, the second color sub-frame of described image frame and the convolution of multiple second fuzzy cores are calculated separately to generate the second subframe of multiple blurrings, and optimization matching is carried out to the first subframe of the multiple blurring and the second subframe of the multiple blurring.
Description
Technical field
The present invention relates to a kind of optical systems, relate more specifically to a kind of Based on Dual-Aperture range-measurement system.
Background technique
Picture system has become one of application system indispensable on current running gear.Described image system is acquired
Signal other than noting down image, apply also for gesture identification (gesture recognition), proximity sensing
(proximity sensing) or apart from detecting etc..
In the application of distance detecting, such as time-of-flight method (time of flight), Structure light method can be used
(structure light), light-field camera (light field camera), code aperture camera (aperture coded
) etc. camera depth is calculated.
However, the above method all has its limitation.For example, time-of-flight method and Structure light method can consume higher power, because
And less suitable for use in running gear.Light-field camera can reduce image resolution ratio.Code aperture camera can then reduce signal-to-noise ratio
(signal-to-noise ratio)。
In view of this, in need propose a kind of scheme, to solve the above problems.
Summary of the invention
The purpose of the present invention is to provide a kind of Based on Dual-Aperture range-measurement systems, compare green light image and multiple fuzzy cores
The convolution results of (blur kernel) and the convolution results of infrared light image and multiple fuzzy cores, so as to generating depth map.
Another object of the present invention is to provide a kind of Based on Dual-Aperture range-measurement systems, compare blue light images and multiple fuzzy cores
Convolution results and infrared light image and multiple fuzzy cores convolution results, so as to generate depth map.
In order to achieve the above object, the present invention illustrates to provide a kind of range-measurement system, it include the first aperture diaphragm, the second aperture diaphragm
And pixel array.First aperture diaphragm includes the thin slice of infrared cut of light optical filtering material and has the first aperture openings.
Second aperture diaphragm includes the thin slice of opaque material and has the second aperture openings, wherein second aperture openings are big
In first aperture openings.The pixel array is made of infrared light pixel and green light pixel, or by infrared light pixel
And blue light pixel is formed, to receive the light across first aperture diaphragm and second aperture diaphragm.
Present invention explanation also provides a kind of range-measurement system, includes the first aperture diaphragm, the second aperture diaphragm, imaging sensor
And processing unit.First aperture diaphragm includes the thin slice of infrared cut of light optical filtering material and has the first aperture openings.
Second aperture diaphragm includes the thin slice of opaque material and has the second aperture openings, wherein second aperture openings are big
In first aperture openings.Described image sensor is to receive across first aperture diaphragm and second aperture light
The light of door screen, and export picture frame.The processing unit is to be divided into the first color sub-frame and the second color for described image frame
Subframe, calculate separately the convolution of first color sub-frame and multiple first fuzzy cores with generate the first subframe of multiple blurrings,
The convolution of second color sub-frame and multiple second fuzzy cores is calculated separately to generate the second subframe of multiple blurrings and right
The first subframe of the multiple blurring and the second subframe of the multiple blurring carry out optimization matching.
The present invention illustrates also to provide a kind of operation method of range-measurement system.The range-measurement system include the first aperture diaphragm,
Second aperture diaphragm and imaging sensor.First aperture diaphragm is infrared cut of light optical filtering material, second aperture
Diaphragm is opaque material, and the aperture openings of second aperture diaphragm are greater than the aperture openings of first aperture diaphragm.Institute
Operation method is stated to comprise the steps of with described image sensor reception across first aperture diaphragm and second aperture
The light of diaphragm generates picture frame；Calculate separately described image frame the first color sub-frame and multiple first fuzzy cores convolution with
Generate the first subframe of multiple blurrings；Calculate separately the second color sub-frame of described image frame and the convolution of multiple second fuzzy cores
To generate the second subframe of multiple blurrings；The first subframe of the multiple blurring and the second subframe of the multiple blurring are carried out
Matching is optimized in the hope of one group of best match subframe；And depth is acquired according to described group of best match subframe.
In order to which above and other objects, features and advantages of the invention can be become apparent from, will hereafter be illustrated appended by cooperation, in detail
Carefully it is described as follows.In addition, identical component is indicated in explanation of the invention with identical symbol, first stated clearly in this.
Detailed description of the invention
Fig. 1 is the cross-sectional view for the Based on Dual-Aperture range-measurement system that the present invention illustrates embodiment.
Fig. 2 is the relative spectral response of different color light.
Fig. 3 A and 3B are the schematic diagram of the pixel distribution for the pixel array that the present invention illustrates embodiment.
Fig. 4 A and 4B are another schematic diagram of the pixel distribution for the pixel array that the present invention illustrates embodiment.
Fig. 5 is the block schematic diagram for the range-measurement system that the present invention illustrates embodiment.
Fig. 6 is the schematic diagram for illustrating the fuzzy core of range-measurement system of embodiment applied to the present invention.
Fig. 7 is the flow chart of the operation method for the range-measurement system that the present invention illustrates embodiment.
Description of symbols
1 range-measurement system
10 substrates
11 first aperture diaphragms
12 second aperture diaphragms
13 lens carriers
141~143 lens
15 imaging sensors
150 pixel arrays
16 infrared cut of light optical filters
17 lens barriers
L1, L2 light beam
Specific embodiment
Fig. 1 is please referred to, the cross-sectional view of the Based on Dual-Aperture range-measurement system of embodiment is illustrated for the present invention.The ranging of the present embodiment
System 1 (is for example shown herein comprising substrate 10, the first aperture diaphragm 11, the second aperture diaphragm 12, lens carrier 13, lens group
Comprising three lens 141~143, but it is not limited to this), imaging sensor 15, infrared cut of light optical filter 16 and lens screen
Barrier 17.
The lens carrier 13 for example substantially has a cylindrical-shaped structure, the multiple lens 141~143, described first
Aperture diaphragm 11 and second aperture diaphragm 12 are then set in the lens carrier 13；Wherein, the lens shown in FIG. 1
The number and type (such as convex lens or concavees lens) of the included lens of group are only to illustrate, and number and type are according to different application
Depending on, have no specific limitation.The lens barrier 17 to protection setting in the lens group in the lens carrier 13, and
Enter the lens carrier 13 for extraneous light with aperture and propagates to the described image sensing positioned at the lens carrier 13
Device 15.
First aperture diaphragm (aperture stop) 11 includes the thin slice (sheet) of infrared cut of light optical filtering material
And there are the first aperture openings；Wherein, the thin slice is, for example, cyclic annular (ring shape) and to stop infrared light.More specifically
It, first aperture openings are the opening of the thin slice, and are located substantially at the center of the thin slice, and but not limited to this.Institute
The first aperture openings such as substantially circular open is stated, but is not limited thereto.In one embodiment, the infrared cut of light filter
The cutoff wavelength of finish matter is 550~650 nanometers, described thin to stop the light beam (such as L2) of the cutoff wavelength or more to penetrate
Piece；Wherein, the cutoff wavelength wants received visible light to determine according to described image sensor 15.Institute referring to figure 2.
Show, is the relative spectral response of different color light.For example, when described image sensor 15 is mainly to sense blue light (blue)
When, shorter (e.g., about 550 nanometers) may be selected in the cutoff wavelength, and when described image sensor 15 is mainly green to sense
When light (green), longer (e.g., about 650 nanometers) are may be selected in the cutoff wavelength, with the interference between elimination and infrared light.
Second aperture diaphragm 12 includes the thin slice of opaque material and has the second aperture openings；Wherein, described thin
Piece is, for example, cyclic annular to stop whole coloured light (such as stopping all spectrum shown by Fig. 2).More specifically, second light
Circle opening is the opening of the thin slice, and is located substantially at the center of the thin slice, and but not limited to this.Second aperture openings
Such as generally circular opening, but be not limited thereto.
In the present embodiment, in order to form Based on Dual-Aperture, second aperture openings are greater than first aperture openings.Whereby,
The aperture that infrared light passes through then is different from the aperture that green/blue light is passed through.As shown in figure 1, light beam L1 is comprising feux rouges, green to example
The light beam of light, blue light and infrared light ingredient, and light beam L2 mainly includes green light and blue light ingredient but does not include infrared light ingredient (quilt
Second aperture diaphragm 12 is obstructed) or the infrared light ingredient that is included it is relatively green/ratio of blue light ingredient is very low.Cause
This, the effective f-number of infrared light is first aperture openings；And the effective f-number of green light and blue light is that second aperture is opened
Mouthful.As can be seen from Figure 2, the region that the spectrum of feux rouges (Red) and infrared light (IR) overlaps each other is more, therefore in present invention explanation,
Described image sensor 15 is set as not sensing red-light spectrum energy, with the interference between reduction and infrared light.
Although it will be appreciated that show that second aperture diaphragm 12 is closer to described image sensor 15 in Fig. 1,
So it is only to illustrate and is not intended to limit the invention explanation.In one embodiment, first aperture diaphragm 11 and described
The position of two aperture diaphragms 12 is interchangeable.In another embodiment, first aperture diaphragm 11 and second aperture diaphragm 12
It can be formed on same transparent plate (plate), and through shading different in formation on the transparent plate (such as coating)
Layer (such as described second aperture diaphragm 12) and filter layer (such as described first aperture diaphragm 11) are described in formation
First aperture openings and second aperture openings.
Described image sensor 15 and the lens carrier 13 are set on the substrate 10.The lens carrier 13 can fit
When mode is incorporated into the substrate 10, such as the modes such as engaging, bonding, have no specific limitation.Described image sensor 15 can be with
Appropriate ways are set to the substrate 10, as long as described image sensor 15 is electrically connected at the substrate 10 to transmit electric signal
, have no specific limitation.Such as Fig. 1 shows that described image sensor 15 is electrically coupled to the substrate through the mode of routing
10, but be not limited thereto.One end (such as close to one end of the substrate 10 in figure) of the lens carrier 13, which has, to be accommodated
Space, described image sensor 15 and its pixel array 150 are then located in the accommodation space.
Described image sensor 15 is, for example, image sensing chip (image sensing chip), preferably includes active
Image sensing element, such as cmos image sensing element, but not limited to this.Described image sensor 15 is with frame frequency (frame
Rate it) receives across first aperture diaphragm 11 and the light of second aperture diaphragm 12, and exports picture frame.
A and 3B referring to figure 3. illustrates the schematic diagram of the pixel distribution of the pixel array 150 of embodiment for the present invention.Institute
State imaging sensor 15 include pixel array 150, it includes multiple pixels with array arrangement to receive across the lens group,
The light of first aperture diaphragm 11 and second aperture diaphragm 12, and export picture frame；Wherein, the pixel array 150
Size have no specific limitation.
In the present embodiment, the pixel array 150 is by multiple first colored pixels 151 and multiple second colored pixels 153
It is formed.For example, first color is green light or blue light, second color is infrared light.In present invention explanation, it is
Interference between reduction and infrared light, first colored pixels 151 do not include red light pixel.Therefore, first color
Pixel 151 is only selected as one of green light pixel and blue light pixel.It will be appreciated that the different colours pixel tool
Have identical circuit structure, and only thereon formed different color light filter layer so that in pixel circuit photosensitive element (such as
Optical diode) sensing different color light light energy.
In the pixel array 150, the multiple first colored pixels 151 and the multiple second colored pixels 153 compared with
Good be distributed with chessboard is arranged, with the subsequent interpolative operation of benefit.Referring to shown in Fig. 4 A and 4B, first colored pixels 151 are along right
The setting of linea angulata direction, and adjacent second colored pixels 153 respectively.
It should be noted that multiple first colored pixels 151 described in present invention explanation and the second colored pixels 153 are not wrapped
Virtual pixel (dummy pixel) containing the pixel array 150.In some embodiments, the edge columns of the pixel array 150
Settable virtual pixel is as one of the mode for eliminating noise.
The infrared cut of light optical filter 16 is set to 150 top of pixel array of described image sensor 15, and has
Cutoff wavelength is about 810 nanometers.It will be appreciated that the cutoff wavelength is sensed according to the multiple second colored pixels 153
Target spectrum depending on, be not limited to 810 nanometers.In addition, although Fig. 1 shows that the infrared cut of light optical filter 16 is set to
On the lens carrier 13, but present invention explanation is not limited thereto.In other embodiments, the infrared cut of light optical filter 16
It can be directly coated on the pixel array 150 of described image sensor 15 for an infrared cut of light filter layer, as long as resistance can be reached
Gear long-wavelength infrared light does not expose to the pixel array 150 of described image sensor 15, and set-up mode has no specific limit
System.
It as previously mentioned, described image sensor 15 can be image sensing chip, and include the pixel array 150 and place
Manage unit；Wherein, the processing unit is directly to the pixel array 150 pixel data collected (such as digital gray level value
Data) it is post-processed.
Shown in referring to figure 5., the block schematic diagram of the range-measurement system of embodiment is illustrated for the present invention.The survey of the present embodiment
It include pixel array 61, reading circuit 63 and processing unit 65 away from system 6.The pixel array 61 can be above-mentioned pixel array
150 (such as Fig. 3 A and 3B), and include multiple first colored pixels 651 and multiple second colored pixels 653.The reading circuit 63
Such as the ash of each pixel of the pixel array 61 is sequentially read according to clock signal (such as produced by sequence controller)
Rank Value Data, such as using double sampling method (the correlated double sampling) reads pixel datas of correlation, but simultaneously
It is not limited.The processing unit 65 is, for example, digital signal processor (DSP), to according to read pixel data meter
It calculates and exports depth data (such as depth map) to external electronic 9 and carry out corresponding control；Wherein, the electronic device 9
For example, portable electronic device or wearable electronic device etc..The processing unit 65 is for example comprising analog digital converting unit
To carry out digital conversion.
The processing unit 65 receives the pixel data for the picture frame F that the reading circuit 63 is exported and handles respectively
One color pixel data and the second color pixel data.For example, the processing unit 65 is by multiple first color pixel data shapes
Be formed as the second color sub-frame as the first color sub-frame (subframe) and by multiple second color pixel datas；Wherein, institute
Stating pixel data is the numerical data after conversion.As previously mentioned, the first color is green light or blue light, the second color is infrared light.
For example, Fig. 5 shows that first color sub-frame is green/blue subframes (G/B subframe) and second color sub-frame is infrared
Photon frame (IR subframe).
The processing unit 65 include the first interpolation unit 6511, the second interpolation unit 6521, first denoising unit 6513,
Second denoising unit 6523, the first compensating unit 6515, the second compensating unit 6525, the first convolution calculator 6517, volume Two
Product calculator 6527, matching primitives device 653 and storage unit 655.It should be noted that though Fig. 5 is by the first interpolation list
First 6511, second interpolation unit 6521, first denoising unit 6513, second denoises unit 6523, the first compensating unit 6515, the
Two compensating units 6525, the first convolution calculator 6517, the second convolution calculator 6527 and matching primitives device 653 are with different function
Square shows, however the multiple function block can be considered and be performed by the processing unit 65, and the multiple function
Square can have no specific limitation with software and or hardware realization.
The storage unit 655 is, for example, memory or buffer, to prestore multiple first fuzzy cores, multiple second moulds
Paste core and the multiple first fuzzy core and the relevant depth information of the multiple second fuzzy core.Referring for example to shown in Fig. 6,
It shows the schematic diagram for being applied to the fuzzy core for the range-measurement system that the present invention illustrates embodiment.The storage unit 655 prestores
Relative to blue light fuzzy core (blur kernel) K1 of different distance, such as N1~N4, and prestore relative to different distance
Infrared light fuzzy core K2, such as N1~N4.Each fuzzy core N1~N4 be, for example, in advance with the range-measurement system 1 relatively not
The fuzzy core that the monochromatic light (such as blue light and infrared light) of same distance is measured and stored.Such as N1 is counterpart away from 5 centimetres, N2
It is counterpart away from 10 centimetres, N3 for counterpart away from 15 centimetres, N4 is counterpart away from 20 centimetres.It will be appreciated that being obscured in Fig. 6
The number and corresponding object distance of core K1 and K2 are only to illustrate, and are not intended to limit the invention explanation.In addition, green light is fuzzy
Core can also be prestored with same way.
The reading circuit 63 sequentially reads each pixel data of the pixel array 61, such as from first row first
Row is sequentially read to last column last line, and the pixel data is sent to the processing unit 65.The processing is single
Picture frame F (it includes the first color pixel data and the second color pixel datas) points that member 65 exports the pixel array 61
It is segmented into the first color sub-frame and the second color sub-frame；For example, by the pixel data shape of relatively the multiple first colored pixels 651
The second color is formed at first color sub-frame and by the pixel data of relatively the multiple second colored pixels 652
Frame.
Since the multiple first colored pixels 651 are only the partial pixel of the pixel array 61, the processing unit
65 the first interpolation unit 6511 is then to described image frame F (pixel datas of i.e. relatively the multiple first colored pixels 651)
Interpolative operation is carried out to generate the first color sub-frame, so that the valid pixel of first color sub-frame and the pixel array 61
With identical size.For example, the pixel data of the pixel data interpolation out position (1,1) using position (1,2) and (2,1)；Benefit
With the pixel data of the pixel data interpolation out position (1,3) of position (1,2), (2,3) and (1,4)；And so on.However, interior
Slotting mode is not limited to this.
Since the multiple second colored pixels 653 are only the partial pixel of the pixel array 61, the processing unit
65 the second interpolation unit 6521 is then to described image frame F (pixel datas of i.e. relatively the multiple second colored pixels 653)
Interpolative operation is carried out to generate the second color sub-frame, so that the valid pixel of second color sub-frame and the pixel array 61
With identical size.For example, the pixel number of the pixel data interpolation out position (1,2) using position (1,1), (2,2) and (1,3)
According to；Utilize the pixel data of the pixel data interpolation out position (1,4) of position (1,3), (2,4) and (1,5)；And so on.So
And the mode of interpolation is not limited to this.
First denoising unit 6513 of the processing unit 65 is for example with digital filter (digital filter) to institute
It states the first color sub-frame and is filtered (denoising) processing, to improve picture quality.Similarly, the second denoising of the processing unit 65
Unit 6523 is for example filtered (denoising) processing to second color sub-frame with digital filter.Digital filter can be ripe
Know the filter to handle image, has no specific limitation.
First compensating unit 6515 of the processing unit 65 is to carry out lens shade benefit to first color sub-frame
It repays, and the second compensating unit 6525 of the processing unit 65 is to carry out lens shadow compensation to second color sub-frame.
The information of lens shadow compensation (lens shading compensation) is according to the multiple 141~143 (reference of lens
It Fig. 1) is previously stored in the processing unit 65, such as is stored in the storage unit 655.
In the present embodiment, the processing unit 65 may be selected executes filtering (denoising) handle and lens shadow compensation handle to
It is one of few, to increase the correctness that distance calculates.
First convolution calculator 6517 of the processing unit 655 (such as goes the first color sub-frame of described image frame F
First color sub-frame after making an uproar and compensating) it is more to generate with multiple first fuzzy cores progress convolution algorithm (convolution) respectively
The first subframe of a blurring.Referring for example to shown in Fig. 6, the first convolution calculator 6517 from the storage unit 655 sequentially
It reads blue light fuzzy core K1 (such as N=1~4), and calculates separately first color sub-frame and blue light fuzzy core N=1~4
Convolution is to generate multiple (herein for 4) blurrings the first subframe (blurred first subframe).It will be appreciated that
The number of the first subframe of generated blurring is identical as the number of the fuzzy core K1 prestored.
Second convolution calculator 6527 of the processing unit 655 (such as goes the second color sub-frame of described image frame F
Second color sub-frame after making an uproar and compensating) carry out convolution algorithm with multiple second fuzzy cores respectively to generate multiple the second sons of blurring
Frame.Referring for example to shown in Fig. 6, it is fuzzy that the second convolution calculator 6527 sequentially reads infrared light from the storage unit 655
Core K2 (such as N=1~4), and the convolution of second color sub-frame and infrared light fuzzy core N=1~4 is calculated separately to generate
The second subframe of multiple blurrings (blurred second subframe).It will be appreciated that generated the second son of blurring
The number of frame is identical as the number of the fuzzy core K2 prestored, such as is herein 4.
The matching primitives device 653 of the processing unit 65 then compare the first subframe of the multiple blurring with it is corresponding
The second subframe of the multiple blurring is in the hope of one group of best match subframe.In one embodiment, the multiple blurring the is compared
One subframe and the second subframe of the multiple blurring refer to the first subframe of the multiple blurring and the multiple blurring second
Subframe carries out optimization matching.The matching primitives device 653 is by the first subframe of blurring of the relatively described fuzzy core N1 and obscures
Change the second subframe and carry out subtraction, such as by the first subframe of the blurring and corresponding picture in the second subframe of the blurring
Prime number subtracts each other according to (pixel-by-pixel), to obtain the first difference frame, and is stored in the storage unit 655；By phase
The first subframe of blurring and the second subframe of blurring to the fuzzy core N2 carry out subtraction to obtain the second difference frame, and
It is stored in the storage unit 655；By the second subframe of the first subframe of blurring of the relatively described fuzzy core N3 and blurring
Subtraction is carried out to obtain third difference frame, and is stored in the storage unit 655；By the relatively described fuzzy core N4's
It is blurred the first subframe and the second subframe of blurring carries out subtraction to obtain the 4th difference frame, and be stored in described deposit
Storage unit 655.The matching primitives device 653 then selects first difference frame into the 4th difference frame, pixel number
Summation reckling institute's the first subframe of mutually corresponding one group of blurring and the second subframe of blurring are as one group of best match subframe.
In the present embodiment, first color sub-frame, second color sub-frame, the multiple blue light fuzzy core K1, institute
State multiple infrared light fuzzy core K2, the first subframe of the multiple blurring, the multiple blurring the second subframe and described first
Difference frame to the 4th difference frame has identical size.
The matching primitives device 653 then first fuzzy core according to corresponding to described group of best match subframe and/or
The relevant depth information of second fuzzy core determines depth.For example, when described group of best match subframe is corresponding fuzzy core N=
1, as previously mentioned, the relevant depth information of the fuzzy core N=1 is, for example, 5 centimetres, then the depth D determined is then 5 centimetres.
Similarly, when described group of best match subframe is corresponding fuzzy core N=2~4, then relevant depth D can be determined.
It is not limited to calculate the first subframe of the multiple blurring and the multiple blurring second in addition, optimizing matching
The subtraction of subframe.In other embodiments, optimize matching can also calculate the first subframe of the multiple blurring with it is corresponding
The second subframe of the blurring correlation (correlation), or calculated using other well-known process, however it is not limited to this
Person disclosed in invention description.As previously mentioned, corresponding fuzzy core N=1~4 for meaning relatively the same object distance described herein.On
It states calculating difference frame to be only to illustrate and be not intended to limit the invention explanation, one group of best match subframe refers to that similarity is highest
The second subframe of one group of first subframe of blurring and blurring.
In some embodiments, when similarity (such as the above-mentioned difference of the second subframe of two groups of first subframes of blurring and blurring
The summation of the pixel number of framing) difference it is little when, also can calculate depth in the way of difference.Such as when similarity is situated between
When between N=1 and N=2, depth D may be selected to be 7.5 centimetres.
Finally, the depth D that determines of the processing unit 65 output is to the external electronic 9.
It please refers to shown in Fig. 7, the flow chart of the operation method of the range-measurement system of embodiment is illustrated for the present invention, it includes
The following steps: image taking sensor receives across the light of the first aperture diaphragm and the second aperture diaphragm and generates picture frame (step
S71)；The first color sub-frame of described image frame and the convolution of multiple first fuzzy cores are calculated separately to generate multiple blurrings the
One subframe (step S72)；The second color sub-frame of described image frame and the convolution of multiple second fuzzy cores are calculated separately to generate
The second subframe (step S73) of multiple blurrings；To the first subframe of the multiple blurring and the second subframe of the multiple blurring
Matching optimize in the hope of one group of best match subframe (step S74)；And it is acquired according to described group of best match subframe
Depth (step S75)；Wherein, step S72~S75 is executed by the processing unit 36 using software and/or hardware, such as
It is executed using the function block of Fig. 5.The detailed embodiment of the present embodiment is being described above, therefore repeats no more in this.
As previously mentioned, the operation method of the present embodiment additionally comprises step: in order to increase operation accuracy with processing unit pair
First color sub-frame and second color sub-frame be filtered (denoising) processing and lens shadow compensation processing at least its
One of.
In addition, the processing unit 36 is first right before to first color sub-frame and second color sub-frame processing
The pixel data of the multiple first colored pixels 651 and the pixel data of the multiple second colored pixels 651 carry out interpolation
It handles (interpolation), to generate first color sub-frame and second color sub-frame respectively.As previously mentioned, institute
Processing unit 36 is stated first to carrying out interpolation processing again after the grayscale value data digital of described image frame F.
It should be noted that the present invention illustrates middle enumerated numerical value, such as of the size of pixel array, fuzzy core
Number, lens numbers and distance etc. be only to illustrate, rather than use limit the present invention explanation.In addition, the reading circuit 63 is also not
It is limited to sequentially to read each pixel of the pixel array 61 from first pixel to a last pixel, also can first reads described
The pixel data of multiple first colored pixels 651, then the pixel data of the multiple second colored pixels 653 is read, have no spy
Definite limitation.
In addition, collocation system light source lights extinguishing, the processing unit 65 can first calculate the relatively described system source point
The difference image frame of second picture frame collected when first picture frame collected extinguishes with the system source relatively when bright,
The operation method of above-mentioned range-measurement system is executed to the difference image frame again.In more detail, the picture frame F of above-described embodiment can be with
Difference image frame replaces, and other runnings are all the same, therefore repeats no more in this.
As described above, known various range-measurement systems have different application limitations.Therefore, present invention explanation also proposes one kind
Based on Dual-Aperture range-measurement system (Fig. 1 and 6) and its operation method (Fig. 7) can calculate depth map according only to an image；That is, institute
Stating depth map includes multiple a depth values.Further, since the present invention illustrates the sensing result using only blue green pixel and does not include
Red pixel can be used to reduce the interference between infrared light to improve detecting accuracy.
Although the present invention is disclosed with examples detailed above, however, it is not to limit the invention, technology belonging to any present invention
Technical staff in field, without departing from the spirit and scope of the present invention, when various change and modification can be made.Therefore this hair
Bright protection scope should be subject to the range that claims are protected.
Claims (17)
1. a kind of range-measurement system, includes:
First aperture diaphragm, the thin slice comprising infrared cut of light optical filtering material simultaneously have the first aperture openings；
Second aperture diaphragm, the thin slice comprising opaque material simultaneously has the second aperture openings, wherein second aperture openings
Greater than first aperture openings；
Pixel array is made of infrared light pixel and green light pixel, or is made of infrared light pixel and blue light pixel, is used
To receive across first aperture diaphragm and the light of second aperture diaphragm；And
Processing unit, to
First color sub-frame of the picture frame that the pixel array is exported carries out convolution algorithm with multiple first fuzzy cores respectively
To generate the first subframe of multiple blurrings, first color is green light or blue light,
It is multiple fuzzy to generate that second color sub-frame of described image frame is carried out to convolution algorithm with multiple second fuzzy cores respectively
Change the second subframe, second color is infrared light, and
It compares the first subframe of the multiple blurring and the second subframe of corresponding the multiple blurring is best in the hope of one group
Match subframe.
2. range-measurement system according to claim 1, wherein the cutoff wavelength of the infrared cut of light optical filtering material is 550
~650 nanometers.
3. range-measurement system according to claim 1, wherein it is described compare for calculate first subframe of the blurring and
The subtraction or correlation of the second subframe of a corresponding blurring.
4. range-measurement system according to claim 1, wherein the processing unit also to first color sub-frame and
Second color sub-frame is filtered and lens shadow compensation handles at least one.
5. range-measurement system according to claim 1, wherein the processing unit is also to carry out interpolation to described image frame
Operation is to generate first color sub-frame and second color sub-frame.
6. range-measurement system according to claim 1, wherein the processing unit also includes storage unit, and the storage is single
Member is related to prestore first fuzzy core, second fuzzy core and first fuzzy core and second fuzzy core
Depth information.
7. range-measurement system according to claim 1, wherein the range-measurement system also includes infrared cut of light optical filter, institute
The cutoff wavelength for stating infrared cut of light optical filter is 810 nanometers.
8. range-measurement system according to claim 1, wherein the range-measurement system also includes lens carrier, the lens branch
First aperture diaphragm and second aperture diaphragm is arranged, it is described that there is the lens carrier frame accommodation space to be installed with
Pixel array.
9. a kind of range-measurement system, includes:
First aperture diaphragm, the thin slice comprising infrared cut of light optical filtering material simultaneously have the first aperture openings；
Second aperture diaphragm, the thin slice comprising opaque material simultaneously has the second aperture openings, wherein second aperture openings
Greater than first aperture openings；
Imaging sensor to receive the light across first aperture diaphragm and second aperture diaphragm, and exports image
Frame；And
Processing unit, to
Described image frame is divided into the first color sub-frame and the second color sub-frame, wherein first color be green light or
Blue light, second color are infrared light,
The convolution of first color sub-frame and multiple first fuzzy cores is calculated separately to generate the first subframe of multiple blurrings,
The convolution of second color sub-frame and multiple second fuzzy cores is calculated separately to generate the second subframe of multiple blurrings, and
Optimization matching is carried out to the first subframe of the multiple blurring and the second subframe of the multiple blurring.
10. range-measurement system according to claim 9, wherein optimization matching is calculates each blurring the
The subtraction or correlation of one subframe and corresponding one the second subframe of the blurring.
11. range-measurement system according to claim 9, wherein the processing unit is also according to one group of best match subframe institute
Corresponding first fuzzy core and/or the relevant depth information of second fuzzy core determine depth.
12. range-measurement system according to claim 9, wherein the processing unit is also to first color sub-frame
And second color sub-frame is filtered and lens shadow compensation handles at least one.
13. range-measurement system according to claim 9, wherein the processing unit is also to in the progress of described image frame
Operation is inserted to generate first color sub-frame and second color sub-frame.
14. a kind of operation method of range-measurement system, the range-measurement system includes the first aperture diaphragm, the second aperture diaphragm and image
Sensor, first aperture diaphragm are infrared cut of light optical filtering material, and second aperture diaphragm is opaque material, described
The aperture openings of second aperture diaphragm are greater than the aperture openings of first aperture diaphragm, and the operation method includes:
It is received with described image sensor across the light of first aperture diaphragm and second aperture diaphragm and generates image
Frame；
The first color sub-frame of described image frame and the convolution of multiple first fuzzy cores are calculated separately to generate multiple blurrings the
One subframe, first color are green light or blue light；
The second color sub-frame of described image frame and the convolution of multiple second fuzzy cores are calculated separately to generate multiple blurrings the
Two subframes, second color are infrared light；
The first subframe of the multiple blurring optimize with the second subframe of the multiple blurring and is matched in the hope of one group
Best match subframe；And
Depth is acquired according to described group of best match subframe.
15. operation method according to claim 14, also includes:
First color sub-frame and second color sub-frame are filtered and lens shadow compensation processing at least its
One of.
16. operation method according to claim 14, also includes:
Interpolative operation is carried out to generate first color sub-frame and second color sub-frame to described image frame.
17. operation method according to claim 14, wherein optimization matching is calculates each blurring the
The subtraction or correlation of one subframe and corresponding one the second subframe of the blurring.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910292985.2A CN110220494B (en) | 2015-10-29 | 2015-10-29 | Double-aperture ranging system and operation method thereof |
| CN201510715859.5A CN106643642B (en) | 2015-10-29 | 2015-10-29 | Based on Dual-Aperture range-measurement system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510715859.5A CN106643642B (en) | 2015-10-29 | 2015-10-29 | Based on Dual-Aperture range-measurement system |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910292985.2A Division CN110220494B (en) | 2015-10-29 | 2015-10-29 | Double-aperture ranging system and operation method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106643642A CN106643642A (en) | 2017-05-10 |
| CN106643642B true CN106643642B (en) | 2019-05-10 |
Family
ID=58829682
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510715859.5A Active CN106643642B (en) | 2015-10-29 | 2015-10-29 | Based on Dual-Aperture range-measurement system |
| CN201910292985.2A Active CN110220494B (en) | 2015-10-29 | 2015-10-29 | Double-aperture ranging system and operation method thereof |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910292985.2A Active CN110220494B (en) | 2015-10-29 | 2015-10-29 | Double-aperture ranging system and operation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN106643642B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673026A (en) * | 2008-09-12 | 2010-03-17 | 夏普株式会社 | Camera and imaging system |
| CN102595171A (en) * | 2012-02-03 | 2012-07-18 | 浙江工商大学 | Imaging method and imaging system of dynamic optical fields of multichannel space-time coding apertures |
| CN103827920A (en) * | 2011-09-28 | 2014-05-28 | 皇家飞利浦有限公司 | Object distance determination from image |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5833546B2 (en) * | 2010-11-09 | 2015-12-16 | パナソニック株式会社 | Distance measuring device and distance measuring method |
| WO2012105157A1 (en) * | 2011-02-01 | 2012-08-09 | パナソニック株式会社 | Stereoscopic imaging device and endoscope |
| WO2012170275A1 (en) * | 2011-06-09 | 2012-12-13 | Eastman Kodak Company | Coupled multi-wavelength confocal systems for distance measurements |
| CN103808305B (en) * | 2012-11-07 | 2017-11-07 | 原相科技股份有限公司 | Detecting system |
| JP6292790B2 (en) * | 2013-08-08 | 2018-03-14 | キヤノン株式会社 | Distance detection device, imaging device, and distance detection method |
-
2015
- 2015-10-29 CN CN201510715859.5A patent/CN106643642B/en active Active
- 2015-10-29 CN CN201910292985.2A patent/CN110220494B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101673026A (en) * | 2008-09-12 | 2010-03-17 | 夏普株式会社 | Camera and imaging system |
| CN103827920A (en) * | 2011-09-28 | 2014-05-28 | 皇家飞利浦有限公司 | Object distance determination from image |
| CN102595171A (en) * | 2012-02-03 | 2012-07-18 | 浙江工商大学 | Imaging method and imaging system of dynamic optical fields of multichannel space-time coding apertures |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106643642A (en) | 2017-05-10 |
| CN110220494A (en) | 2019-09-10 |
| CN110220494B (en) | 2021-11-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Monno et al. | A practical one-shot multispectral imaging system using a single image sensor | |
| CN105009568B (en) | For handling the system of visible spectrum image and infrared image, method and the readable medium of non-volatile machine | |
| CN107563971A (en) | A kind of very color high-definition night-viewing imaging method | |
| CN105830090A (en) | A method to use array sensors to measure multiple types of data at full resolution of the sensor | |
| Hu et al. | Convolutional sparse coding for RGB+ NIR imaging | |
| CN109076139A (en) | It is merged for the colour of macroshot and the parallax mask of monochrome image | |
| CN102687502A (en) | Reducing noise in a color image | |
| CN102484721A (en) | Four-channel color filter array pattern | |
| US20190297279A1 (en) | Dual-aperture ranging system | |
| TW200837658A (en) | Method of sharpening using panchromatic pixels | |
| Teranaka et al. | Single-sensor RGB and NIR image acquisition: toward optimal performance by taking account of CFA pattern, demosaicking, and color correction | |
| CN103236433A (en) | Broadband imager | |
| JP6182396B2 (en) | Imaging device | |
| CN103636199B (en) | Three-dimensional image pickup device, image processing apparatus, image processing method | |
| KR20160065464A (en) | Color filter array, image sensor having the same and infrared data acquisition method using the same | |
| CN108024054A (en) | Image processing method, device and equipment | |
| TW201931195A (en) | Capturing images under coded illumination patterns to reduce effects of ambient lighting conditions | |
| CN107707789A (en) | Monochromatic color mapping is carried out using monochromatic imager and color mapped sensors | |
| CN109788216A (en) | For the anti-interference method of TOF, device and TOF sensor chip | |
| US11696049B2 (en) | Systems and methods for dark current compensation in single photon avalanche diode imagery | |
| CN112823291A (en) | Time-of-flight RGB-IR image sensor | |
| US20240112404A1 (en) | Image modification techniques | |
| CN106851139A (en) | For the image element circuit and imaging system of global shutter correction | |
| TW202334899A (en) | Systems and methods for generating synthetic depth of field effects | |
| CN117355768A (en) | System and method for obtaining color images using single photon avalanche diodes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |