CN106534721B - The imaging device and its operation method of resolution prospect - Google Patents
The imaging device and its operation method of resolution prospect Download PDFInfo
- Publication number
- CN106534721B CN106534721B CN201510581322.4A CN201510581322A CN106534721B CN 106534721 B CN106534721 B CN 106534721B CN 201510581322 A CN201510581322 A CN 201510581322A CN 106534721 B CN106534721 B CN 106534721B
- Authority
- CN
- China
- Prior art keywords
- subframe
- pixel
- image
- picture frame
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
A kind of imaging device includes collector lens and imaging sensor.Described image sensor penetrates the light of the collector lens and includes picture element matrix, light shield layer, multiple lenticules and infrared light filter layer to sense.The picture element matrix includes multiple infrared ray pixels, multiple first pixels and multiple second pixels.The light shield layer masking is above the first area of the multiple first pixel and above the second area of the multiple second pixel, wherein the first area and the second area are along first direction formation mirror symmetry.The multiple lenticule is arranged above the picture element matrix.The infrared light filter layer is covered on above the multiple infrared ray pixel.
Description
Technical field
The present invention in relation to a kind of optical sensor, a kind of imaging device particularly in relation to distinguishable prospect and background and its
Operation method.
Background technique
Various electronic device is widely used to the default application of activation system by differentiating gesture, such as can be taken
Formula electronic device or wearable electronic device.The mode of resolution prospect and background known to a kind of is to use system source lighting operation
Object in range.When the system source polishing, imaging sensor obtains a bright image；And work as the system source not
When polishing, described image sensor obtains a dark image.By the difference image for calculating the bright image and the dark image
The interference of background can then be eliminated.
However, portable electronic devices or wearable electronic device are operated at often under strong light, such as under sunlight.
Being compared due to the brightness of system source with sunlight seems very faint, passes through the difference image of the bright dark image of known calculations
Mode and background interference can not be effectively eliminated.
In view of this, a kind of weak imaging device of environmental light intensity that is not only restricted to is urgently needed for current industry.
Summary of the invention
The present invention proposes the imaging device and its operation method of a kind of distinguishable prospect, and phase-detection of arranging in pairs or groups is strong to overcome
Limitation caused by environment light.
The present invention proposes a kind of imaging device of distinguishable prospect that different operating modes can be used under varying environment light
And its operation method.
The present invention provides a kind of imaging device, and it includes collector lens and imaging sensors.Described image sensor is used
The light of the collector lens is penetrated with sensing and includes picture element matrix, light shield layer, multiple lenticules and infrared light filter layer.Institute
Stating picture element matrix includes multiple infrared ray pixels, multiple first pixels and multiple second arranged along first direction and second direction
Pixel.The light shield layer covers above the first area of the multiple first pixel and the secondth area of the multiple second pixel
Above domain, wherein the first area and the second area are along first direction formation mirror symmetry.It is the multiple micro-
Mirror is arranged above the picture element matrix.The infrared light filter layer is covered on above the multiple infrared ray pixel.
The present invention also provides a kind of imaging device, and it includes infrared light supply, collector lens, imaging sensor and processing
Device.Described image sensor penetrates the light of the collector lens to sense, and bright figure is exported when the infrared light supply is lighted
Dark picture frame is exported as frame and when the infrared light supply extinguishes.Described image sensor includes picture element matrix, light shield layer, multiple
Lenticule and infrared light filter layer.The picture element matrix includes the multiple infrared ray pictures arranged along first direction and second direction
Plain, multiple first pixels and multiple second pixels.The light shield layer masking is above the first area of the multiple first pixel
And above the second area of the multiple second pixel, wherein the first area and the second area are along the first party
To formation mirror symmetry.The multiple lenticule is arranged above the picture element matrix.The infrared light filter layer is covered on institute
It states above multiple infrared ray pixels.The processor is to by the bright picture frame and the dark picture frame, correspondence is described more
The infrared view region of a infrared ray pixel forms infrared ray subframe, the first image-region of corresponding the multiple first pixel
The first subframe is formed, the second image-region of corresponding the multiple second pixel forms the second subframe, and calculates in first mode
The difference image of the infrared ray subframe of the infrared ray subframe and dark picture frame of the bright picture frame is to isolate
At least one foreground image, or at least one prospect is isolated according to first subframe and second subframe in second mode
Image.
The present invention also provides the operation method of a kind of imaging device, the imaging device includes infrared light supply, multiple infrared
Line pixel, multiple first pixels, multiple second pixels and multiple lenticules, the multiple first pixel and the multiple second picture
Element receives the incident light of out of phase by the first part of the multiple lenticule and second part respectively.The operation method
It comprises the steps of and exports bright picture frame when the infrared light supply is lighted with the imaging device and put out in the infrared light supply
Dark picture frame is exported when going out；The infrared view region of the multiple infrared ray pixel of correspondence is formed into infrared ray subframe, it will be right
The first image-region of the multiple first pixel is answered to form the first subframe, by the second image of the multiple second pixel of correspondence
Region forms the second subframe；In the first mode, the infrared ray subframe of the calculating bright picture frame and the dark picture frame
The infrared ray subframe difference image to isolate at least one foreground image；And in a second mode, according to described
First subframe and second subframe isolate at least one foreground image.
The present invention also provides a kind of imaging sensors, filter comprising picture element matrix, light shield layer, multiple lenticules and infrared light
Layer.The picture element matrix includes the multiple infrared ray pixels arranged along first direction and second direction, multiple first pixels and more
A second pixel.Light shield layer masking is above the first area of the multiple first pixel and the multiple second pixel
Above second area, wherein the first area and the second area are along first direction formation mirror symmetry.It is described more
A lenticule is arranged above the picture element matrix.The infrared light filter layer is covered on above the multiple infrared ray pixel.
In order to which above and other objects, features and advantages of the invention can be become apparent from, will hereafter be illustrated appended by cooperation, in detail
Carefully it is described as follows.In addition, identical component is indicated in explanation of the invention with identical symbol, first state clearly herein.
Detailed description of the invention
Fig. 1 is the block schematic diagram of the imaging device of the present invention one embodiment of explanation；
Fig. 2A -2B is the cross-sectional view of the imaging sensor for the imaging device that the present invention illustrates some embodiments；
Fig. 3 A-3D is the schematic diagram of the configuration for the light shield layer that the present invention illustrates some embodiments；
Fig. 4 is the schematic diagram of the operation method of the imaging device of the present invention one embodiment of explanation；
Fig. 5 is the schematic diagram of the operation method of the first mode of the imaging device of the present invention one embodiment of explanation；
Fig. 6 is the flow chart of the operation method of the imaging device of the present invention one embodiment of explanation.
Description of symbols
1 imaging device
10 collector lenses
11 imaging sensors
13 processors
130 storage elements
131 light source control modules
133 selecting modules
135 difference blocks
137 calculations of offset modules
139 application modules
15 infrared light supplies
9 objects
P1-P4、PinfPixel
F picture frame
Ls infrared light
Lrs reflection light
La environment light
Specific embodiment
It please refers to shown in Fig. 1 and 2A-2B, Fig. 1 is the block schematic diagram of the imaging device of the present invention one embodiment of explanation, figure
2A-2B is the cross-sectional view of the imaging sensor for the imaging device that the present invention illustrates some embodiments.Imaging device 1 includes that optically focused is saturating
Mirror 10, imaging sensor 11, processor 13 and infrared light supply 15.In some embodiments, the processor 13 for example can be with institute
It states imaging sensor 11 and the infrared light supply 15 is arranged in same chip.In some embodiments, the processor 13 can be
Processing unit outside described image sensor 11, to receive and process picture frame F acquired in described image sensor 11,
It is isolated at least with Selection utilization first mode (such as general mode) or second mode (such as strong optical mode) from background image
One foreground image (foreground image)；For example, when the imaging device 1 is applied to gesture identification, described at least one
A foreground image can be the hand of user or the object of user's gripping.The processor 13 may be, for example, microcontroller (MCU), center
The picture frame F person that processor (CPU), digital signal processor (DSP) etc. are exported to handle described image sensor 11.
The infrared light supply 15 may be, for example, light emitting diode or laser diode, to issue described in infrared light Ls illumination
Imaging device 1 can opereating specification；Wherein, it is described can opereating specification for example determined by component parameter.Described in entering when object 9
When can be in opereating specification, then the infrared light Ls be reflected to form reflection light Lrs towards described image sensor 11.Certain realities
It applies in example, the imaging device 1 may include that at least one optical module (not being painted) is sent out with homogenizing the infrared light supply 15
Light out.
The collector lens 10 can for example be located in the camera lens of image-taking device (such as camera), can for single lens or
Along the lens group of optical axis (optical axis) arrangement, have no specific limitation, and in order to simplify schema only show herein it is single
Mirror.The collector lens 10 is used as camera lens window (lens window), to obtain the reflection light Lrs from the object 9
Or environment light La, and guide the reflection light Lrs and environment light La to described image sensor 11.The collector lens 10 with
The distance of described image sensor 11 is desirably equal to the first focal length of the collector lens 10 (such as close to described image sensor
The focal length of 11 sides).It will be appreciated that the reflection light Lrs also can include component environment light when there are environment light La
Reflected light.
Described image sensor 11 (for example being indicated herein with pixel array) is based on default focal length sensing and penetrates the optically focused
The light (such as reflection light Lrs and environment light La) of lens 10 simultaneously exports picture frame F；For example, described image sensor 11 exists
The infrared light supply 15 exports bright picture frame and exports dark picture frame when the infrared light supply 15 extinguishes when lighting.Described image
Sensor 11 include picture element matrix 111 (such as illustrating by taking 9 × 9 picture element matrixs as an example), light shield layer 113, multiple lenticules 115 with
And infrared light filter layer 117 (referring to Fig. 2A and 2B)；Wherein, to cover the pixel after the light shield layer 113 is patterned
At least part for multiple pixels that matrix 111 is included, so that the non-shielded area of the multiple pixel is by the multiple micro-
The different piece of lens 115 receives the incident light of out of phase.The default focal length refer to the collector lens 10 with it is the multiple
The second focal length of the incident side positioned at the collector lens 10 is collectively formed in lenticule 115, is sometimes referred to simply as institute in the present invention
State the default focal length of collector lens 10 or described image sensor 11.
It has been found that when second focal length that object 9 is located at the collector lens 10 (is passed for example away from described image
The focal length of 11 side of sensor, i.e., the described default focal length) at reflection infrared light Ls or environment light La to the imaging device 1 when, it is described
Position of the object images in picture frame F that imaging sensor 11 is exported in the subframe of the pixel of relatively different masking kenels
Offset will not be generated, and when the object 9 is not located at second focal length of the collector lens 10, described image sensor
Position of the object images in the subframe of the pixel of relatively different masking kenels in the 11 picture frame F exported can be towards difference
Direction deviates (shift), after citing is specified in.Therefore, it is pre- to can determine whether the position offset is located at for the imaging device 1
If in range, to judge whether the object 9 is positioned at can foreground image in opereating specification.In other words, present invention explanation
Imaging device 1 have can opereating specification, positioned at it is described can imaging of the object on described image frame F in opereating specification it is then fixed
Justice is foreground image.
In one embodiment, the picture element matrix 111 includes multiple infrared ray pixel Pinf, multiple first pixel P1And it is multiple
Second pixel P2It is arranged along first direction (such as X-direction) and second direction (such as Y-direction).It should be noted that the present invention
In explanation, the multiple first pixel P1And the multiple second pixel P2Refer to that the region covered by the light shield layer 113 is different.
For example, in monochrome image sensor, the multiple first pixel P1And the multiple second pixel P2Pixel itself it is identical,
And the masking kenel (cover pattern) of light shield layer 113 on it is different (as shown in Figure 1).For example, being passed in color image
In sensor, the multiple first pixel P1And the multiple second pixel P2Red pixel (such as shape in pixel can be separately included
At red filter layer), green pixel (such as green color filter is formed in pixel), blue pixel (such as in pixel formed blue
Filter layer) or other colored pixels, and the multiple first pixel P1And the multiple second pixel P2On light shield layer 113
It is different to cover kenel.The multiple infrared ray pixel PinfTop is covered with infrared light filter layer 117 but not by the light shield layer
113 maskings.
The light shield layer 113 for example forms (such as the M1- in CMOS processing procedure using the metal layer as conductivity pathway
Wherein at least one layer of M10), the black shading layer that can also be additionally formed in addition for metal layer, or be the combination of the two, have no spy
Definite limitation, as long as incident light can be stopped.In the present embodiment, the masking of light shield layer 113 is in the multiple first pixel
P1First area (hatched example areas) above and the multiple second pixel P2Second area (hatched example areas) above.In Fig. 1,
The first area is located at the side along the first direction (such as X-direction), and the second area is located at along described
The opposite direction side of first direction, and the multiple first pixel P1The first area and the multiple second pixel P2Institute
It states second area and forms mirror symmetry along the first direction.In addition, the multiple first pixel P1With the first area
Non- shielded area (white space) and the multiple second pixel P in addition2It is (empty with the non-shielded area other than the second area
White region)；Wherein, the multiple first pixel P1Non- shielded area and the multiple second pixel P2Non- shielded area lead to respectively
The different piece for crossing the multiple lenticule 115 receives the incident light (as shown in Figure 2 A) of out of phase.
Example as shown in figure 1, the multiple first pixel P1First area be the multiple first pixel P1Upside and it is described
Multiple second pixel P2Second area be the multiple second pixel P2Downside.It should be noted that although Fig. 1 shows institute
State the 50% of the substantially single elemental area of first area and the second area, but its be only to illustrate and not to limit
Fixed present invention explanation.In other embodiments, the first area and the second area can be the 5%- of single elemental area
95%, have no specific limitation.
The multiple lenticule 115 is arranged above the picture element matrix 111, and respectively to positioned at a pixel.It is described
Light shield layer 113 and the infrared light filter layer 117 are then between the picture element matrix 111 and the multiple lenticule 115；Its
In, the vertical range of the light shield layer 113 and the infrared light filter layer 117 and the picture element matrix 111 can be equal or different,
Have no specific limitation.Whereby, the multiple first pixel P1And the multiple second pixel P2Pass through the multiple lenticule respectively
(opposite Fig. 1 is, for example, the lower half portion of lenticule 115, and opposite Fig. 2A is, for example, right the half of lenticule 115 for 115 first part
Part) and the second part (left side that the top half that Fig. 1 relatively is, for example, lenticule 115, relatively Fig. 2A are, for example, lenticule 115
Half part) receive out of phase incident light.It should be noted that although Fig. 2A shows the multiple first pixel P1And it is described
Multiple second pixel P2Non- shielded area generally relative to the multiple lenticule 115 half, but its be only to illustrate and simultaneously
It is non-to limit explanation of the present invention.It will be appreciated that light penetrates the multiple lenticule 115 and can reach non-shielded area
Part determined according to the shaded portions of the light shield layer 113.In present invention explanation, the first of the multiple lenticule 115
Part and second part can be configured to the 5%-95% of the multiple lenticule 115, have no specific limitation.
In Fig. 1, the picture element matrix 111 also multiple third pictures comprising being arranged along the second direction (such as Y-direction)
Plain P3And multiple 4th pixel P4.The light shield layer 113 is also covered in the multiple third pixel P3Third region (oblique line area
Domain) top and the multiple 4th pixel P4The fourth region (hatched example areas) above；Wherein, the third region be located at along
The side of the second direction (such as Y-direction), and the fourth region is located at the opposite direction side along the second direction.Example
As shown in figure 1, the third region is located at the multiple third pixel P3Left side, and the fourth region is located at the multiple the
Four pixel P4Right side, and the third region and the fourth region form mirror symmetry along the second direction.
In more detail, the light shield layer 113 is covered in the top of the picture element matrix 111, and includes the first masking kenel
Masking is in the multiple first pixel P1First area above；Second masking kenel masking is in the multiple second pixel P2's
Above second area；Third covers kenel masking in the multiple third pixel P3Third overlying regions；4th masking kenel
Masking is in the multiple 4th pixel P4The fourth region above；Wherein, the first area and the second area are along first
Direction forms mirror symmetry；The third region and the fourth region form mirror symmetry in a second direction.In one embodiment,
The first direction is perpendicular to the second direction.It should be noted that the multiple first pixel P1To the multiple 4th
Pixel P4Configuration be not limited to shown in Fig. 1, can preferably be uniformly distributed in everywhere in the pixel array 111.In addition, certain
In embodiment, the picture element matrix 111 only includes the multiple first pixel P1And the multiple second pixel P2It or only include institute
State multiple third pixel P3And the multiple 4th pixel P4, hold depending on different application.
In one embodiment, all first areas, the second area, the third area of the picture element matrix 111
Domain and the fourth region all have same area (as shown in Figure 1), the 5%-95% of for example, single elemental area.
It should be noted that although showing that the first area and the second area form mirror along first direction in Fig. 1
As symmetrical rectangle, and the third region and the fourth region form the rectangle of mirror symmetry in a second direction, but it is only
Explanation is not intended to limit the invention to illustrate.In other embodiments, the first area to the fourth region can be simultaneously
Non- is rectangle.Referring for example to shown in Fig. 3 A-3D, non-shielded area (white space) example of the first area to the fourth region
It is such as configured to increase along default direction (being shown as the adjacent direction of two pixels in figure) or be increased monotonically, and its shape is described
Preset direction is mirrored into symmetrically.Due in the multiple first pixel P1(or the multiple third pixel P3) and the multiple
Two pixel P2(or the multiple 4th pixel P4) when receiving incident beam, close to the multiple first pixel P1(or it is the multiple
Third pixel P3) and the multiple second pixel P2(or the multiple 4th pixel P4) the received incident light of center institute difference
Between phase difference be not obvious, and close to the multiple first pixel P1(or the multiple third pixel P3) and it is the multiple
Second pixel P2(or the multiple 4th pixel P4) the received incident light of edge institute difference between phase difference it is then larger, because
This, configures the non-shielded area greater than generic pixel center for the non-shielded area at generic pixel edge, can promote phase-detection
Accuracy.It should be noted that Fig. 3 A-3D is only to illustrate, it is not intended to limit the invention explanation.
The processor 13 is to correspond to the multiple infrared ray pixel P in bright picture frame and dark picture frameinfIt is red
Outside line image-region IinfForm infrared ray subframe Finf, corresponding the multiple first pixel P1The first image-region IP1Form the
One subframe FP1, corresponding the multiple second pixel P2The second image-region IP2Form the second subframe FP2, as shown in Figure 4.It is described
Processor 13 simultaneously calculates the infrared ray subframe F of bright picture frame in first modeinf_BWith the infrared ray subframe F of dark picture frameinf_D's
Difference image is to isolate at least one foreground image, or in second mode according to the first subframe FP1And second subframe
FP2Isolate at least one foreground image；Wherein, the first mode is, for example, general mode, and the second mode is for example, strong
Optical mode.In details of the words, the first subframe FP1By the multiple first pixel P1The luma data of output and formed, described
Two subframe FP2By the multiple second pixel P2The luma data of output and formed, the infrared ray subframe FinfBy the multiple
Infrared ray pixel PinfThe luma data of output and formed.
When the picture element matrix 111 includes four kinds of pixel configurations, the processor 13 is also to by bright picture frame and secretly
In picture frame, corresponding the multiple third pixel P3Third image-region IP3Form third subframe FP3, corresponding the multiple the
Four pixel P3The 4th image-region IP4Form the 4th subframe IP4, as shown in Figure 4.In details of the words, the third subframe FP3By institute
State multiple third pixel P3The luma data of output and formed, the 4th subframe FP4By the multiple 4th pixel P4Output
Luma data and formed.
Referring again to Fig. 1, the processor 13 include light source control module 131, selecting module 133, difference block 135,
Calculations of offset module 137 and application module 139；Wherein, the light source control module 131, selecting module 133, difference block
135, calculations of offset module 137 and application module 139 can for example be realized in a manner of software and/or hardware, have no specific limitation.
For convenience of description, the light source control module 131, selecting module 133, difference block 135, calculations of offset module 137 and application
Module 139 is shown as being separated from each other, and actually its running is completed by the processor 13.The processor 13 is preferably also
Required parameter when comprising storage element 130 to store running in advance, for example, it is opposite can opereating specification default bias amount model
It encloses.
The light source control module 131 is obtained to control the image of the opposite described image sensor 11 of the infrared light supply 15
It takes and lights and extinguish, so that the relatively described infrared light supply 15 of described image sensor 11 obtains when lighting and exports bright picture frame
And it obtains when the infrared light supply 15 extinguishes relatively and exports dark picture frame.
The selecting module 133 of the processor 13 is selected to the picture frame F exported according to described image sensor 11
Operation mode.In one embodiment, the selecting module 133 of the processor 13 is to select institute according to the average brightness of dark picture frame
State first mode or the second mode.For example, the selecting module 133 of the processor 13 only calculates the red of the dark picture frame
Outer light image region IinfAverage brightness or calculate the ensemble average brightness of the dark picture frame, and the average brightness
With luminance threshold (it is for example stored in the storage element 130).When the average brightness is less than the luminance threshold, expression
Environment light La is not very by force, then to enter the first mode, therefore the first mode can be described as general mode or dim light mode；
When the average brightness is greater than the luminance threshold, indicates that environment light La is very strong, then enter the second mode, therefore described the
Two modes can be described as strong optical mode.
As previously mentioned, the luminance difference of bright picture frame and dark picture frame is not obvious when environment light La is too strong.Therefore, separately
In one embodiment, the selecting module 133 of the processor 13 according to the mean luminance differences of bright picture frame and dark picture frame to select
Select the first mode or the second mode.When the mean luminance differences are greater than luminance difference threshold value, (it is for example stored in described
In storage element 130), indicate that environment light La is not very strong, therefore enter the first mode；When the mean luminance differences are less than
The luminance difference threshold value indicates that environment light La is very strong, therefore enters the second mode.
Referring to figure 5., the operation method of the first mode illustrated for the present invention.In first mode, the processor 13
The infrared ray subframe F of bright picture frame that exports described image sensor 11 of difference block 135inf_BAnd dark picture frame is infrared
Line subframe Finf_DCarry out calculus of differences.For example, it is assumed that the infrared ray subframe Finf_BInclude object images I9And background image Ia,
And the infrared ray subframe Finf_DIt only include background image Ia.As the infrared ray subframe Finf_BSubtract the infrared ray subframe
Finf_DAfterwards, difference image (Finf_B-Finf_D) only it is left the object images I9, use the interference for eliminating background image.
Fig. 1 and Fig. 4 is please referred to, then illustrates the function mode of second mode.In the present embodiment, such as with point object 9
It is illustrated for the incident side of the collector lens 10.Described image sensor 11 is based on default focal length and obtains and export figure
As frame F (it can be bright picture frame or dark picture frame) to the processor 13.Assuming that described in the selection of selecting module 133 entrance
Described image frame F is divided into the first subframe F by the calculations of offset module 137 of second mode, the processor 13P1And second subframe
FP2；Wherein, the first subframe FP1It is relevant to the multiple first pixel P1And the second subframe FP2It is relevant to the multiple
Second pixel P2.As previously mentioned, when the object 9 is located at the second focal length (the i.e. described pre- focal length) of the collector lens 10,
The image-region of the related object 9 is in the first subframe FP1And the second subframe FP2In be located substantially at opposite position
Without shifting.When the object 9 is not located at the second focal length of the collector lens 10, the figure of the related object 9
As region is in the first subframe FP1And the second subframe FP2In can shift without be located at opposite position.The place
The calculations of offset module 137 of device 13 is managed then to the first subframe F according to bright picture frame or dark picture frameP1And described second
Subframe FP2Isolate at least one foreground image.As previously mentioned, when environment light La is enough strong, bright picture frame or dark picture frame it is bright
It is unobvious to spend difference, therefore can be used to separate foreground image.
For example, Fig. 4 shows the first subframe FP1In the first image-region I91From offsetting up for middle line (such as dotted line)
Amount is S1, and the second subframe FP2In the second image-region I92It is S from the amount of offseting downward of middle line (such as dotted line)2.It is described
The calculations of offset module 137 of processor 13 is then to calculate S1And S2The first offset between the two, such as (S1-S2).It must say
Bright, the calculating of offset is not limited to using middle line as baseline, herein only for facilitating explanation by taking middle line as an example, is deviated
The calculating of amount can compare (block matching) or motion detection (motion detection) according to such as block also come real
It is existing, specific limitation is had no, as long as the first subframe F can be calculatedP1With the second subframe FP2In corresponding image-region
(such as I91、I92) between first offset；Wherein, whether two image-regions are corresponding for example can two figure
Brightness or shape as region determine.The calculations of offset module 137 of the processor 13 will be described in preset range
At least one corresponding image-region of first offset is identified as at least one foreground image, and by the figure outside the default scoping
As region recognition is background image.
When the picture element matrix 111 includes four kinds of pixel configurations, the calculations of offset module 137 of the processor 13 will also
Described image frame F is divided into third subframe FP3And the 4th subframe FP4；Wherein, the third subframe FP3It is relevant to the multiple
Three pixel P3, and the 4th subframe FP4It is relevant to the multiple 4th pixel P4.Under the second mode, the processor 13
Calculations of offset module 137 then to the third subframe F according to bright picture frame or dark picture frameP3And the 4th subframe
FP4Isolate at least one foreground image.
For example, Fig. 4 shows the third subframe FP3Middle third image-region I93From the offset to the right of middle line (such as dotted line)
Amount is S3, and the 4th subframe FP4In the 4th image-region I94It is S from the offset to the left of middle line (such as dotted line)4.It is described
The calculations of offset module 137 of processor 13 is then to calculate S3And S4The second offset between the two, such as (S3-S4), such as preceding institute
It states, the calculation of offset is not limited to subtraction.The calculations of offset module 137 of the processor 13 will be between default model
At least one corresponding image-region of second offset in enclosing is identified as at least one foreground image, and by the default
Image-region outside range is identified as background image.
As previously mentioned, the preset range is pre-stored in the storage element 130, for relative to can opereating specification offset
Amount.In other words, when first offset and/or second offset exceed the preset range, then it represents that the figure
As region belongs to background image.
It should be noted that although Fig. 4 shows the first image region I91Offset up S1, the two image-regions I92
Offset downward S2, the third image-region I93S is deviated to the right3, the four image-regions I94S is deviated to the left4, only to say
It is bright rather than to limit the present invention.The offset direction of the image-region of the relatively described object 9 is according to the object 9 from described second
Focal length is separate or close to the collector lens 10 and the multiple first pixel P1To the multiple 4th pixel P4Light shield layer
Depending on 113 shaded areas, it is not limited to person shown by Fig. 4.
The application module 139 is then according at least one foreground object output control signal Sc being identified, such as root
The control signal Sc is exported according to the variation of the direction of displacement of at least one foreground object, speed and quantity to control application
The running of program.
It should be noted that image-region is illustrated by taking round (opposite point object 9) as an example in the present embodiment, but this hair
Bright to be not limited thereto, image-region may be, for example, edge (edge) in described image frame F etc. and clearly can show to deviate
Amount person has no specific limitation.
In addition, judging accuracy to increase, the processor 13 also corrects first son using shadowing method (shading)
Frame FP1And the second subframe FP2Brightness be it is consistent, so, it is possible correctly to judge the first subframe FP1And second son
Frame FP2In corresponding image-region (such as the identical image-region of brightness), such as I91、I92.When the picture element matrix 111 wraps
When containing four kinds of pixel configurations, the processor 13 also corrects the third subframe F using shadowing methodP3And the 4th subframe FP4
Brightness be it is consistent, so, it is possible correctly to judge the third subframe FP3And the 4th subframe FP4In corresponding image district
Domain (such as the identical image-region of brightness), such as I93、I94。
Referring to shown in Fig. 1,2A-2B and 4-6, Fig. 6 is the running side for the imaging device that the present invention illustrates embodiment
Method, such as the imaging device 1 suitable for Fig. 1.As previously mentioned, imaging device 1 includes infrared light supply 15, multiple infrared ray pixels
Pinf, multiple first pixel P1, multiple second pixel P2And multiple lenticules 115.The multiple first pixel P1And it is the multiple
Second pixel P2The incident light of out of phase is received by the first part of the multiple lenticule 115 and second part respectively,
Such as Fig. 1 shows the upside that the first part is located at the downside of pixel and the second part is located at pixel, but its position and
It is not limited to shown in Fig. 1 with the ratio of pixel.
The operation method of the present embodiment is comprised the steps of exports bright picture frame with imaging device when infrared light supply is lighted
And dark picture frame (step S61) is exported when the infrared light supply extinguishes；By the infrared ray of the multiple infrared ray pixel of correspondence
Second image district of image-region, the first image-region of corresponding the multiple first pixel and corresponding the multiple second pixel
Domain is respectively formed infrared ray subframe, the first subframe and the second subframe (step S62)；Select first mode or second mode (step
S63)；In the first mode, the described red of the infrared ray subframe of the bright picture frame and the dark picture frame is calculated
The difference image of outside line subframe is to isolate at least one foreground image (step S64)；And in the second mode, according to
First subframe and described second isolate at least one foreground image (step S65).
Step S61: the light source control module 131 of the processor 13 controls the opposite described image of the infrared light supply 15 and passes
The image of sensor 11 obtains and lights and extinguish, to export bright picture frame when the infrared light supply 15 is lighted and described infrared
Light source 15 exports dark picture frame when extinguishing.In more detail, the frame per second (frame rate) of described image sensor 11 is at least institute
Twice for stating the ignition frequency of infrared light supply 15, such as twice, four times ....
Step S62: described image sensor 11 is by acquired every picture frame F (such as bright picture frame and dark picture frame)
Output to the processor 11 is post-processed.For example, the processor 11 is by every picture frame F, correspondence is the multiple red
Outside line pixel PinfInfrared view region IinfForm infrared ray subframe Finf, by the multiple first pixel P of correspondence1?
One image-region IP1Form the first subframe FP1And it will corresponding the multiple second pixel P2The second image-region IP2Form second
Subframe FP2, as shown in Figure 4.Positional relationship of the mode of subframe for example according to script positioned at described image frame F is formed to reformulate
The infrared ray subframe Finf, the first subframe FP1And the second subframe FP2。
In some embodiments, the imaging device 1 also includes multiple third pixel P3And multiple 4th pixel P4, described more
A third pixel P3And the multiple 4th pixel P4Pass through the Part III and Part IV of the multiple lenticule 115 respectively
The incident light of out of phase is received, such as Fig. 1 shows that the Part III is located at the right side of pixel and the Part IV is located at
The left side of pixel, but its position and be not limited to shown in Fig. 1 with the ratio of pixel.In embodiment shown in FIG. 1, described first
Point and the second part be the multiple lenticule 115 along the two opposite sides of first axis (such as X-axis) the third portion
Divide and the Part IV is the multiple lenticule 115 along the two opposite sides of second axial (such as Y-axis).
When the picture element matrix 111 includes four kinds of pixel configurations, in step S62, the processor 11 is by every image
In frame F, corresponding the multiple third pixel P3Third image-region IP3Form third subframe FP3, by correspondence the multiple 4th
Pixel P4The 4th image-region IP4Form the 4th subframe FP4.In some embodiments, when first mode, 13 difference of processor
Module 135 only generates infrared ray subframe FinfWithout generating the first subframe FP1To the 4th subframe FP4；When second mode, the processing
13 calculations of offset module 137 of device only generates the first subframe FP1To the second subframe FP4Without generating infrared ray subframe Finf。
Step S63: the selecting module 133 of the processor 13 judges environment light according to the average brightness of described image frame F
It is whether too strong.In one embodiment, the selecting module 133 of the processor 13 selects the first mould according to the average brightness of dark picture frame
Formula or second mode.In another embodiment, the selecting module 133 of the processor 13 is flat according to bright picture frame and dark picture frame
Equal luminance difference selection first mode or second mode.In present invention explanation, the first mode is, for example, general mode or dim light
Mode, the second mode are, for example, strong optical mode.Whereby, the processor 13 can be suitble to according to the strong and weak selection of environment light
Algorithm isolates an at least foreground image.
Step S64: when environment light is not very strong, then enter first mode.At this point, the difference block of the processor 13
135 can directly calculate the infrared ray subframe F of bright picture frameinf_BWith the infrared ray subframe F of dark picture frameinf_DDifference image with
Isolate at least one foreground object image I9, as shown in Figure 5.In more detail, infrared ray pixel PinfThe grayscale value of output is general
It is used to calculate without being calculated in strong optical mode in logical mode.
Step S65: when environment light is very strong, then enter second mode.At this point, the offset of the processor 13 calculates mould
Block 137 is according to the first subframe FP1And the second subframe FP2Calculate at least one the first offset (such as S shown in Fig. 41With
S2Offset), and an at least image-region corresponding to first offset in preset range is identified as at least
One foreground image.As previously mentioned, the preset range be correspondence can opereating specification default bias amount range.
When the picture element matrix 111 includes four kinds of pixel configurations, in step S65, the offset meter of the processor 13
Module 137 is calculated further accordance with the third subframe FP3And the 4th subframe FP4Calculating at least the second offset (such as S shown in the 4th3
With S4Offset), and at least one image-region corresponding to second offset in preset range is identified as
At least one foreground image.It should be noted that before being identified according to first offset and second offset
Scape image might not be identical, may only part it is identical or entirely different, end view described image frame F in characteristics of image and
It is fixed.
Finally, the application module 139 of the processor 13 can be according to the foreground image output control signal Sc identified
To carry out different application, such as progress gesture identification etc..
In addition, in order to increase identification accuracy, the processor 13 can correct the first subframe F using shadowing methodP1And
The second subframe FP2Brightness be consistent and correct the third subframe FP3And the 4th subframe FP4Brightness be it is consistent,
With when calculating offset, in the first subframe FP1With the second subframe FP2In be easier to find corresponding image district
Domain, and it is described in third subframe FP3With the 4th subframe FP4In be easier to find corresponding image-region.
In summary, it is known that gesture recognition system is when environment light is stronger, it may appear that the situation that can not correctly operate.Cause
This, the present invention illustrates to propose a kind of imaging device (Fig. 1) and its operation method (Fig. 6), using making under varying environment luminous intensity
Foreground image is isolated with algorithms of different, to overcome the problems, such as known gesture recognition system.
Although the present invention is disclosed by previous examples, it is not intended to limit the invention, skill belonging to any present invention
With the technical staff of usual knowledge in art field, without departing from the spirit and scope of the present invention, when various changes can be made
With modification.Therefore protection scope of the present invention is subject to the range defined depending on appended claims.
Claims (17)
1. a kind of imaging device, which includes:
Infrared light supply；
Collector lens；
Imaging sensor exports bright image when the infrared light supply is lighted to sense the light for penetrating the collector lens
Frame simultaneously exports dark picture frame when the infrared light supply extinguishes, and described image sensor includes:
Picture element matrix includes multiple infrared ray pixels, multiple first pixels and multiple along first direction and second direction arrangement
Second pixel；
Light shield layer covers above the first area of the multiple first pixel and on the second area of the multiple second pixel
Side, wherein the first area and the second area are along first direction formation mirror symmetry；
Multiple lenticules are arranged above the picture element matrix；And
Infrared light filter layer is covered on above the multiple infrared ray pixel；And
Processor, to correspond to the infrared ray of the multiple infrared ray pixel in the bright picture frame and the dark picture frame
Image-region forms infrared ray subframe, and the first image-region of corresponding the multiple first pixel forms the first subframe, corresponding institute
The second image-region for stating multiple second pixels forms the second subframe, and calculates the described red of the bright picture frame in first mode
The difference image of the infrared ray subframe of outside line subframe and the dark picture frame to isolate at least one foreground image, or
Second mode isolates at least one foreground image according to first subframe and second subframe.
2. imaging device according to claim 1, wherein the processor is to according to the average bright of the dark picture frame
Degree selects the first mode or the second mode.
3. imaging device according to claim 1, wherein the processor is to according to the bright picture frame and described dark
The mean luminance differences of picture frame select the first mode or the second mode.
4. imaging device according to claim 1, wherein the processor is to according to the bright picture frame or described dark
First subframe and second subframe of picture frame isolate at least one described foreground image.
5. imaging device according to claim 4, wherein the processor is according to first subframe and second son
Frame calculates at least one first offset, and image-region corresponding to first offset in preset range is known
It Wei not at least one described foreground image.
6. imaging device according to claim 1, wherein the picture element matrix also includes multiple third pixels and multiple
Four pixels, the light shield layer also cover the multiple third pixel third overlying regions and the multiple 4th pixel
Four overlying regions, wherein the third region and the fourth region are along second direction formation mirror symmetry.
7. imaging device according to claim 6, wherein the first area, the second area, the third region
And the fourth region is the 5%-95% of single elemental area.
8. imaging device according to claim 6, wherein the processor is also to by the bright picture frame and described dark
In picture frame, the third image-region of corresponding the multiple third pixel forms third subframe, corresponding the multiple 4th pixel
The 4th image-region form the 4th subframe, and in the second mode according to the institute of the bright picture frame or the dark picture frame
It states third subframe and the 4th subframe isolates at least one described foreground image.
9. imaging device according to claim 8, wherein the processor is according to the third subframe and the 4th son
Frame calculates at least one second offset, and image-region corresponding to second offset in preset range is known
It Wei not at least one described foreground image.
10. imaging device according to claim 8, wherein the processor also utilizes shadowing method to correct first subframe
And the brightness of second subframe be it is consistent, it is consistent for correcting the brightness of the third subframe and the 4th subframe.
11. the operation method of a kind of imaging device, the imaging device includes infrared light supply, multiple infrared ray pixels, Duo Ge
One pixel, multiple second pixels and multiple lenticules, the multiple first pixel and the multiple second pixel pass through institute respectively
The first part and second part that state multiple lenticules receive the incident light of out of phase, and the operation method includes:
Bright picture frame and the output when the infrared light supply extinguishes are exported when the infrared light supply is lighted with the imaging device
Dark picture frame；
The infrared view region of the multiple infrared ray pixel of correspondence is formed into infrared ray subframe, by correspondence the multiple first
First image-region of pixel forms the first subframe, and the second image-region of the multiple second pixel of correspondence is formed the second son
Frame；
In the first mode, the infrared ray subframe of the bright picture frame and the infrared ray of the dark picture frame are calculated
The difference image of frame is to isolate at least one foreground image；And
In a second mode, at least one foreground image is isolated according to first subframe and second subframe.
12. operation method according to claim 11, which also includes:
The first mode or the second mode are selected according to the average brightness of the dark picture frame.
13. operation method according to claim 11, which also includes:
The first mode or the second mode are selected according to the mean luminance differences of the bright picture frame and the dark picture frame.
14. operation method according to claim 11, which also includes:
At least one first offset is calculated according to first subframe and second subframe, and will be in preset range
Image-region corresponding to first offset is identified as at least one described foreground image.
15. operation method according to claim 11, wherein the imaging device also includes multiple third pixels and multiple
4th pixel, the multiple third pixel and the multiple 4th pixel pass through respectively the multiple lenticule Part III and
Part IV receives the incident light of out of phase, and the first part and the second part are the multiple lenticule along first
Axial two opposite sides and the Part III and the Part IV are that the multiple lenticule is opposite along second axial two
Side.
16. operation method according to claim 15, which also includes:
The third image-region of the multiple third pixel of correspondence is formed into third subframe；
4th image-region of the multiple 4th pixel of correspondence is formed into the 4th subframe；And
At least one second offset is calculated according to the third subframe and the 4th subframe, and will be in preset range
Image-region corresponding to second offset is identified as at least one described foreground image.
17. operation method according to claim 16, which also includes:
Be using the brightness that shadowing method corrects first subframe and second subframe it is consistent, correct the third subframe and institute
It is consistent for stating the brightness of the 4th subframe.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910706667.6A CN110598685B (en) | 2015-09-14 | 2015-09-14 | Imaging device for resolving foreground |
| CN201510581322.4A CN106534721B (en) | 2015-09-14 | 2015-09-14 | The imaging device and its operation method of resolution prospect |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510581322.4A CN106534721B (en) | 2015-09-14 | 2015-09-14 | The imaging device and its operation method of resolution prospect |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910706667.6A Division CN110598685B (en) | 2015-09-14 | 2015-09-14 | Imaging device for resolving foreground |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106534721A CN106534721A (en) | 2017-03-22 |
| CN106534721B true CN106534721B (en) | 2019-08-27 |
Family
ID=58348992
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510581322.4A Active CN106534721B (en) | 2015-09-14 | 2015-09-14 | The imaging device and its operation method of resolution prospect |
| CN201910706667.6A Active CN110598685B (en) | 2015-09-14 | 2015-09-14 | Imaging device for resolving foreground |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910706667.6A Active CN110598685B (en) | 2015-09-14 | 2015-09-14 | Imaging device for resolving foreground |
Country Status (1)
| Country | Link |
|---|---|
| CN (2) | CN106534721B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008058976A (en) * | 2007-09-10 | 2008-03-13 | Semiconductor Energy Lab Co Ltd | Semiconductor device |
| CN101689557A (en) * | 2008-07-11 | 2010-03-31 | 松下电器产业株式会社 | Solid-state imaging device |
| CN102289131A (en) * | 2010-06-17 | 2011-12-21 | 奥林巴斯株式会社 | Image pickup apparatus |
| CN104221149A (en) * | 2012-03-28 | 2014-12-17 | 富士胶片株式会社 | Imaging element and imaging device |
| CN104318199A (en) * | 2014-06-23 | 2015-01-28 | 上海箩箕技术有限公司 | Compound optical sensor and manufacture method and use method thereof |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101779154B (en) * | 2007-08-13 | 2012-03-07 | 松下电器产业株式会社 | Imaging device and camera |
| JP5366619B2 (en) * | 2008-08-12 | 2013-12-11 | キヤノン株式会社 | Image processing apparatus, imaging apparatus, image processing method, and image processing program |
| KR101608903B1 (en) * | 2009-11-16 | 2016-04-20 | 삼성전자주식회사 | Infrared image sensor |
| JP5585208B2 (en) * | 2010-05-20 | 2014-09-10 | ソニー株式会社 | Solid-state imaging device and electronic apparatus |
| US8408821B2 (en) * | 2010-10-12 | 2013-04-02 | Omnivision Technologies, Inc. | Visible and infrared dual mode imaging system |
| JP6119193B2 (en) * | 2012-02-24 | 2017-04-26 | 株式会社リコー | Distance measuring device and distance measuring method |
| TWI595219B (en) * | 2012-05-08 | 2017-08-11 | Sony Corp | Infrared conversion element, imaging device and imaging method |
| CN104662589B (en) * | 2012-08-21 | 2017-08-04 | 派力肯影像公司 | For the parallax detection in the image using array camera seizure and the system and method for correction |
-
2015
- 2015-09-14 CN CN201510581322.4A patent/CN106534721B/en active Active
- 2015-09-14 CN CN201910706667.6A patent/CN110598685B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008058976A (en) * | 2007-09-10 | 2008-03-13 | Semiconductor Energy Lab Co Ltd | Semiconductor device |
| CN101689557A (en) * | 2008-07-11 | 2010-03-31 | 松下电器产业株式会社 | Solid-state imaging device |
| CN102289131A (en) * | 2010-06-17 | 2011-12-21 | 奥林巴斯株式会社 | Image pickup apparatus |
| CN104221149A (en) * | 2012-03-28 | 2014-12-17 | 富士胶片株式会社 | Imaging element and imaging device |
| CN104318199A (en) * | 2014-06-23 | 2015-01-28 | 上海箩箕技术有限公司 | Compound optical sensor and manufacture method and use method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110598685B (en) | 2023-06-30 |
| CN106534721A (en) | 2017-03-22 |
| CN110598685A (en) | 2019-12-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11418729B2 (en) | Image sensor having symmetrical opaque layer | |
| JP6110862B2 (en) | Object distance determination from images | |
| US20200093368A1 (en) | Method of identifying iris | |
| CN103155539B (en) | Distinguish the method for scene background and prospect and replace the method for background in scene image | |
| KR20200123483A (en) | 3D depth detection and adjustment of camera exposure for 2D imaging | |
| JP6286481B2 (en) | Writing system and method for object enhancement | |
| US20160134851A1 (en) | Method and System for Projector Calibration | |
| TW201337660A (en) | High resolution and high sensitivity cursor maneuvering device using multiple color light sources | |
| JP2009043139A (en) | Position detecting device | |
| Koppal et al. | Exploiting dlp illumination dithering for reconstruction and photography of high-speed scenes | |
| CN106534721B (en) | The imaging device and its operation method of resolution prospect | |
| JP7062751B2 (en) | Projection control device, projection device, projection control method, and projection control program | |
| JP3818028B2 (en) | 3D image capturing apparatus and 3D image capturing method | |
| US9906705B2 (en) | Image pickup apparatus | |
| JP2020515100A (en) | Vignetting compensation | |
| JP6512806B2 (en) | Imaging device | |
| CN117396734A (en) | Method, electronic device, computer-readable storage medium and terminal device for improving quality of image acquired by image sensor having image plane phase difference pixels | |
| JP2019202580A (en) | Light distribution controller, light projection system, and light distribution control method | |
| JP2017097363A (en) | Display device, projection device, display method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |