Summary of the invention
Because the problem of above-mentioned existing skill, object of the present invention is providing a kind of image-taking device, image depth generation device and method thereof exactly, to improve the resolution of image depth and to improve accuracy simultaneously.
Because the problem of above-mentioned existing skill, object of the present invention is providing a kind of image-taking device, image depth generation device and method thereof exactly, the easily problem of mistake of image depth while having monochromatic block or repeat patterns in solution image.
Because the problem of above-mentioned existing skill, object of the present invention is providing a kind of image-taking device, image depth generation device and method thereof exactly, improves with the photoflash lamp of improveing the accuracy and the convenience that on handheld apparatus, calculate image depth.
According to object of the present invention, a kind of image depth generation device is proposed, it comprises a pattern light projection module, the first photographing module, the second photographing module, perspective view pattern depth calculation module, image-type depth calculation module and degree of depth decision module.Pattern light projection module system is to the default pattern of external environment condition projection.The first photographing module captures respectively the first image of external environment condition, and has the 3rd image of the acquiescence pattern being out of shape.The second photographing module captures respectively the second image of external environment condition, and has the 4th image of the acquiescence pattern being out of shape.Perspective view pattern depth calculation module can according to acquiescence pattern, the deformation extent in the 3rd image and the 4th image calculates a plurality of the first image depth, a plurality of the first confidence level, a plurality of the second image depth and a plurality of the second confidence level respectively.Image-type depth calculation module can calculate a plurality of the 3rd image depth and a plurality of the 3rd confidence level according to the first image and the second eiconometer.Degree of depth decision module can, according to a plurality of the first confidence levels, a plurality of the second confidence level and a plurality of the 3rd confidence level, be combined into a plurality of the 4th image depth from a plurality of the first image depth, a plurality of the second image depth and a plurality of the 3rd image depth.
Preferably, the first image and the 3rd image system are made up of a plurality of pixel, and the quantity system of a plurality of the first image depth, a plurality of the first confidence levels equals the quantity of a plurality of pixels.
Preferably, when the capture resolution of the second photographing module is during lower than the first photographing module, image depth generation device more comprises video conversion module, and its image that the second photographing module can be captured carries out a conversion of resolution, and the pixel quantity that the second image is comprised equals the first image.
Preferably, degree of depth decision module more can be according to a plurality of the first confidence levels, a plurality of the second confidence level and a plurality of the 3rd confidence level, from a plurality of the first image depth, a plurality of the second image depth and a plurality of the 3rd image depth, eliminate image depth with a low credibility respectively, and degree of depth decision module can be combined into a plurality of the 4th image depth according to a plurality of the first image depth that are not eliminated, a plurality of the second image depth and a plurality of the 3rd image depth.
Preferably, pattern light projection module comprises photoflash lamp and light transmission pattern shielding, and light transmission pattern shielding has default pattern and is arranged on the light direction of photoflash lamp, and default pattern can comprise any lines, colored block or its combination.
Based on above-mentioned purpose, the present invention provides a kind of image depth production method again, and it comprises the following step.First, utilize pattern light projection module to the default pattern of external environment condition projection.Then, utilize the first photographing module to capture respectively the first image of external environment condition, and there is the 3rd image of the acquiescence pattern being out of shape.Utilize the second photographing module to capture respectively the second image of external environment condition, and there is the 4th image of the acquiescence pattern being out of shape.Deformation extent according to acquiescence pattern in the 3rd image calculates a plurality of the first image depth and a plurality of the first confidence level.Then, according to acquiescence pattern, the deformation extent in the 4th image calculates a plurality of the second image depth and a plurality of the second confidence level.The first image and the second image are carried out to an image-type depth calculation processing, to obtain a plurality of the 3rd image depth and a plurality of the 3rd confidence level.Finally, according to a plurality of the first confidence levels, a plurality of the second confidence level and a plurality of the 3rd confidence level, from a plurality of the first image depth, a plurality of the second image depth and a plurality of the 3rd image depth, be combined into a plurality of the 4th image depth.
Preferably, default pattern can comprise any lines, colored block or its combination.
Preferably, the first image and the 3rd image system are made up of a plurality of pixel, and the quantity system of a plurality of the first image depth, a plurality of the first confidence levels equals the quantity of a plurality of pixels.
Preferably, when the capture resolution of the second photographing module is during lower than the first photographing module, image depth production method more comprised the image that the second photographing module is captured and carries out a conversion of resolution before the step of calculating image depth, and the pixel quantity that the second image is comprised equals the first image.
Based on above-mentioned purpose, the present invention provides a kind of image-taking device again, and it comprises a pattern light projection module, one first photographing module, one second photographing module, a perspective view pattern depth calculation module, an image-type depth calculation module, a degree of depth decision module and a storage module.Pattern light projection module is in order to preset pattern to external environment condition projection.The first photographing module captures respectively the first image of external environment condition, and has the 3rd image of the acquiescence pattern being out of shape.The second photographing module captures respectively the second image of external environment condition, and has the 4th image of the acquiescence pattern being out of shape.Perspective view pattern depth calculation module can according to acquiescence pattern, the deformation extent in the 3rd image and the 4th image calculates a plurality of the first image depth, a plurality of the first confidence level, a plurality of the second image depth and a plurality of the second confidence level respectively.Image-type depth calculation module can calculate a plurality of the 3rd image depth and a plurality of the 3rd confidence level according to the first image and the second eiconometer.Degree of depth decision module can, according to a plurality of the first confidence levels, a plurality of the second confidence level and a plurality of the 3rd confidence level, be combined into a plurality of the 4th image depth from a plurality of the first image depth, a plurality of the second image depth and a plurality of the 3rd image depth.Storage module can store the first image as image output, and stores the depth information of a plurality of the 4th image depth as image output.
Preferably, pattern light projection module comprises photoflash lamp and light transmission pattern shielding, and light transmission pattern shielding system is arranged on the light direction of photoflash lamp.
Embodiment
Please refer to Fig. 1, is the calcspar for image depth generation device of the present invention.In figure, image depth generation device 1 comprises a pattern light projection module 10, one first photographing module 20, one second photographing module 30, a perspective view pattern depth calculation module 40, an image-type depth calculation module 50 and a degree of depth decision module 60.
Pattern light projection module 10 is the structured light to external environment condition projection acquiescence pattern 11.In enforcement, wherein pattern light projection module 10 can comprise a photoflash lamp and light transmission pattern shielding, and light transmission pattern shielding has default pattern 11 and is arranged on the light direction of photoflash lamp 12.Default pattern 11 can comprise any lines, colored block or its combination.
In addition, if photoflash lamp has two light source portion, light transmission pattern shielding can only hide a wherein light source portion, projects the structured light of acquiescence pattern 11 or the light without acquiescence pattern 11 just so alternative.
After pattern light projection module 10 projective structure light, the first photographing module 20 and the second photographing module 30 capture respectively the 3rd image 22 and the 4th image 32 with the default pattern 11 being out of shape.Because each object of external environment condition can not be positioned at same plane, or the profile of object itself is not just at grade, so the structured light of projection acquiescence pattern 11 just can produce distortion on these a little objects, and be ingested the 3rd image 22 and the 4th image 32.
After capturing the 3rd image 22 and the 4th image 32, the first photographing module 20 and the second photographing module 30 capture respectively the first image 21 and the second image 31 again.In the first image 21 and the second image 31, substantially do not comprise the default pattern 11 being out of shape, but not as restriction, if comprise the default pattern 11 being out of shape in the first image 21 and the second image 31, also do not affect follow-up processing, still belong in the present invention's interest field.
According to acquiescence pattern 11, the deformation extent in the 3rd image 22 and the 4th image 32 calculates a plurality of the first image depth 41, a plurality of the first confidence level 42, a plurality of the second image depth 43 and a plurality of the second confidence level 44 to perspective view pattern depth calculation module 40 respectively.
Image-type depth calculation module 50 can calculate a plurality of the 3rd image depth 51 and a plurality of the 3rd confidence level 52 according to the first image 21 and the second image 31.
In this embodiment, the first image 21 and the 3rd image 22 are made up of a plurality of pixel 211, as shown in Figure 2, so preferably the quantity of the first image depth 41 and the first confidence level 42 equals the quantity of a plurality of pixels 211 of the first image 21, be that each pixel 211 has the first corresponding image depth 41 and the first confidence level 42, but this is non-for limiting.For example, a plurality of pixels 211 separable groups of the first image 2, the pixel group 212 that for example four pixels 211 in Fig. 2 form, and each pixel group 212 just has the first corresponding image depth 41 and the first confidence level 42.So can reduce required operand, but the resolution of the first image depth 41 and the first confidence level 42 also can be lowered into the first image 21 resolution 1/4th.
The amount of pixels (resolution) of the first photographing module 20 can be more than or equal to the amount of pixels of the second photographing module 30.In enforcement, the second photographing module 30 is likely auxiliary with video camera, so based on cost consideration, its capture resolution may be lower than the first photographing module 20 as main video camera as one.
In the case, image depth generation device 1 more can comprise a video conversion module, the image that video conversion module can be captured the second photographing module 30 carries out a conversion of resolution, for example interpolation processing, the pixel quantity that the second image 31 is comprised can equal the first image 21, is beneficial to follow-up calculating.And the correlation technique of conversion of resolution for this reason the operator in field know, therefore do not repeat them here.
Degree of depth decision module 60 can, according to a plurality of the first confidence levels 42, a plurality of the second confidence level 44 and a plurality of the 3rd confidence level 52, be combined into a plurality of the 4th image depth 61 from a plurality of the first image depth 41, a plurality of the second image depth 43 and a plurality of the 3rd image depth 51.
For example, degree of depth decision module 60 can be first according to a plurality of the first confidence levels 42, a plurality of the second confidence level 44 and a plurality of the 3rd confidence level 52, from a plurality of the first image depth 41, a plurality of the second image depth 43 and a plurality of the 3rd image depth 51, eliminate image depth with a low credibility respectively, and degree of depth decision module 60 can be combined into a plurality of the 4th image depth 61 according to a plurality of the first image depth 41 that are not eliminated, a plurality of the second image depth 43 and a plurality of the 3rd image depth 51.
By this, if while having a region to occur monochromatic block or repetitions patterns in the first image 21 and the second image 31, the 3rd confidence level 52 of the 3rd image depth 51 that image-type depth calculation module 50 calculates for this region can be on the low side, if use this 3rd corresponding image depth 51 easily to occur wrong situation; On the other hand, perspective view pattern depth calculation module 40 does not allow to be subject to the impact of monochromatic area piece or repetitions patterns, the second confidence level 44 of therefore the first confidence level 42 of its first image depth 41 calculating for this region, or the second image depth 43 can be higher.
After so degree of depth decision module 60 is eliminated the 3rd image depth 51 in this region, can decide the 4th image depth 61 for this region with reference to first confidence level 42 in this region or the second confidence level 44, for example, can be directly using corresponding the first image depth 41 of the greater or the second image depth 43 between the first confidence level 42 or the second confidence level 44 as the 4th image depth 61 in this region.
Or if the first confidence level 42 or the second confidence level 44 are all high and be close, the mean value of available corresponding the first image depth 41 and the second image depth 43 is as the 4th image depth 61 in this region.
On the other hand, in the time that external ambient light is strong, when projective structure light is not obvious in the 3rd image 22 and the 4th image 32, can cause the first confidence level 42 and the second confidence level 44 on the low side, now degree of depth decision module 60 just can be mainly using a plurality of the 3rd image depth 51 as a plurality of the 4th image depth 61.
So, the present invention's image depth generation device just can be taken into account resolution and the fineness of image depth.
Refer to Fig. 3, it is the calcspar of image-taking device of the present invention.In figure, image-taking device 9 comprises photoflash lamp 12, light transmission pattern shielding 13, main photographing module 23, auxiliary photographing module 33, perspective view pattern depth calculation module 40, image-type depth calculation module 50, degree of depth decision module 60, storage module 70 and video conversion module 80.Wherein the capture resolution of main photographing module 23 system is greater than auxiliary photographing module 33, and main photographing module 23 is the diverse location being arranged on image-taking device 9 with auxiliary photographing module 33.
Photoflash lamp 12 comprises can distinguish luminous the first light source portion 121 and secondary light source portion 122, and light transmission pattern shielding 13 is to be arranged on the light direction of secondary light source portion 122, so in the time that secondary light source portion 122 is luminous, image-taking device 9 just can be to the structured light of an external environment condition projection acquiescence pattern 11.Therefore,, when image-taking device 9 is taken general photo, can only drive the first light source portion 121 luminous; In the time that needs produce image depth, can first drive secondary light source portion 122 luminous, wait until when main photographing module 23 will capture respectively the first image 21 and the second image 31 with auxiliary photographing module 33, then drive the first light source portion 121 luminous.
Main photographing module 23 captures respectively the first image 21 of external environment condition, and has the 3rd image 22 of the default pattern 11 being out of shape.Auxiliary photographing module 33 captures respectively the second image 31 of external environment condition, and has the 4th image 32 of the default pattern 11 being out of shape.
Video conversion module 80 can utilize the mode of interpolation that the resolution of the image of assisting photographing module 33 to capture is improved, and the consistency of increase and the image that mainly photographing module 23 is captured, be beneficial to the calculating of follow-up perspective view pattern depth calculation module 40 and image-type depth calculation module 50.
The operation principles of perspective view pattern depth calculation module 40, image-type depth calculation module 50 and degree of depth decision module 60 is all identical with aforementioned content, therefore do not repeat them here.
Finally, storage module 70 can store the first image 21 as image output 71, and stores the depth information of a plurality of the 4th image depth 61 as image output 71, as follow-up image application.
Refer to Fig. 4, it is according to the flow chart of image depth production method of the present invention.In figure, the image depth generation device 1 of collocation the 1st figure of image depth production method system is to describe, and the method comprises the following step.At step S10, utilize the structured light of a pattern light projection module 10 to an external environment condition projection one acquiescence pattern 11.
At step S20, utilize one first photographing module 20 to capture respectively one of external environment condition the first image 21, and there is one the 3rd image 22 of the default pattern 11 being out of shape.
At step S21, utilize one second photographing module 30 to capture respectively one of external environment condition the second image 31, and there is one the 4th image 32 of the default pattern 11 being out of shape.
At step S30, according to default pattern 11, the deformation extent in the 3rd image 22 calculates a plurality of the first image depth 41 and a plurality of the first confidence level 42 respectively, and the deformation extent in the 4th image 32 calculates a plurality of the second image depth 43 and a plurality of the second confidence level 44 according to default pattern 11.
At step S40, the first image 21 and the second image 31 are carried out to an image-type depth calculation processing, to obtain a plurality of the 3rd image depth 51 and a plurality of the 3rd confidence level 52.
Wherein, the first image 21 and the 3rd image 22 are made up of a plurality of pixel, and the quantity system of a plurality of the first image depth 41, a plurality of the first confidence levels 42 equals the quantity of a plurality of pixels.And, when the capture resolution of the second photographing module 30 is during lower than the first photographing module 20, image depth production method more comprised before the step of calculating image depth: the image that the second photographing module 30 is captured carries out a conversion of resolution, and the pixel quantity that the second image 31 is comprised equals the first image 21.
Finally, at step S50, according to a plurality of the first confidence levels 42, a plurality of the second confidence level 44 and a plurality of the 3rd confidence level 52, from a plurality of the first image depth 41, a plurality of the second image depth 43 and a plurality of the 3rd image depth 51, be combined into a plurality of the 4th image depth 61.
The foregoing is only illustrative, but not be restricted person.Anyly do not depart from spirit of the present invention and category, and equivalent modifications or change that it is carried out all should be contained in appending claims.