CN104811680A - Image obtaining device and image deformation correction method thereof - Google Patents
Image obtaining device and image deformation correction method thereof Download PDFInfo
- Publication number
- CN104811680A CN104811680A CN201410044039.3A CN201410044039A CN104811680A CN 104811680 A CN104811680 A CN 104811680A CN 201410044039 A CN201410044039 A CN 201410044039A CN 104811680 A CN104811680 A CN 104811680A
- Authority
- CN
- China
- Prior art keywords
- image
- feature point
- reference picture
- pictures
- comparison value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention provides an image obtaining device and an image deformation correction method thereof. The image obtaining device includes a first image sensor and a second image sensor. The image deformation correction method comprises the following steps that image groups are obtained via the first and second image sensors, each image group comprises a first image and a second image, and the image groups include a reference image group; whether image deformation occurs in a first reference image and a second reference image of the reference image group is detected; and when it is detected that image deformation occurs in the reference image group, present correction parameters are updated according to the comparison values of characteristic points corresponding the image groups, and first and second images are corrected by using the present correction parameters.
Description
Technical field
The invention relates to a kind of image acquiring device, and relate to a kind of image acquiring device and image deformation bearing calibration thereof especially.
Background technology
For current picture depth detection technology, using the image acquiring device with twin-lens to obtain the image corresponding to different visual angles is a kind of common method, can be calculated the three-dimensional depth information of object by the image corresponding to different visual angles.Therefore, in order to obtain the three-dimensional depth information of object accurately from two dimensional image, the spatial placement relation between these two camera lenses needs through special design, and the parameter correction of precision is necessary step.Furthermore, when factory's manufacture has the image acquiring device of twin-lens, each self-corresponding locus of twin-lens and direction cannot be arranged in default set point clock like precision.Therefore, in the process manufacturing image acquiring device, the prior twin-lens module for setting corrects by factory, thus obtains the correction parameter of one group of factory preset.In the future, operate in the process of image acquiring device user, image acquiring device can utilize the correction parameter of factory preset to correct the image obtained by twin-lens, to overcome processing procedure disappearance accurate not.
But, in user's operation or carry in the process of image acquiring device, when image acquiring device be squeezed, clash into or fall affect time, camera lens may be caused to produce change on the locus such as displacement or rotation.Once camera lens produces the situation of displacement or distortion, the default correction parameter of inside plants no longer meets current application feature, and image acquiring device also just cannot obtain correct depth information.For example, if during the unbalance problem of the level that produces between the twin-lens of stereopicture obtaining device, because shooting left and right picture level out after unbalance does not mate, will 3 D stereo shooting effect be caused further not good.
Summary of the invention
In view of this, the invention provides a kind of image acquiring device and image deformation bearing calibration thereof, can adjust for the displacement state of imageing sensor the correction parameter carrying out image rectification (imagerectification) adaptively.
The present invention proposes a kind of image deformation bearing calibration, is applicable to the image acquiring device with the first imageing sensor and the second imageing sensor.Image acquiring device has the current correction parameter being associated with the first imageing sensor and the second imageing sensor, and this image deformation bearing calibration comprises the following steps.Obtain multiple groups of pictures by the first imageing sensor and the second imageing sensor, wherein each groups of pictures comprises the first image and the second image respectively, and these groups of pictures comprise a reference picture group.Detect the first reference picture in this reference picture group and whether the second reference picture image deformation occurs.When detecting that image deformation occurs in reference picture group, the multiple characteristic point comparison value corresponding according to these groups of pictures upgrade current correction parameter.This current correction parameter is in order to carry out image rectification to each second image of each first image and correspondence.
In one embodiment of this invention, the step whether above-mentioned detection reference picture group image deformation occurs comprises the following steps.Detect the fisrt feature point of the first reference picture and the second feature point of the second reference picture.Judge whether fisrt feature point and second feature o'clock exceed threshold value respectively at the side-play amount between the first reference picture and the image coordinates of the second reference picture.If be judged as YES, judge that image deformation occurs in reference picture group.
In one embodiment of this invention, the step whether above-mentioned detection reference picture group image deformation occurs comprises the following steps.Carry out three dimensional depth estimation according to the first reference picture and the second reference picture, to produce the reference depth information of the reference focus target thing in reference picture group, and obtain the degree of depth focusing position about reference target thing according to reference depth information.Obtained about the auto-focusing position of reference target thing by auto-focusing program.Judge whether meet auto-focusing position with reference to the degree of depth focusing position corresponding to focus target thing.If be judged as NO, judge that image deformation occurs in reference picture group.
In one embodiment of this invention, above-mentioned upgrade the step of current correction parameter in these characteristic point comparison value corresponding according to these groups of pictures before, this method for correcting image also comprises the following steps.Three dimensional depth estimation is carried out, to produce the depth information of each groups of pictures for these groups of pictures.Depth information according to each groups of pictures determines whether retain groups of pictures.
In one embodiment of this invention, the step that above-mentioned these characteristic point comparison value corresponding according to these groups of pictures upgrades current correction parameter also comprises the following steps.Feature point detection is carried out to these first images and these the second images, and obtains multiple fisrt feature points of these the first images and multiple second feature points of these the second images.The coordinate position of these fisrt feature points of comparison and the coordinate position of multiple second feature points corresponding with these fisrt feature points respectively, to obtain the multiple characteristic point comparison value between these fisrt feature points and these second feature point.Record the multiple characteristic point comparison value between these fisrt feature points and these second feature point.
In one embodiment of this invention, the step of the above-mentioned characteristic point comparison value recorded between these fisrt feature points and these second feature point also comprises the following steps.Each characteristic point comparison value is classified to multiple statistics groove according to the coordinate position of these fisrt feature points and/or the coordinate position of these second feature point.
In one embodiment of this invention, the step that above-mentioned these characteristic point comparison value corresponding according to groups of pictures upgrades current correction parameter comprises the following steps.According to the multiple depth values corresponding to the quantity of multiple characteristic point comparison value in these statistics grooves and characteristic point comparison value, judge whether these characteristic point comparison value enough carry out computing.If be judged as YES, upgrade current correction parameter according to these characteristic point comparison value.Wherein these depth values obtain by carrying out three dimensional depth estimation to fisrt feature point with corresponding second feature point.
From another viewpoint, the present invention proposes a kind of image acquiring device, and this image acquiring device has the first imageing sensor and the second imageing sensor.This image acquiring device also comprises memory cell and processing unit.The multiple module of unit records, and store the current correction parameter being associated with the first imageing sensor and the second imageing sensor.Processing unit couples the first imageing sensor, the second imageing sensor and memory cell, to access and to perform the described module recorded in memory cell.These modules comprise acquisition module, shape changing detection module and parameter update module.Acquisition module obtains multiple groups of pictures by the first imageing sensor and the second imageing sensor.Each groups of pictures comprises the first image of the first imageing sensor and the second image of the second imageing sensor respectively, and these groups of pictures comprise a reference picture group.Whether the first reference picture in shape changing detection module detection reference picture group and the second reference picture there is image deformation.When there is image deformation in reference picture group, multiple characteristic point comparison value renewal current correction parameters that parameter update module is corresponding according to these groups of pictures.Current correction parameter is in order to carry out image rectification to each second image of each first image and correspondence.
Based on above-mentioned, in the embodiment of image deformation bearing calibration of the present invention, when current correction parameter cannot carry out accurately image rectification time, utilize and a database is set up to the characteristic point information in the groups of pictures captured by different scene and different time points, to upgrade current control information by information complete in database.Thus, even if left images transducer produces displacement, image acquiring device also can the dynamic and correction parameter that adaptive generation is new, carries out inaccurate image rectification to avoid utilizing the correction parameter not being inconsistent present situation.By this, can when user without the action automatically carrying out parameter renewal when discovering, to guarantee the shooting quality of image acquiring device and to promote user's experience.
For above-mentioned feature and advantage of the present invention can be become apparent, special embodiment below, and coordinate accompanying drawing to be described in detail below.
Accompanying drawing explanation
Fig. 1 is the calcspar of the image acquiring device shown by one embodiment of the invention;
Fig. 2 is the flow chart of the image deformation bearing calibration shown by one embodiment of the invention;
Fig. 3 A to Fig. 3 B is the detail flowchart of the step S202 shown by one embodiment of the invention;
Fig. 4 is the flow chart of the image deformation bearing calibration shown by one embodiment of the invention;
Fig. 5 A to Fig. 5 B is the embodiment schematic diagram of the characteristic of division point comparison value shown by one embodiment of the invention to statistics groove.
Description of reference numerals:
100: image acquiring device;
110: the first imageing sensors;
120: the second imageing sensors;
130: focusing unit;
140: processing unit;
150: memory cell;
151: acquisition module;
152: shape changing detection module;
153: parameter update module;
154: the degree of depth selects module;
TH: predetermined threshold value;
A, B: fisrt feature point;
Z1 ~ Z9: image block;
S1 ~ S9: statistics groove;
Δ d
a, Δ d
b: characteristic point comparison value;
S201 ~ S203: each step of the image deformation bearing calibration described in one embodiment of the invention;
S2011 ~ S2023: each sub-step of the step S202 of one embodiment of the invention;
S2024 ~ S2027: each sub-step of the step S202 of one embodiment of the invention;
S401 ~ S409: each step of the image deformation bearing calibration described in one embodiment of the invention.
Embodiment
When image acquiring device dispatches from the factory, between its twin-lens, spatial placement relation has passed through accurate calculating and adjustment, and produces the correction parameter of one group of factory preset according to this.The correction parameter of this factory preset in order to image rectification that different camera lens is obtained to through design and fixing coordinate parameters relationship.Cause because of twin-lens generation displacement or rotation the situation that the correction parameter of factory preset is no longer suitable for solve, the present invention produces according to the depth information of image and pixel position the database recording multiple characteristic point information, and utilizes the adaptive renewal correction parameter of the information of accumulating in database.In order to make content of the present invention more clear, below enumerate the embodiment that embodiment can be implemented really accordingly as the present invention.
Fig. 1 is the calcspar of the image acquiring device shown by one embodiment of the invention.Please refer to Fig. 1, the image acquiring device 100 of the present embodiment is such as digital camera, digital camera, or other have the portable electric device of image-acquisition functions, similarly are smart mobile phone, panel computer etc., are not limited to above-mentioned.Image acquiring device 100 comprises the first imageing sensor 110, second imageing sensor 120, focusing unit 130, processing unit 140 and memory cell 150.
First imageing sensor 110 and the second imageing sensor 120 can comprise camera lens and photo-sensitive cell.Photo-sensitive cell is such as charge coupled cell (Charge Coupled Device, be called for short CCD), Complimentary Metal-Oxide semiconductor (Complementary Metal-Oxide Semiconductor, be called for short CMOS) element or other elements, first imageing sensor 110 and the second imageing sensor 120 also can comprise aperture etc., neither limit at this.In addition, according to the camera lens setting position of the first imageing sensor 110 and the second imageing sensor 120, the camera lens of the first imageing sensor 110 and the second imageing sensor 120 can divide into left camera lens and right camera lens.
In the present embodiment, focusing unit 130 couples the first imageing sensor 110, second imageing sensor 120 and processing unit 140, obtains the focal length of image in order to control the first imageing sensor 110 and the second imageing sensor 120.In other words, the lens moving of camera lens and the second imageing sensor 120 that unit 130 of focusing controls the first imageing sensor 110 is to focusing position.Focusing unit 130 such as controls step number (step) position of camera lens by voice coil motor (Voice Coil Motor is called for short VCM) or other dissimilar motors, to change the focal length of the first imageing sensor 110 and the second imageing sensor 120.
Processing unit 140 can be such as CPU (Central Processing Unit, be called for short CPU), microprocessor (Microprocessor), Application Specific Integrated Circuit (Application SpecificIntegrated Circuits, be called for short ASIC), programmable logic device (Programmable Logic Device, be called for short PLD) or other possess the hardware unit of operational capability.Memory cell 150 is such as random access memory (random access memory), fast storage (Flash) or other memory, in order to store current correction parameter and multiple module, and processing unit 140 couples memory cell 150 and in order to perform these modules.Above-mentioned module comprises acquisition module 151, shape changing detection module 152, parameter update module 153 and the degree of depth and selects module 154, and these modules are such as computer programs, and it can be loaded into processing unit 140, thus performs the function of correcting image deformation.
Fig. 2 is the flow chart of the image deformation bearing calibration shown by one embodiment of the invention.The method of the present embodiment is applicable to the image acquiring device 100 of Fig. 1, and namely each component of arranging in pairs or groups in image acquiring device 100 illustrates the detailed step of the present embodiment image deformation bearing calibration below.
First, in step S201, acquisition module 151 obtains multiple groups of pictures by the first imageing sensor 110 and the second imageing sensor 120.Each groups of pictures comprises the first image and the second image respectively, and groups of pictures at least comprises a reference picture group.That is, in the present embodiment, single image group has two photos, and the first image in same groups of pictures and the second image are at one time for two images that Same Scene obtains by left camera lens and right camera lens.In other words, the first image is such as the left image obtained by left camera lens, and the right image of the second image relatively for being obtained by right camera lens.In the present embodiment, the first image and the second image are such as the instant preview images (live-view image) obtained under preview state.
Similarly, the groups of pictures that reference picture group obtains for image acquiring device 100 one of them, therefore reference picture group has the first reference picture and the second reference picture that correspond to the first imageing sensor 110 and the second imageing sensor 120 equally.In step S202, shape changing detection module 152 detects the first reference picture in reference picture group and whether the second reference picture image deformation occurs.It should be noted that, the mode that shape changing detection module 152 can regularly detect carries out the detection of image deformation to part groups of pictures, also can carry out the detection of image deformation for all groups of pictures, and reference picture group represents shape changing detection module 152 in order to detect one of the object whether image-type occurring and becomes in this.
It should be noted that, the correction parameter of factory preset is applicable to two left images to carry out image rectification (image rectification) more respectively, allows two true pictures become and only has horizontal aberration or only have vertical aberration (because the relation that lens location is put causes).Such as, narrow difference having the angle elevation angle between twin-lens etc.Corrected by the correction parameter carries out image of factory preset, true picture can be converted to left and right camera lens is put same capture plane, only remaining level or upright position variant.That is, under the horizontally disposed prerequisite of left and right camera lens, each pixel on the left images of image rectification should only remaining horizontal level be variant.Now, change if the shooting direction of left and right camera lens produces, the upright position of each pixel on the left images of image rectification still has difference, and this phenomenon is referred to herein as image deformation.At this, shape changing detection module 152 can according to the side-play amount of characteristic of correspondence point mutual on the first reference picture and the second reference picture or carrying out three dimensional depth for the first reference picture and the second reference picture estimates to judge whether reference picture group image deformation occurs.
Clearer, Fig. 3 A is the detail flowchart of the step S202 shown by one embodiment of the invention.In the embodiment as shown in fig. 3 a, in step S2021, shape changing detection module 152 detects the fisrt feature point on the first reference picture and the second feature point on the second reference picture.Afterwards, in step S2022, shape changing detection module 152 judges whether fisrt feature point and the side-play amount of second feature point respectively between the first reference picture and the image coordinates of the second reference picture exceed threshold value.If fisrt feature point and the side-play amount of second feature point respectively between the first reference picture and the image coordinates of the second reference picture exceed threshold value, in step S2023, shape changing detection module 152 judges that image deformation occurs in this reference picture group.That is, by analyzing and add up the displacement information between fisrt feature point and second feature point, whether the first reference picture and the second reference picture generation image deformation can be learnt accordingly.
In other words, shape changing detection module 152 can detect arbitrary characteristic point of reference group image according to the calculation method of existing feature point detection.The calculation method of feature point detection is in order to detect the majority characteristic point in image, and be such as rim detection (edge detection), corner detection (Conner detection) or other feature point detection algorithms, the present invention does not limit this.Afterwards, shape changing detection module 152 judges whether the coordinate position side-play amount between mutually corresponding fisrt feature point and second feature point exceedes above-mentioned threshold value, detects reference group image accordingly and whether image deformation occurs.For example, shape changing detection module 152 can judge whether the vertical offset (the displacement difference distance in vertical direction) of mutually corresponding fisrt feature point and second feature point exceedes above-mentioned threshold value.When shape changing detection module 152 judges that above-mentioned vertical offset exceedes above-mentioned threshold value, represent reference picture group and image deformation occurs.
In another embodiment, Fig. 3 B is the detail flowchart of step S202 shown by one embodiment of the invention.In the embodiment shown in figure 3b, in step S2024, shape changing detection module 152 carries out three dimensional depth estimation according to the first reference picture and the second reference picture, to produce the reference depth information of the reference focus target thing in reference picture group, and obtain the degree of depth focusing position about reference target thing according to reference depth information.Then, in step S2025, shape changing detection module 152 obtains about the auto-focusing position of reference target thing by auto-focusing program.In step S2026, shape changing detection module 152 judges whether meet auto-focusing position with reference to the degree of depth focusing position corresponding to focus target thing.If be judged as NO, in step S2027, shape changing detection module 152 judges that image deformation occurs in reference picture group.
Specifically, shape changing detection module 152 carries out image procossing by stereovision technique, in the hope of the depth information of each point in object three-dimensional coordinate position in space and image.Moreover, be such as obtain focusing position about object according to the depth information query depth table of comparisons according to the depth information step obtained about the degree of depth focusing position of object.Therefore, by the current value of step number or voice coil motor of trying to achieve stepper motor in advance and the corresponding relation of object definition, then can inquire the step number of the stepper motor corresponding to this depth information or the current value of voice coil motor according to the depth information of the object obtained at present, and obtain the degree of depth focusing position about object accordingly.
On the other hand, the process performing auto-focusing program can be automatically control camera lens module by focusing unit 130 to move on a large scale, to adjust the camera lens of the first imageing sensor 110 and the second imageing sensor 120 respectively to required focusing position, to obtain the auto-focusing position about object.Focusing unit 130 is such as utilize the climbing method (hill-climbing) used in Autofocus Technology to obtain auto-focusing position about object, but the present invention is not as limit.Therefore, under the condition of image deformation does not occur for the first reference picture and the second reference picture, image acquiring device 100 can obtain desirable depth information, cause degree of depth focusing position can with auto-focusing position consistency.If image acquiring device 100 cannot obtain desirable depth information further according to current correction parameter, also just cannot be estimated by depth information and the depth information query depth table of comparisons stored in advance and correct degree of depth focusing position, therefore degree of depth focusing position will produce difference with the auto-focusing position obtained by auto-focusing program.Accordingly, according to the difference between degree of depth focusing position and auto-focusing position, shape changing detection module 152 judges whether reference picture group image deformation occurs.
Referring again to Fig. 2, when shape changing detection module 152 detects that image deformation occurs in reference picture group, in step S203, multiple characteristic point comparison value renewal current correction parameters that parameter update module 153 is corresponding according to multiple groups of pictures, wherein current correction parameter is in order to carry out image rectification to each second image of each first image and correspondence.That is, when image acquiring device 100 judge the first imageing sensor 110 and the second imageing sensor 120 produce be out of shape or be shifted and cause the parameter coordinate between the first reference picture and the second reference picture to change time, represent current correction parameter and cannot carry out image rectification accurately to image.
Therefore, in one embodiment, parameter update module 153 starts the characteristic point comparison value collecting multiple groups of pictures captured after reference picture group, to come in produce desirable current correction parameter by producing the rear image obtained that is out of shape or is shifted at the first imageing sensor 110 and the second imageing sensor 120.Specifically parameter update module 153 obtains these characteristic point comparison value by the coordinate position of comparison fisrt feature point and the coordinate position of second feature point corresponding with these fisrt feature points respectively.Moreover parameter update module 153 also can produce new current correction parameter according to the coordinate position of the depth information of image and pixel.Below will enumerate another embodiment to describe it in detail.
Fig. 4 is the flow chart of a kind of image deformation bearing calibration shown by one embodiment of the invention.Please refer to Fig. 4, the method for the present embodiment is applicable to the image acquiring device 100 of Fig. 1, and namely each component of arranging in pairs or groups in image acquiring device 100 illustrates the detailed step of the present embodiment image deformation bearing calibration below.
First, in step S401, acquisition module 151 obtains multiple groups of pictures by the first imageing sensor 110 and the second imageing sensor 120.Each groups of pictures comprises the first image respectively with one second image, and groups of pictures comprises reference picture group.In step S402, shape changing detection module 152 detects the first reference picture in reference picture group and whether the second reference picture image deformation occurs.Step S201 and the step S202 of step S401 and step S402 and previous embodiment are similar or identical, do not repeat them here.
If when shape changing detection module judges that image deformation occurs in reference picture group, in step S403, the degree of depth selects module 154 to carry out three dimensional depth estimation for groups of pictures, to produce the depth information of each groups of pictures, and determines whether retain groups of pictures according to the depth information of each groups of pictures.Furthermore, the degree of depth selects module 154 to produce the three dimensional depth figure being associated with the first reference picture and the second reference picture by the image processing techniques of stereoscopic vision.Based on the depth information in three dimensional depth figure, degree of depth selection module 154 can obtain the field depth corresponding to above-mentioned three dimensional depth figure, and decides retain or abandon groups of pictures according to field depth.
Specifically, suppose that the minimum value of depth value is set as 0 and maximum is set as 128, namely the depth value of groups of pictures falls within the number range of 0 ~ 128.If the degree of depth selects module 154 to collect the groups of pictures that field depth is depth value 100 to depth value 128, the degree of depth is selected to fall within groups of pictures in depth value 100 to depth value 128 by not retaining other field depths after module 154.Otherwise, if the degree of depth selects module 154 to judge that the field depth of current groups of pictures falls within outside depth value 100 to depth value 128, it is such as the groups of pictures of depth value 0 to depth value 80, the degree of depth selects module 154 will retain this groups of pictures, to utilize the characteristic point information of this groups of pictures further.
In other words, the degree of depth selects module 154 to judge whether each groups of pictures is effective groups of pictures according to the depth of view information of each groups of pictures.If the field depth of the image module of up-to-date acquisition has had most overlapping with the field depth of previous groups of pictures, the degree of depth has selected module 154 will filter it accordingly.Base this, in one embodiment, except by judge field depth whether the overlapping reservation carrying out groups of pictures with filter except, the degree of depth selects module 154 also can determine whether retaining groups of pictures according to the Duplication of field depth.Base this, can guarantee that the degree of depth selects module 154 to collect the information corresponding to all or most of field depth, and the field depth simultaneously corresponding to each groups of pictures filters unnecessary information, to reduce data processing amount and to promote data processing speed.
Afterwards, in step S404, parameter update module 153 carries out feature point detection to the first image and the second image, and obtains multiple fisrt feature point of the first image and multiple second feature points of the second image.In step S405, the coordinate position of parameter update module 153 comparison fisrt feature point and the coordinate position of second feature point corresponding with fisrt feature point respectively, to obtain the characteristic point comparison value between fisrt feature point and second feature point.In step S406, parameter update module 153 records the characteristic point comparison value between fisrt feature point and second feature point.
Furthermore, parameter update module 153 can detect the characteristic point of the first image and the second image in each groups of pictures equally according to the calculation method of existing feature point detection, with the fisrt feature point on the first image got and the second feature point on the second image.Then, parameter update module 153 judges the fisrt feature point of mutually coupling and the side-play amount of second feature point under same coordinate system (offset) and using side-play amount as characteristic point comparison value.Wherein, mutually the fisrt feature point of coupling and the same position in second feature spot projection extremely scene being shot.In other words, characteristic point comparison value also can be considered the aberration between fisrt feature point and second feature point.Afterwards, the characteristic point comparison value between fisrt feature point and second feature point is recorded to a database by parameter update module 153, to set up the correction database upgrading current correction parameter.It is worth mentioning that, when judging that deformation occurs in reference picture group, acquisition module 151 still continues obtain image and obtain multiple groups of pictures, and parameter update module 153 also continues the characteristic point comparison value of each groups of pictures produced by calculating to be recorded to correction database.
In step S407, parameter update module 153 classifies each characteristic point comparison value to multiple statistics groove according to the coordinate position of the coordinate position of fisrt feature point and/or second feature point.That is, parameter update module 153, except being recorded to except correction database by characteristic point comparison value, also corresponding to the coordinate position under a coordinate system according to characteristic point comparison value and is classified to different statistics groove.Specifically, in one embodiment, the first image area that the first imageing sensor 110 obtains can be divided into multiple image block by parameter update module 153, and each image block corresponds to a statistics groove.Therefore according to the coordinate position of fisrt feature point, parameter update module 153 characteristic point comparison value sequentially can be corresponded to these image block one of them, thus above-mentioned characteristic point comparison value is classified to corresponding statistics groove.
For example, Fig. 5 A and Fig. 5 B is that characteristic of division point comparison value shown by one embodiment of the invention is to the embodiment schematic diagram adding up groove.In the present embodiment, first with reference to Fig. 5 A, the first image Img1 is divided into 9 image block Z1 ~ Z9 that three take advantage of three by parameter update module 153.Parameter update module 153 more decides fisrt feature point according to the coordinate position of fisrt feature point and drops on that image block.As shown in Figure 5A, according to the coordinate position of fisrt feature point A, parameter update module 153 can learn that fisrt feature point A is positioned at image block Z2.Similarly, according to the coordinate position of fisrt feature point B, parameter update module 153 can learn that fisrt feature point B is positioned at image block Z6.
Please refer to Fig. 5 B, in the present embodiment, the first image Img1 is split into 9 image block Z1 ~ Z9, and image block Z1 ~ Z9 corresponds to statistics groove S1 ~ S9 respectively.Wherein, image block Z1 corresponds to statistics groove S1, and image block Z2 corresponds to statistics groove S2, and the rest may be inferred.Base this, because fisrt feature point A is positioned at image block Z2, the characteristic point comparison value Δ d corresponding to fisrt feature point A
abe classified to statistics groove S2.Because fisrt feature point B is positioned at image block Z6, and the characteristic point comparison value Δ d corresponding to fisrt feature point B
bbe classified to statistics groove S6.It should be noted that, Fig. 5 A and Fig. 5 B is only a kind of exemplary embodiment, non-to limit the present invention.
In addition, in one embodiment, based on the same position point that fisrt feature point and corresponding second feature point are in scene being shot, on the first image and the second image, characteristic of correspondence point will be projected to the same seat punctuate under three-dimensional coordinate system after calculating via coordinates translation.Therefore, characteristic point comparison value can be classified to corresponding statistics groove according to fisrt feature point with the coordinate position be projected under three-dimensional coordinate system of second feature point by parameter update module 153.Further, carry out image procossing based on stereovision technique, parameter update module 153 can try to achieve the depth information of each point in image and the three-dimensional coordinate position of correspondence.The horizontal component of the tripleplane's point corresponding to fisrt feature point and second feature point and vertical component can be decided the statistics groove corresponding to characteristic point comparison value by parameter update module 153.
So in step S408, the multiple depth values of parameter update module 153 corresponding to the quantity of characteristic point comparison value in each statistics groove and characteristic point comparison value, whether judging characteristic point comparison value enough carries out computing.Wherein depth value obtains by carrying out three dimensional depth estimation to fisrt feature point with corresponding second feature point.It is appreciated that the rising of quantity along with groups of pictures, the amount of information recorded in correction database also gets more and more.At this, the depth value of quantity and characteristic point comparison value that parameter update module 153 can add up characteristic point comparison value in groove according to each judges that whether current record is enough in the data volume of correction database.
Based on aforementioned known, these characteristic point comparison value are classified to corresponding statistics groove by parameter update module 153 according to characteristic point information.Therefore, according to the total number of characteristic point comparison value in each statistics groove, parameter update module 153 can judge whether correction database possesses enough data volumes.Specifically, meet camera lens instantly arrange situation to be become a full member by left images to the current correction parameter of perfect condition to produce, what provide that the characteristic point of characteristic point information preferably can be average is distributed on image.By the characteristic point information that the characteristic point being evenly distributed in each region on image provides, rotation amount or the skew condition of whole image can be calculated more accurately.
In the present embodiment, these characteristic point comparison value are classified to corresponding statistics groove by parameter update module 153 according to the coordinate position of characteristic point, and therefore the sum of the characteristic point comparison value of each statistics corresponding to groove can indicate the space distribution situation of characteristic point.Therefore, parameter update module 153 judges that whether the quantity of the characteristic point comparison value in each statistics groove is enough, is used as the decision mechanism whether determination data amount is enough to calculate current correction parameter accurately.
Be described for Fig. 5 B, characteristic point comparison value is classified in 9 statistics groove S1 ~ S9 by parameter update module 153, wherein adds up groove S1 and at least comprises characteristic point comparison value Δ dA, and statistics groove S6 at least comprises characteristic point comparison value Δ dB.When each characteristic point comparison value of the lasting record different images group of parameter update module 153 is in each statistics groove, the characteristic point comparison value in each statistics groove also continues accumulation (as shown in the dotted line of Fig. 5 B).Once the characteristic point comparison value in each statistics groove S1 ~ S9 is enough, parameter update module 153 can start the update action of carrying out current correction parameter.For example, parameter update module 153 can judge whether the number that each statistics groove S1 ~ S9 is corresponding exceedes predetermined threshold value TH and determine that whether the data volume in correction database is enough.So, above-described embodiment is only a kind of exemplary embodiment, and is not used to limit the present invention.Have in this technical field and usually know that the knowledgeable is when selecting the mode classification of feature comparison value according to the actual requirements and according to adding up the whether enough Rule of judgment of groove decision data amount, repeat no more herein.
In addition, whether the data volume that also judges in current correction database by depth information of parameter update module 153 is enough.Specifically, estimate can obtain each self-corresponding depth value of these characteristic point comparison value (depth value) by carrying out three dimensional depth to fisrt feature point and corresponding second feature point.Based on aforementioned known, the degree of depth selects module 154 the excessive groups of pictures of field depth repeatability to be filtered out, and parameter update module 153 will determine whether to collect characteristic of correspondence point comparison value for most depth value.In other words, in one embodiment, parameter update module 153 to be classified each characteristic point comparison value according to each self-corresponding depth value of characteristic point comparison value equally, and judge that whether the number of the characteristic point comparison value corresponding to each depth value is enough, and judge that whether the data volume in correction database is enough.
It is worth mentioning that, if in the process setting up correction database, when the spatial placement relation between the first imageing sensor 110 and the second imageing sensor changes again, the recorded data represented in correction database cannot use.In one embodiment, parameter update module 153 also can before recording feature point comparison value to correction database, judges that whether this characteristic point comparison value has quite poor with the data in correction database different.If so, previous recorded data can abandon by parameter update module 153, and restarts to set up another correction database, therefore obtains optimal current correction parameter.
That is, once parameter update module 153 judges that the data volume in correction database is enough, parameter update module 153 just can stop recording and start to calculate new current correction parameter.Otherwise parameter update module 153 continues to record new characteristic point comparison value to correction database.Therefore, if step S408 is judged as YES, in step S409, parameter update module 153 upgrades current correction parameter according to characteristic point comparison value.Parameter update module 153 such as utilizes the feature comparison value in correction database to find out best current correction parameter to carry out optimization algorithm, causes two images after current correction parameter correction to may correspond to desirable parameter coordinate relation.Optimization algorithm is such as gradient descent method (gradient decentmethod), Lai Wenbeige-Ma quart method (Levenberg-Marquardt method, be called for short LMmethod) or Gauss-Newton algorithm (Gauss-Newton method) etc., the present invention does not limit this.
In sum, in one embodiment of this invention, when image acquiring device detects image generation deformation timely, corrected the current correction parameter being pre-stored in image acquiring device adaptively by the foundation of correction database, to correct left images to desirable parameter coordinate relation.Thus, can when user without the adjustment carrying out current correction parameter when discovering, to guarantee the shooting quality of image acquiring device.Moreover whether the characteristic point information that embodiments of the invention can judge in database according to depth information and characteristic point position is further collected complete.By this, once collect enough data volumes, new current correction parameter can be produced by the characteristic point comparison value recorded in correction database immediately, thus significantly shorten gather data and the required time carrying out correcting.
Last it is noted that above each embodiment is only in order to illustrate technical scheme of the present invention, be not intended to limit; Although with reference to foregoing embodiments to invention has been detailed description, those of ordinary skill in the art is to be understood that: it still can be modified to the technical scheme described in foregoing embodiments, or carries out equivalent replacement to wherein some or all of technical characteristic; And these amendments or replacement, do not make the essence of appropriate technical solution depart from the scope of various embodiments of the present invention technical scheme.
Claims (14)
1. an image deformation bearing calibration, be applicable to the image acquiring device with the first imageing sensor and the second imageing sensor, wherein this image acquiring device has the current correction parameter being associated with this first imageing sensor and this second imageing sensor, it is characterized in that, this image deformation bearing calibration comprises:
Obtain multiple groups of pictures by this first imageing sensor and this second imageing sensor, wherein each described groups of pictures comprises the first image and the second image respectively, and described groups of pictures comprises reference picture group;
Detect the first reference picture in this reference picture group and whether the second reference picture image deformation occurs; And
When detecting that this image deformation occurs in this reference picture group, the multiple characteristic point comparison value corresponding according to described groups of pictures upgrade this current correction parameter, and wherein this current correction parameter is in order to carry out image rectification to each described second image of each described first image and correspondence.
2. image deformation bearing calibration according to claim 1, is characterized in that, detects the step whether this reference picture group this image deformation occurs and comprises:
Detect the fisrt feature point of this first reference picture and the second feature point of this second reference picture; And
Judge whether this fisrt feature point and the side-play amount of this second feature point respectively between this first reference picture and image coordinates of this second reference picture exceed threshold value, if so, judge that this image deformation occurs in this reference picture group.
3. image deformation bearing calibration according to claim 1, is characterized in that, detects the step whether this reference picture group this image deformation occurs and comprises:
Three dimensional depth estimation is carried out according to this first reference picture and this second reference picture, to produce the reference depth information of the reference focus target thing in this reference picture group, and obtain the degree of depth focusing position about this reference target thing according to this reference depth information;
Obtained about the auto-focusing position of this reference target thing by auto-focusing program; And
Judge whether this degree of depth focusing position corresponding to reference focus target thing meets this auto-focusing position, if not, judges that this image deformation occurs in this reference picture group.
4. image deformation bearing calibration according to claim 1, is characterized in that, before the described characteristic point comparison value corresponding according to described groups of pictures upgrades the step of this current correction parameter, this method for correcting image also comprises:
Three dimensional depth estimation is carried out, to produce the depth information of each described groups of pictures for described groups of pictures; And
This depth information according to each described groups of pictures determines whether retain described groups of pictures.
5. image deformation bearing calibration according to claim 1, is characterized in that, the step that the described characteristic point comparison value corresponding according to described groups of pictures upgrades this current correction parameter also comprises:
Feature point detection is carried out to described first image and described second image, and obtains multiple fisrt feature point of described first image and multiple second feature points of described second image;
The coordinate position of fisrt feature point described in comparison and the coordinate position of described second feature point corresponding with described fisrt feature point respectively, to obtain the described characteristic point comparison value between described fisrt feature point and described second feature point; And
Record the described characteristic point comparison value between described fisrt feature point and described second feature point.
6. image deformation bearing calibration according to claim 5, is characterized in that, the step recording the described characteristic point comparison value between described fisrt feature point and described second feature point also comprises:
According to the coordinate position of described fisrt feature point and/or each described characteristic point comparison value of coordinate position classification extremely multiple statistics groove of described second feature point.
7. image deformation bearing calibration according to claim 6, is characterized in that, the step that the described characteristic point comparison value corresponding according to described groups of pictures upgrades this current correction parameter comprises:
According to the multiple depth values corresponding to the quantity of characteristic point comparison value described in described statistics groove and described characteristic point comparison value, judge whether described characteristic point comparison value enough carries out computing, if, upgrade this current correction parameter according to described characteristic point comparison value, wherein said depth value obtains by carrying out three dimensional depth estimation to described fisrt feature point with corresponding described second feature point.
8. an image acquiring device, have the first imageing sensor and the second imageing sensor, it is characterized in that, this image acquiring device comprises:
Memory cell, records multiple module, and stores the current correction parameter being associated with this first imageing sensor and this second imageing sensor; And
Processing unit, couples this first imageing sensor, this second imageing sensor and memory cell, and to access and to perform in this memory cell the described module recorded, described module comprises:
Acquisition module, multiple groups of pictures is obtained by this first imageing sensor and this second imageing sensor, each described groups of pictures comprises the first image of this first imageing sensor and the second image of this second imageing sensor respectively, and described groups of pictures comprises reference picture group;
Shape changing detection module, detects the first reference picture in this reference picture group and whether the second reference picture image deformation occurs;
Parameter update module, when there is this image deformation in this reference picture group, this parameter update module multiple characteristic point comparison value corresponding according to described groups of pictures upgrade this current correction parameter, and wherein this current correction parameter is in order to carry out image rectification to each described second image of each described first image and correspondence.
9. image acquiring device according to claim 8, it is characterized in that, this shape changing detection module detects the fisrt feature point of this first reference picture and the second feature point of this second reference picture, this shape changing detection module judges whether this fisrt feature point and the side-play amount of this second feature point respectively between this first reference picture and image coordinates of this second reference picture exceed threshold value, if so, this shape changing detection module judges that this image deformation occurs in this reference picture group.
10. image acquiring device according to claim 8, it is characterized in that, this shape changing detection module carries out three dimensional depth estimation according to this first reference picture and this second reference picture, to produce the reference depth information of the reference focus target thing in this reference picture group, and obtain the degree of depth focusing position about this reference target thing according to this reference depth information, this shape changing detection module obtains about the auto-focusing position of this reference target thing by auto-focusing program, and this shape changing detection module judges whether this degree of depth focusing position corresponding to reference focus target thing meets this auto-focusing position, if not, this shape changing detection module judges that this image deformation occurs in this reference picture group.
11. image acquiring devices according to claim 8, is characterized in that, described module also comprises:
The degree of depth selects module, this degree of depth selects module to carry out three dimensional depth estimation for described groups of pictures, to produce the depth information of each described groups of pictures, and this degree of depth selects module to determine whether retain described groups of pictures according to this depth information of each described groups of pictures.
12. image acquiring devices according to claim 8, it is characterized in that, this parameter update module carries out feature point detection to described first image and described second image, and obtain multiple fisrt feature point of described first image and multiple second feature points of described second image, this parameter update module is to the coordinate position of the coordinate position of described fisrt feature point and described second feature point corresponding with described fisrt feature point respectively, to obtain the described characteristic point comparison value between described fisrt feature point and described second feature point, described characteristic point comparison value described in this parameter update module record between fisrt feature point and described second feature point.
13. image acquiring devices according to claim 12, is characterized in that, this parameter update module classifies each described characteristic point comparison value to multiple statistics groove according to the coordinate position of the coordinate position of described fisrt feature point and/or described second feature point.
14. image acquiring devices according to claim 13, it is characterized in that, the multiple depth values of this parameter update module corresponding to the quantity of characteristic point comparison value described in described statistics groove and described characteristic point comparison value, judge whether described characteristic point comparison value enough carries out computing, if, this parameter update module upgrades this current correction parameter according to described characteristic point comparison value, and wherein said depth value obtains by carrying out three dimensional depth estimation to described fisrt feature point with corresponding described second feature point.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410044039.3A CN104811680B (en) | 2014-01-28 | 2014-01-28 | image obtaining device and image deformation correction method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410044039.3A CN104811680B (en) | 2014-01-28 | 2014-01-28 | image obtaining device and image deformation correction method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104811680A true CN104811680A (en) | 2015-07-29 |
| CN104811680B CN104811680B (en) | 2017-04-12 |
Family
ID=53696135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410044039.3A Active CN104811680B (en) | 2014-01-28 | 2014-01-28 | image obtaining device and image deformation correction method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104811680B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106817575A (en) * | 2015-11-30 | 2017-06-09 | 聚晶半导体股份有限公司 | The method of the method and automatic corrigendum of image capturing device and its generation depth information |
| CN110580718A (en) * | 2018-06-07 | 2019-12-17 | 钰立微电子股份有限公司 | image device correction method, and related image device and arithmetic device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201201114A (en) * | 2010-06-17 | 2012-01-01 | Pixart Imaging Inc | Image sensing module |
| TW201310004A (en) * | 2011-08-18 | 2013-03-01 | Nat Applied Res Laboratories | Correlation arrangement device of digital images |
| CN102313515B (en) * | 2010-07-08 | 2013-10-23 | 国立清华大学 | Three-dimensional digital image correlation (3D-DIC) correcting method |
| CN103444191A (en) * | 2010-12-07 | 2013-12-11 | 罗伯特·博世有限公司 | Method and device for processing image information of two sensors suitable for capturing images, in a stereo-ensor system |
-
2014
- 2014-01-28 CN CN201410044039.3A patent/CN104811680B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201201114A (en) * | 2010-06-17 | 2012-01-01 | Pixart Imaging Inc | Image sensing module |
| CN102313515B (en) * | 2010-07-08 | 2013-10-23 | 国立清华大学 | Three-dimensional digital image correlation (3D-DIC) correcting method |
| CN103444191A (en) * | 2010-12-07 | 2013-12-11 | 罗伯特·博世有限公司 | Method and device for processing image information of two sensors suitable for capturing images, in a stereo-ensor system |
| TW201310004A (en) * | 2011-08-18 | 2013-03-01 | Nat Applied Res Laboratories | Correlation arrangement device of digital images |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106817575A (en) * | 2015-11-30 | 2017-06-09 | 聚晶半导体股份有限公司 | The method of the method and automatic corrigendum of image capturing device and its generation depth information |
| CN110580718A (en) * | 2018-06-07 | 2019-12-17 | 钰立微电子股份有限公司 | image device correction method, and related image device and arithmetic device |
| CN110580718B (en) * | 2018-06-07 | 2023-08-08 | 钰立微电子股份有限公司 | Correction method for image device, related image device and computing device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104811680B (en) | 2017-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI511081B (en) | Image capturing device and method for calibrating image deformation thereof | |
| US9697604B2 (en) | Image capturing device and method for detecting image deformation thereof | |
| US20160295097A1 (en) | Dual camera autofocus | |
| CN105453136B (en) | The three-dimensional system for rolling correction, method and apparatus are carried out using automatic focus feedback | |
| US9313390B2 (en) | Systems and methods to calibrate a multi camera device | |
| CN102984530B (en) | Image processing system and Atomatic focusing method | |
| US9501828B2 (en) | Image capturing device, image capturing device control method, and program | |
| KR20170056698A (en) | Autofocus method, device and electronic apparatus | |
| EP3416370A1 (en) | Photography focusing method, device, and apparatus for terminal | |
| US10764486B2 (en) | Multi-camera autofocus synchronization | |
| US9693041B2 (en) | Image capturing device and method for calibrating image deformation thereof | |
| TWI471677B (en) | Auto focus method and auto focus apparatus | |
| CN104102068A (en) | Automatic focusing method and automatic focusing device | |
| CN106031148B (en) | Imaging device, method of auto-focusing in an imaging device and corresponding computer program | |
| US20170230649A1 (en) | Calibration of hybrid auto focus (af) imaging systems | |
| CN102196166A (en) | Imaging device and display method | |
| JP6694281B2 (en) | Stereo camera and imaging system | |
| US20140327743A1 (en) | Auto focus method and auto focus apparatus | |
| CN104811680A (en) | Image obtaining device and image deformation correction method thereof | |
| CN104811688B (en) | Image acquiring device and its image deformation detection method | |
| TWI621831B (en) | Image capturing device and calibration method for phase detection auto focus thereof | |
| CN108243305B (en) | Image capturing device and phase focusing correction method thereof | |
| CN104811604B (en) | Image acquiring device and its image deformation bearing calibration | |
| CN104935815A (en) | Shooting method, shooting device, camera and mobile terminal | |
| CN110839126A (en) | Zoom tracking method and device and zoom camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |