CN104811680B - image obtaining device and image deformation correction method thereof - Google Patents
image obtaining device and image deformation correction method thereof Download PDFInfo
- Publication number
- CN104811680B CN104811680B CN201410044039.3A CN201410044039A CN104811680B CN 104811680 B CN104811680 B CN 104811680B CN 201410044039 A CN201410044039 A CN 201410044039A CN 104811680 B CN104811680 B CN 104811680B
- Authority
- CN
- China
- Prior art keywords
- image
- reference picture
- feature point
- group
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention provides an image obtaining device and an image deformation correction method thereof. The image obtaining device includes a first image sensor and a second image sensor. The image deformation correction method comprises the following steps that image groups are obtained via the first and second image sensors, each image group comprises a first image and a second image, and the image groups include a reference image group; whether image deformation occurs in a first reference image and a second reference image of the reference image group is detected; and when it is detected that image deformation occurs in the reference image group, present correction parameters are updated according to the comparison values of characteristic points corresponding the image groups, and first and second images are corrected by using the present correction parameters.
Description
Technical field
The invention relates to a kind of image acquiring device, and in particular to a kind of image acquiring device and its image
Deformation correction method.
Background technology
For current picture depth detection technology, correspondence is obtained using the image acquiring device with twin-lens
Image to different visual angles is a kind of common method, can calculate the three-dimensional of object by the image of correspondence to different visual angles
Depth information.Therefore, in order to accurately from two dimensional image obtain object three-dimensional depth information, this two camera lenses it
Between to arrange relation needed through being specifically designed in space, and the parameter correction of precision is necessary step.Furthermore, it is understood that working as
During factory's image acquiring device of the manufacture with twin-lens, each self-corresponding locus of twin-lens cannot be extremely accurate with direction
Be arranged in default setting value.Therefore, during manufacture image acquiring device, factory is directed to what is set in advance
Bimirror head module is corrected, so as to obtain the correction parameter of one group of factory preset.In the future, obtain in user operation diagram picture and fill
During putting, image acquiring device can be corrected by the image acquired in twin-lens using the correction parameter of factory preset,
With the disappearance for overcoming processing procedure not accurate enough.
However, during user is operated or carries image acquiring device, when image acquiring device is squeezed, hits
During the impact hit or fall, camera lens may be caused to produce the change on the locus such as displacement or rotation.Once camera lens produces position
The situation moved or deform, the default correction parameter of inside plants have been no longer complies with current application feature, and image obtains dress
Put.For example, if level is produced between the twin-lens of stereopicture obtaining device
During unbalance problem, as the left and right picture level shot out after unbalance is mismatched, 3 D stereo bat is will further result in
Take the photograph effect on driving birds is not good.
The content of the invention
In view of this, the present invention provides a kind of image acquiring device and its image deformation bearing calibration, can pass for image
The displacement state of sensor and adaptively adjust to carry out image rectification（image rectification）Correction parameter.
The present invention proposes a kind of image deformation bearing calibration, it is adaptable to the first imageing sensor and the second image sensing
The image acquiring device of device.Image acquiring device has the current school for being associated with the first imageing sensor and the second imageing sensor
Just parameter, and this image deformation bearing calibration comprises the following steps.By the first imageing sensor and the second imageing sensor
Multiple images group is obtained, wherein each groups of pictures includes the first image and the second image respectively, these groups of pictures include
One reference picture group.Detect whether the first reference picture and the second reference picture in this reference picture group occur image shape
Become.When reference picture group generation image deformation is detected, according to the corresponding multiple characteristic point comparison values of these groups of pictures
Update current correction parameter.This current correction parameter is to carry out image and entangle to each first image and corresponding each second image
Just.
In one embodiment of this invention, the step of whether above-mentioned detection reference picture group occurs image deformation includes
The following steps.Detect the second feature point of the fisrt feature point and the second reference picture of the first reference picture.Judge fisrt feature
Whether point and side-play amount of the second feature o'clock between the image coordinates of the first reference picture and the second reference picture exceed
Threshold value.If being judged as YES, judge that reference picture group occurs image deformation.
In one embodiment of this invention, the step of whether above-mentioned detection reference picture group occurs image deformation includes
The following steps.Three dimensional depth estimation is carried out according to the first reference picture and the second reference picture, to produce in reference picture group
Reference focus object reference depth information, and according to reference depth information obtain with regard to reference target thing depth focus
Position.Obtained by auto-focusing program with regard to reference target thing auto-focusing position.Judge with reference to focusing object
Whether corresponding depth focusing position meets auto-focusing position.If being judged as NO, judge that reference picture group occurs image
Deformation.
In one embodiment of this invention, it is above-mentioned according to corresponding these characteristic point comparison values of these groups of pictures more
Before the step of new current correction parameter, this method for correcting image also comprises the following steps.Three are carried out for these groups of pictures
Dimension depth estimation, to produce the depth information of each groups of pictures.Reservation figure is decided whether according to the depth information of each groups of pictures
As group.
In one embodiment of this invention, it is above-mentioned to be updated according to corresponding these characteristic point comparison values of these groups of pictures
The step of current correction parameter, also comprises the following steps.Feature point detection is carried out to these first images and these second images,
And obtain multiple fisrt feature points of these the first images and multiple second feature points of these the second images.Compare these first
The coordinate position of the coordinate position of characteristic point and multiple second feature points corresponding with these fisrt feature points respectively, to obtain
Take the multiple characteristic point comparison values between these fisrt feature points and these second feature points.Record these fisrt feature points and this
Multiple characteristic point comparison values between a little second feature points.
In one embodiment of this invention, the above-mentioned spy recorded between these fisrt feature points and these second feature points
The step of levying comparison value also comprises the following steps.According to the coordinate position and/or these second feature of these fisrt feature points
The coordinate position of point classifies each characteristic point comparison value to multiple statistics grooves.
In one embodiment of this invention, it is above-mentioned to update current according to corresponding these characteristic point comparison values of groups of pictures
The step of correction parameter, comprises the following steps.Compared with characteristic point according to the quantity of multiple characteristic point comparison values in these statistics grooves
The corresponding multiple depth values of value, judge whether these characteristic point comparison values carry out computing enough.If being judged as YES, according to these
Characteristic point comparison value updates current correction parameter.Wherein these depth values are by fisrt feature point and corresponding second feature
Point carries out three dimensional depth estimation and obtains.
From the point of view of another viewpoint, the present invention proposes a kind of image acquiring device, and this image acquiring device has the first image
Sensor and the second imageing sensor.This image acquiring device also includes memory element and processing unit.Unit records
Multiple modules, and storage is associated with the current correction parameter of the first imageing sensor and the second imageing sensor.Processing unit coupling
The first imageing sensor, the second imageing sensor and memory element are connect, described in accessing and perform
Module.These modules include acquisition module, shape changing detection module and parameter update module.Acquisition module is passed by the first image
Sensor and the second imageing sensor obtain multiple images group.Each groups of pictures includes the first of the first imageing sensor respectively
Second image of image and the second imageing sensor, and these groups of pictures include a reference picture group.Shape changing detection mould
Whether the first reference picture and the second reference picture in block detection reference picture group there is image deformation.When referring to group of pictures
When group occurs image deformation, parameter update module updates current school according to the corresponding multiple characteristic point comparison values of these groups of pictures
Positive parameter.Current correction parameter is to carry out image rectification to each first image and corresponding each second image.
Based on above-mentioned, in the embodiment of the image deformation bearing calibration of the present invention, when current correction parameter cannot
When carrying out accurate image rectification, using to the characteristic point information in the groups of pictures captured by different scenes and different time points
A data base is set up, current control information is updated with by information complete in data base.Even if consequently, it is possible to left and right figure
As sensor produces displacement, image acquiring device can also it is dynamic and it is adaptive produce new correction parameter, to avoid utilizing
The correction parameter for not being inconsistent present situation carries out inaccurate image rectification.Thereby, can automatically enter in the case where user is without discovering
The action that line parameter updates, to guarantee the shooting quality of image acquiring device and lift user experience.
It is that the features described above and advantage of the present invention can be become apparent, special embodiment below, and it is detailed to coordinate accompanying drawing to make
Carefully it is described as follows.
Description of the drawings
Fig. 1 is the block chart of the image acquiring device shown by one embodiment of the invention；
Fig. 2 is the flow chart of the image deformation bearing calibration shown by one embodiment of the invention；
Fig. 3 A to Fig. 3 B are the detail flowcharts of step S202 shown by one embodiment of the invention；
Fig. 4 is the flow chart of the image deformation bearing calibration shown by one embodiment of the invention；
Fig. 5 A to Fig. 5 B are that the characteristic of division point comparison value shown by one embodiment of the invention is illustrated to the enforcement for counting groove
It is intended to.
Description of reference numerals：
100：Image acquiring device；
110：First imageing sensor；
120：Second imageing sensor；
130：Focusing unit；
140：Processing unit；
150：Memory element；
151：Acquisition module；
152：Shape changing detection module；
153：Parameter update module；
154：Depth selecting module；
TH：Predetermined threshold value；
A、B：Fisrt feature point；
Z1～Z9：Image block；
S1～S9：Statistics groove；
ΔdA、ΔdB：Characteristic point comparison value；
S201～S203：Each step of the image deformation bearing calibration described in one embodiment of the invention；
S2011～S2023：The step of one embodiment of the invention S202 each sub-step；
S2024～S2027：The step of one embodiment of the invention S202 each sub-step；
S401～S409：Each step of the image deformation bearing calibration described in one embodiment of the invention.
Specific embodiment
When image acquiring device dispatches from the factory, between its twin-lens, space arranges relation through accurate calculating and tune
It is whole, and the correction parameter of one group of factory preset is produced according to this.The correction parameter of this factory preset is to by acquired in different camera lenses
Image rectification design to Jing and fixed coordinate parameters relationship.In order to solve to cause work because twin-lens produces displacement or rotation
The no longer applicable situation of the default correction parameter of factory, the depth information according to image of the invention produce record with pixel position
The data base of multiple characteristic point informations, and using the adaptive renewal correction parameter of the information accumulated in data base.In order that this
The content of invention becomes apparent from, and is exemplified below the embodiment that embodiment can actually be implemented accordingly as the present invention.
Fig. 1 is the block chart of the image acquiring device shown by one embodiment of the invention.Refer to Fig. 1, the present embodiment
Image acquiring device 100 be, for example, digital camera, digital camera, or other have image-acquisition functions hand-held electricity
Sub-device, seems smart mobile phone, panel computer etc., is not limited to above-mentioned.Image acquiring device 100 includes the first imageing sensor
110th, the second imageing sensor 120, focusing unit 130, processing unit 140 and memory element 150.
First imageing sensor 110 and the second imageing sensor 120 may include camera lens and photo-sensitive cell.Photo-sensitive cell example
Charge coupled cell in this way（Charge Coupled Device, abbreviation CCD）, Complimentary Metal-Oxide quasiconductor
（Complementary Metal-Oxide Semiconductor, abbreviation CMOS）Element or other elements, the first image sensing
Device 110 and the second imageing sensor 120 may also include aperture etc., and here is neither limited.Additionally, according to the first imageing sensor
110 and second imageing sensor 120 camera lens set location, the mirror of the first imageing sensor 110 and the second imageing sensor 120
Head can divide into left camera lens with right camera lens.
In the present embodiment, unit 130 of focusing couples the first imageing sensor 110, the second imageing sensor 120 and place
Reason unit 140, obtains the focal length of image to control the first imageing sensor 110 and the second imageing sensor 120.In other words,
Focusing unit 130 controls the camera lens of the camera lens and the second imageing sensor 120 of the first imageing sensor 110 and is moved to focusing position
Put.Focusing unit 130 for example passes through voice coil motor（Voice Coil Motor, abbreviation VCM）Or other different types of motors
To control the step number of camera lens（step）Position, to change the focal length of the first imageing sensor 110 and the second imageing sensor 120.
Processing unit 140 may, for example, be CPU（Central Processing Unit, abbreviation CPU）, it is micro-
Processor（Microprocessor）, ASIC（Application Specific Integrated
Circuits, abbreviation ASIC）, programmable logic device（Programmable Logic Device, abbreviation PLD）Or other tools
The hardware unit of standby operational capability.Memory element 150 is, for example, random access memory（random access memory）, it is fast
Fast memorizer（Flash）Or other memorizeies, to store current correction parameter and multiple modules, and 140 coupling of processing unit
Connect memory element 150 and to perform these modules.Above-mentioned module includes acquisition module 151, shape changing detection module 152, parameter
Update module 153 and depth selecting module 154, these modules are, for example, computer program, and which can be loaded into processing unit 140, from
And perform the function of correction image deformation.
Fig. 2 is the flow chart of the image deformation bearing calibration shown by one embodiment of the invention.The method of the present embodiment is fitted
For the image acquiring device 100 of Fig. 1, each component arranged in pairs or groups in image acquiring device 100 below illustrates the present embodiment image
The detailed step of deformation correction method.
First, in step S201, acquisition module 151 passes through the first imageing sensor 110 and the second imageing sensor 120
Obtain multiple images group.Each groups of pictures includes the first image and the second image respectively, and groups of pictures at least includes one
Reference picture group.That is, in the present embodiment, single image group has two photos, in same groups of pictures
First image and the second image are two images with right camera lens in the same time acquired in Same Scene by left camera lens.
In other words, the first image is e.g. by the left image acquired in left camera lens, and the second image is with respect to by being obtained by right camera lens
The right image for taking.In the present embodiment, the first image and the second image are, for example, instant preview figure acquired under preview state
Picture（live-view image）.
Similarly, reference picture group is one of groups of pictures acquired in image acquiring device 100, is therefore referred to
Groups of pictures equally has the first reference picture of correspondence to the first imageing sensor 110 and the second imageing sensor 120 and the
Two reference pictures.The first reference picture and the second ginseng in step S202, the detection reference picture of shape changing detection module 152 group
Examine whether image occurs image deformation.It should be noted that, shape changing detection module 152 can be in the way of timing be detected to part image
Group carries out the detection of image deformation, it is also possible to carry out the detection of image deformation for all groups of pictures, and reference picture group
Group represents shape changing detection module 152 to detect whether one of object for occurring that image-type becomes in this.
It should be noted that, the correction parameter of factory preset is suitable for two left images to be carried out image rectification again respectively
（image rectification）, allow two true pictures to become only horizontal aberration or only vertical aberration（Because camera lens position
Put the relation put and cause）.For example, narrow difference at the angled elevation angle of meeting etc. between twin-lens.By factory preset
Correction parameter performs image rectification, and it is to put same capture plane that true picture can be converted into left and right camera lens, only remaining water
Flat or upright position is variant.That is, on the premise of left and right camera lens is horizontally disposed, through the left images of image rectification
On each pixel should only remaining horizontal level it is variant.Now, if the shooting direction of left and right camera lens produces change, pass through
The upright position of each pixel on the left images of image rectification still has difference, and this phenomenon is referred to herein as image shape
Become.Here, shape changing detection module 152 can according to the first reference picture with the second reference picture mutual corresponding characteristic point it is inclined
Shifting amount or carrying out three dimensional depth for the first reference picture and the second reference picture estimates whether to judge reference picture group
Generation image deformation.
For becoming apparent from, Fig. 3 A are the detail flowcharts of step S202 shown by one embodiment of the invention.In Fig. 3 A institutes
In the embodiment shown, in step S2021, shape changing detection module 152 detects the fisrt feature point and second on the first reference picture
Second feature point on reference picture.Afterwards, in step S2022, shape changing detection module 152 judges that fisrt feature point is special with second
Whether the side-play amount levied a little respectively between the image coordinates of the first reference picture and the second reference picture exceedes threshold value.If the
One characteristic point and second feature point side-play amount respectively between the image coordinates of the first reference picture and the second reference picture surpasses
Threshold value is crossed, in step S2023, shape changing detection module 152 judges that this reference picture group occurs image deformation.That is,
By analyzing and counting the displacement information between fisrt feature point and second feature point, the first reference picture can be known whether accordingly
There is image deformation with the second reference picture.
In other words, shape changing detection module 152 can be according to the detection of the calculation method of existing feature point detection with reference to group figure
Any feature point of picture.The calculation method of feature point detection is to detect the more several characteristic points in image, e.g. edge
Detection（edge detection）, corner detection（Conner detection）Or other feature point detection algorithms, the present invention
This is not intended to limit.Afterwards, shape changing detection module 152 judges the seat mutually between corresponding fisrt feature point and second feature point
Whether cursor position side-play amount exceedes above-mentioned threshold value, and whether detection accordingly occurs image deformation with reference to group image.For example, shape
Become the vertical offset that detection module 152 can determine whether mutual corresponding fisrt feature point and second feature point（In vertical direction
Displacement difference away from）Whether above-mentioned threshold value is exceeded.When shape changing detection module 152 judges that above-mentioned vertical offset exceedes above-mentioned threshold value
When, represent reference picture group and image deformation occurs.
In another embodiment, Fig. 3 B are the detail flowcharts of step S202 shown by one embodiment of the invention.In Fig. 3 B
In shown embodiment, in step S2024, shape changing detection module 152 is carried out with the second reference picture according to the first reference picture
Three dimensional depth is estimated, to produce the reference depth information of the focusing object of the reference in reference picture group and deep according to reference
Degree information obtains the depth focusing position with regard to reference target thing.Then, in step S2025, shape changing detection module 152 is by certainly
Dynamic focusing program and obtain with regard to reference target thing auto-focusing position.In step S2026, shape changing detection module 152 is sentenced
Whether the disconnected depth focusing position with reference to corresponding to focusing object meets auto-focusing position.If being judged as NO, in step
S2027, shape changing detection module 152 judge that reference picture group occurs image deformation.
Specifically, shape changing detection module 152 can carry out image procossing by stereovision technique, exist in the hope of object
The depth information of each point in three-dimensional coordinate position and image in space.Furthermore, obtained with regard to object according to depth information
Depth focusing position the step of be, for example, obtaining the focusing position with regard to object according to depth information query depth synopsis
Put.Therefore, by trying to achieve the corresponding relation of current value and the object definition of the step number or voice coil motor of stepper motor in advance,
Then can be according to the step number of the stepper motor corresponding to the depth information of currently acquired object inquires this depth information or sound
The current value of coil motor, and the depth focusing position with regard to object is obtained accordingly.
On the other hand, the process for performing auto-focusing program can automatically control camera lens module by unit 130 of focusing
Moved on a large scale, to adjust the camera lens of the first imageing sensor 110 and the second imageing sensor 120 respectively to required
Focusing position, to obtain the auto-focusing position with regard to object.Focusing unit 130 is e.g. using institute in Autofocus Technology
The climbing method for using（hill-climbing）To obtain the auto-focusing position with regard to object, but the present invention not as
Limit.Therefore, under conditions of the first reference picture and the second reference picture do not occur image deformation, image acquiring device 100 can
Preferable depth information is obtained, causes the depth focusing position can be with auto-focusing position consistency.If 100 nothing of image acquiring device
Method further obtains preferable depth information according to current correction parameter, also just cannot pass through depth information and be previously stored
Depth information query depth synopsis estimate correct depth focusing position, therefore depth focusing position with by automatically right
The auto-focusing position obtained by burnt program will produce difference.Accordingly, shape changing detection module 152 according to depth focusing position with from
Difference between dynamic focusing position is judging whether reference picture group occurs image deformation.
Referring again to Fig. 2, when shape changing detection module 152 detects reference picture group, and image deformation occurs, in step
S203, parameter update module 153 update current correction parameter according to the corresponding multiple characteristic point comparison values of multiple images group, its
Middle current correction parameter is to carry out image rectification to each first image and corresponding each second image.That is, when figure
As acquisition device 100 judges that the first imageing sensor 110 and the second imageing sensor 120 are deformed or shift and cause first
When parameter coordinate between reference picture and the second reference picture changes, representing current correction parameter cannot carry out standard to image
True image rectification.
Therefore, in one embodiment, parameter update module 153 starts to collect many captured by after reference picture group
The characteristic point comparison value of individual groups of pictures, with by being deformed with the second imageing sensor 120 in the first imageing sensor 110
Or acquired image comes in produce preferable current correction parameter after displacement.Specifically, parameter update module 153
The seat of the coordinate position and second feature point corresponding with these fisrt feature points respectively of comparison fisrt feature point can be passed through
Cursor position is obtaining these characteristic point comparison values.Furthermore, parameter update module 153 can also be according to the depth information of image and pixel
The coordinate position of point is producing new current correction parameter.Another embodiment will below be enumerated to describe it in detail.
Fig. 4 is a kind of flow chart of the image deformation bearing calibration shown by one embodiment of the invention.Fig. 4 is refer to, this
Image acquiring device 100 of the method for embodiment suitable for Fig. 1, each component arranged in pairs or groups in image acquiring device 100 below are said
The detailed step of bright the present embodiment image deformation bearing calibration.
First, in step S401, acquisition module 151 passes through the first imageing sensor 110 and the second imageing sensor 120
Obtain multiple images group.Each groups of pictures is included with reference to figure with one second image, and groups of pictures including the first image respectively
As group.The first reference picture and second in step S402, the detection reference picture of shape changing detection module 152 group is with reference to figure
It seem no generation image deformation.Step S401 and step S402 are similar to S201 the step of previous embodiment and step S202
Or it is identical, will not be described here.
If shape changing detection module judges that reference picture group occurs image deformation, in step S403, depth selecting module
154 carry out three dimensional depth estimation for groups of pictures, to produce the depth information of each groups of pictures, and according to each groups of pictures
Depth information decides whether to retain groups of pictures.Furthermore, it is understood that depth selecting module 154 can pass through at the image of stereoscopic vision
Reason technology and produce the three-dimensional depth map for being associated with the first reference picture and the second reference picture.Based on the depth in three-dimensional depth map
Degree information, depth selecting module 154 can obtain the field depth corresponding to above-mentioned three-dimensional depth map, and according to field depth determining
It is fixed to retain or abandon groups of pictures.
Specifically, it is assumed that the minima of depth value is set as 0 and maximum is set as the depth of 128, i.e. groups of pictures
Value is fallen within 0～128 numerical range.If depth selecting module 154 have been gathered by field depth for depth value 100 to depth value
128 groups of pictures, will not retain other field depths after depth selecting module 154 and fall within depth value 100 to depth value 128
Interior groups of pictures.If conversely, depth selecting module 154 judges that the field depth of current groups of pictures falls within depth value 100
To outside depth value 128, the e.g. groups of pictures of depth value 0 to depth value 80, depth selecting module 154 will retain this image
Group, with the characteristic point information further with this groups of pictures.
In other words, whether depth selecting module 154 judges each groups of pictures according to the depth of view information of each groups of pictures
For effective groups of pictures.If the field depth of the image module of newest acquisition depth of field model with previous groups of pictures
Most overlap is with, depth selecting module 154 will filter it accordingly.Base this, in one embodiment, except by judging scape
Reservation that whether deep scope overlaps to carry out groups of pictures with filter outside, depth selecting module 154 is also dependent on field depth
Duplication come decide whether retain groups of pictures.Base this, it can be ensured that depth selecting module 154 collected correspondence to all or big
The information of part field depth, and while field depth according to corresponding to each groups of pictures is filtering unnecessary information, to drop
Low data processing amount simultaneously lifts data processing speed.
Afterwards, in step S404,153 pairs of the first images of parameter update module and the second image carry out feature point detection, and
Obtain multiple second feature points of multiple fisrt feature points and the second image of the first image.In step S405, parameter updates mould
Block 153 compares the coordinate position of the coordinate position and second feature point corresponding with fisrt feature point respectively of fisrt feature point
Put, to obtain the characteristic point comparison value between fisrt feature point and second feature point.In step S406, parameter update module 153
Characteristic point comparison value between record fisrt feature point and second feature point.
Furthermore, it is understood that parameter update module 153 equally can be each according to the detection of the calculation method of existing feature point detection
The characteristic point of the first image and the second image in groups of pictures, with the fisrt feature point and the second image on the first image for taking
Second feature point.Then, parameter update module 153 judges the fisrt feature point being mutually matched with second feature point in same seat
Side-play amount under mark system（offset）And using side-play amount as characteristic point comparison value.Wherein, the fisrt feature point being mutually matched
With the same position in second feature spot projection to scene being shot.In other words, characteristic point comparison value also can be considered fisrt feature point
With the aberration between second feature point.Afterwards, parameter update module 153 is by the feature between fisrt feature point and second feature point
Point comparison value is recorded to a data base, to set up the correction database to update current correction parameter.It is noted that working as
When judging that reference picture group deforms upon, acquisition module 151 still persistently obtains image and obtains multiple images group, and parameter
Update module 153 is also persistently recorded the characteristic point comparison value of each groups of pictures produced by calculating to correction database.
In step S407, seat of the parameter update module 153 according to the coordinate position and/or second feature point of fisrt feature point
Cursor position classifies each characteristic point comparison value to multiple statistics grooves.That is, parameter update module 153 is except characteristic point is compared
Value record is classified to corresponding to the coordinate position under a coordinate system always according to characteristic point comparison value to outside correction database
Different statistics grooves.Specifically, in one embodiment, parameter update module 153 can be by acquired in the first imageing sensor 110
First image divides into multiple images block, and each image block correspondence to a statistics groove.Therefore according to fisrt feature point
Coordinate position, parameter update module 153 one of can sequentially correspond to characteristic point comparison value to these image blocks, so as to
Features described above point comparison value is classified to into corresponding statistics groove.
For example, Fig. 5 A and Fig. 5 B are the characteristic of division point comparison value extremely statistics grooves shown by one embodiment of the invention
Embodiment schematic diagram.In the present embodiment, with reference first to Fig. 5 A, the first image Img1 is divided into three by parameter update module 153
Take advantage of three 9 image block Z1～Z9.More according to the coordinate position of fisrt feature point, parameter update module 153 determines that first is special
Levy.As shown in Figure 5A, parameter update module 153 can be according to the coordinate position of fisrt feature point A
Learn that fisrt feature point A is located in image block Z2.Similarly, parameter update module 153 can be according to the coordinate of fisrt feature point B
Position and learn fisrt feature point B be located at image block Z6 in.
Fig. 5 B are refer to, in the present embodiment, the first image Img1 is divided into 9 image block Z1～Z9, and image
Block Z1～Z9 is respectively corresponding to count groove S1～S9.Wherein, image block Z1 correspondence extremely statistics groove S1, and image block Z2 pair
Should be to statistics groove S2, the rest may be inferred.Base this, due to fisrt feature point A be located at image block Z2 in, corresponding to fisrt feature point A
Characteristic point comparison value Δ dAIt is classified to statistics groove S2.As fisrt feature point B is located in image block Z6, and fisrt feature
Characteristic point comparison value Δ d corresponding to point BBIt is classified to statistics groove S6.It should be noted that, Fig. 5 A and Fig. 5 B are only a kind of demonstration
Property embodiment, it is non-limiting the present invention.
In addition, in one embodiment, it is same in scene being shot based on fisrt feature point and corresponding second feature point
Location point, the first image will be projected to three-dimensional coordinate system after calculating via coordinates translation with corresponding characteristic point on the second image
Under identical coordinate points.Therefore, parameter update module 153 can be projected on three-dimensional seat according to fisrt feature point and second feature point
Characteristic point comparison value is classified to corresponding statistics groove by the coordinate position under mark system.For further, based on stereopsis
Feel technology carries out image procossing, and parameter update module 153 can try to achieve the depth information of each point in image and corresponding three-dimensional seat
Cursor position.Parameter update module 153 can be by the horizontal component of the tripleplane's point corresponding to fisrt feature point and second feature point
The statistics groove corresponding to characteristic point comparison value is determined with vertical component.
Then, in step S408, parameter update module 153 is according to the quantity of characteristic point comparison value and spy in each statistics groove
The multiple depth values corresponding to a comparison value are levied, whether judging characteristic point comparison value carries out computing enough.Wherein depth value is logical
Crossing carries out three dimensional depth estimation with corresponding second feature point and obtains to fisrt feature point.It is appreciated that with image
The rising of the quantity of group, the quantity of information for being recorded in correction database are also more and more.Here, parameter update module 153
Current record can be judged in school with the depth value of characteristic point comparison value according to the quantity of characteristic point comparison value in each statistics groove
Whether the data volume in correction data storehouse is enough.
Understood based on aforementioned, these characteristic point comparison values are classified according to characteristic point information by parameter update module 153
To corresponding statistics groove.Therefore, parameter update module 153 can be judged according to the total number of characteristic point comparison value in each statistics groove
Whether correction database possesses enough data volumes.Specifically, in order to produce meet instantly camera lens setting situation with by left and right
Image is become a full member to the current correction parameter of perfect condition, there is provided what the characteristic point of characteristic point information preferably can be average is distributed in
On image.The characteristic point information provided by the characteristic point for being evenly distributed in each region on image, more accurately can be calculated
The rotation amount or skew condition of whole image.
In the present embodiment, these characteristic point comparison values by parameter update module 153 according to the coordinate position of characteristic point
Corresponding statistics groove is classified to, therefore the sum of each characteristic point comparison value corresponding to statistics groove can represent the sky of characteristic point
Between distribution situation.Therefore, parameter update module 153 judges whether the quantity of the characteristic point comparison value in each statistics groove is enough,
It is used as whether determination data amount be enough to calculate the decision mechanism of accurate current correction parameter.
Illustrate by taking Fig. 5 B as an example, characteristic point comparison value is classified to 9 statistics groove S1～S9 by parameter update module 153
In, wherein statistics groove S1 at least includes that characteristic point comparison value Δ dA, statistics groove S6 at least include characteristic point comparison value Δ dB.DANGSHEN
Number update module 153 persistently records each characteristic point comparison value of different images group when each statistics groove, in each statistics groove
Characteristic point comparison value is also continued to build up（As shown in the dotted line of Fig. 5 B）.Once respectively count the characteristic point comparison value foot in groove S1～S9
Enough, parameter update module 153 can proceed by the update action of current correction parameter.For example, parameter update module 153 can
Judge whether each statistics groove S1～corresponding numbers of S9 exceed predetermined threshold value TH and determine the data volume in correction database
It is whether enough.So, above-described embodiment is only a kind of exemplary embodiment, is not limited to the present invention.In this technical field
Tool usually intellectual judges data with according to statistics groove when the mode classification that can select feature comparison value according to the actual requirements
Amount whether enough Rule of judgment, here is omitted.
In addition, parameter update module 153 also can judge the data volume in current correction database by depth information
It is whether enough.Specifically, this can be obtained by three dimensional depth estimation is carried out to fisrt feature point and corresponding second feature point
The each self-corresponding depth value of a little characteristic point comparison value（depth value）.Understood based on aforementioned, depth selecting module 154 will
The field depth excessive groups of pictures of repeatability is filtered out, and parameter update module 153 will determine whether to have been for it is most
Depth value collected corresponding characteristic point comparison value.In other words, in one embodiment, parameter update module 153 is also according to spy
The each self-corresponding depth value of a comparison value is levied come each characteristic point comparison value of classifying, and judges the feature corresponding to each depth value
Whether enough the number of point comparison value, and judges whether the data volume in correction database is enough.
If it is noted that during correction database is set up, the first imageing sensor 110 and the second image
When space setting relation between sensor changes again, represent the recorded data in correction database and cannot use.
In one embodiment, parameter update module 153 can also judge this characteristic point in recording feature point comparison value to before correction database
Whether comparison value has quite poor different with the data in correction database.If so, parameter update module 153 previously can have been remembered
The data of record are abandoned, and begin setting up again another correction database, therefore and obtain optimal current correction parameter.
That is, once parameter update module 153 judges data volume in correction database enough, parameter update module
153 just stop recording can start to calculate new current correction parameter.Conversely, parameter update module 153 persistently records new spy
A comparison value is levied to correction database.Therefore, if step S408 is judged as YES, in step S409,153 basis of parameter update module
Characteristic point comparison value updates current correction parameter.Parameter update module 153 is for example using the feature comparison value in correction database
Optimal current correction parameter is searched out to carry out optimization algorithm, two images Jing after current correction parameter correction are caused
Can correspond to preferable parameter coordinate relation.Optimization algorithm is, for example, gradient descent method（gradient decent
method）, Lai Wenbeige-horse quart method（Levenberg-Marquardt method, abbreviation LM method）Or Gauss cattle
Algorithm（Gauss-Newton method）Deng of the invention that this is not limited.
In sum, in one embodiment of this invention, deform upon when image acquiring device timely detects image
When, the current correction parameter for being pre-stored in image acquiring device is adaptively corrected by the foundation of correction database, by a left side
Right image is corrected to preferable parameter coordinate relation.Consequently, it is possible to current correction can be carried out in the case where user is without discovering
The adjustment of parameter, to guarantee the shooting quality of image acquiring device.Furthermore, embodiments of the invention further can be believed according to depth
With characteristic point position, breath judges whether the characteristic point information in data base is collected completely.Thereby, once collecting enough numbers
According to amount, can be immediately by the characteristic point comparison value that recorded in correction database producing new current correction parameter, so as to
Significantly shorten gather data and the required time being corrected.
Finally it should be noted that：Various embodiments above only to illustrate technical scheme, rather than a limitation；To the greatest extent
Pipe has been described in detail to the present invention with reference to foregoing embodiments, it will be understood by those within the art that：Its according to
So the technical scheme described in foregoing embodiments can be modified, or which part or all technical characteristic are entered
Row equivalent；And these modifications or replacement, do not make the essence of appropriate technical solution depart from various embodiments of the present invention technology
The scope of scheme.
Claims (14)
1. a kind of image deformation bearing calibration, it is adaptable to which the image with the first imageing sensor and the second imageing sensor is obtained
Device, the wherein image acquiring device have the current correction for being associated with first imageing sensor and second imageing sensor
Parameter, it is characterised in that the image deformation bearing calibration includes：
The multiple images group captured by different time points is obtained by first imageing sensor and second imageing sensor
Group, wherein each described image group includes the first image and the second image respectively, described image group includes reference picture group
Group；
Detect whether the first reference picture and the second reference picture in the reference picture group occur image deformation；And
When detecting the reference picture group image deformation occurring, according to the corresponding multiple characteristic point ratios of described image group
Update the current correction parameter to value, wherein the current correction parameter is to each described first image and corresponding each described
Second image carries out image rectification.
2. image deformation bearing calibration according to claim 1, it is characterised in that detect whether the reference picture group is sent out
The step of raw image deformation, includes：
Detect the fisrt feature point of first reference picture and the second feature point of second reference picture；And
Judge the fisrt feature point with the second feature point respectively in first reference picture and the image of second reference picture
Whether the side-play amount between coordinate exceedes threshold value, if so, judges that the reference picture group occurs the image deformation.
3. image deformation bearing calibration according to claim 1, it is characterised in that detect whether the reference picture group is sent out
The step of raw image deformation, includes：
Three dimensional depth estimation is carried out according to first reference picture and second reference picture, to produce in the reference picture group
Reference focus the reference depth information of object, and the depth with regard to the reference target thing is obtained according to the reference depth information
Focusing position；
Obtained by auto-focusing program with regard to the reference target thing auto-focusing position；And
Judge whether the depth focusing position corresponding to reference focusing object meets the auto-focusing position, if it is not, judging
There is the image deformation in the reference picture group.
4. image deformation bearing calibration according to claim 1, it is characterised in that corresponding according to described image group
Before the step of characteristic point comparison value updates the current correction parameter, the method for correcting image also includes：
Three dimensional depth estimation is carried out for described image group, to produce the depth information of each described image group；And
Decide whether to retain described image group according to the depth information of each described image group.
5. image deformation bearing calibration according to claim 1, it is characterised in that according to corresponding institute of described image group
Stating the step of characteristic point comparison value updates the current correction parameter also includes：
Feature point detection is carried out to described first image and second image, and obtain described first image multiple first are special
Levy the multiple second feature points a little with second image；
Compare the coordinate position and the second feature corresponding with the fisrt feature point respectively of the fisrt feature point
The coordinate position of point, to obtain the characteristic point comparison value between the fisrt feature point and second feature point；And
Record the characteristic point comparison value between the fisrt feature point and second feature point.
6. image deformation bearing calibration according to claim 5, it is characterised in that record the fisrt feature point with it is described
The step of characteristic point comparison value between second feature point, also includes：
According to each characteristic point of coordinate position classification of the coordinate position and/or second feature point of the fisrt feature point
Comparison value is to multiple statistics grooves.
7. image deformation bearing calibration according to claim 6, it is characterised in that according to corresponding institute of described image group
Stating the step of characteristic point comparison value updates the current correction parameter includes：
Multiple depth according to corresponding to the quantity for counting characteristic point comparison value described in groove is with the characteristic point comparison value
Value, judges whether the characteristic point comparison value carries out computing enough, if so, updates the current school according to the characteristic point comparison value
Positive parameter, wherein the depth value is by carrying out three dimensional depth with the corresponding second feature point to the fisrt feature point
Estimate and obtain.
8. a kind of image acquiring device, with the first imageing sensor and the second imageing sensor, it is characterised in that the image is obtained
Taking device includes：
Memory element, records multiple modules, and storage is associated with working as first imageing sensor and second imageing sensor
Front correction parameter；And
Processing unit, couples first imageing sensor, second imageing sensor and memory element, to access and perform this
The module recorded in memory element, the module include：
Acquisition module, obtains multiple captured by different time points by first imageing sensor and second imageing sensor
Groups of pictures, each described image group the first image and second imageing sensor respectively including first imageing sensor
The second image, and described image group includes reference picture group；
Shape changing detection module, detects whether the first reference picture and the second reference picture in the reference picture group occur image
Deformation；
Parameter update module, when the reference picture group occurs the image deformation, the parameter update module is according to described image
The corresponding multiple characteristic point comparison values of group update the current correction parameter, and wherein the current correction parameter is to each described
One image and corresponding each second image carry out image rectification.
9. image acquiring device according to claim 8, it is characterised in that the shape changing detection module detects first reference
The fisrt feature point of image and the second feature point of second reference picture, the shape changing detection module judge the fisrt feature point with
Whether side-play amount of the second feature point respectively between first reference picture and the image coordinates of second reference picture surpasses
Threshold value is crossed, if so, the shape changing detection module judges that the reference picture group occurs the image deformation.
10. image acquiring device according to claim 8, it is characterised in that the shape changing detection module is according to first ginseng
Examining image carries out three dimensional depth estimation with second reference picture, to produce the focusing object of the reference in the reference picture group
Reference depth information, and the depth focusing position with regard to the reference target thing, the deformation are obtained according to the reference depth information
Detection module obtained by auto-focusing program with regard to the reference target thing auto-focusing position, and the shape changing detection mould
Block judges whether the depth focusing position corresponding to reference focusing object meets the auto-focusing position, if it is not, the deformation
Detection module judges that the reference picture group occurs the image deformation.
11. image acquiring devices according to claim 8, it is characterised in that the module also includes：
Depth selecting module, the depth selecting module carry out three dimensional depth estimation for described image group, each described to produce
The depth information of groups of pictures, and the depth selecting module decides whether reservation according to the depth information of each described image group
Described image group.
12. image acquiring devices according to claim 8, it is characterised in that the parameter update module is to first figure
As feature point detection being carried out with second image, and obtain multiple fisrt feature points and second figure of described first image
Multiple second feature points of picture, the parameter update module is to the coordinate position of the fisrt feature point and respectively with described first
The coordinate position of the corresponding second feature point of characteristic point, with obtain the fisrt feature point and the second feature point it
Between the characteristic point comparison value, the parameter update module records the institute between the fisrt feature point and second feature point
State characteristic point comparison value.
13. image acquiring devices according to claim 12, it is characterised in that the parameter update module is according to described first
The coordinate position of the coordinate position of characteristic point and/or second feature point classifies each characteristic point comparison value to multiple statistics
Groove.
14. image acquiring devices according to claim 13, it is characterised in that the parameter update module is according to the statistics
The quantity of characteristic point comparison value described in groove and the multiple depth values corresponding to the characteristic point comparison value, judge the characteristic point
Whether comparison value carries out computing enough, and if so, the parameter update module updates the current correction according to the characteristic point comparison value
Parameter, wherein the depth value is estimated by carrying out three dimensional depth with the corresponding second feature point to the fisrt feature point
Survey and obtain.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410044039.3A CN104811680B (en) | 2014-01-28 | 2014-01-28 | image obtaining device and image deformation correction method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410044039.3A CN104811680B (en) | 2014-01-28 | 2014-01-28 | image obtaining device and image deformation correction method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104811680A CN104811680A (en) | 2015-07-29 |
| CN104811680B true CN104811680B (en) | 2017-04-12 |
Family
ID=53696135
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410044039.3A Active CN104811680B (en) | 2014-01-28 | 2014-01-28 | image obtaining device and image deformation correction method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104811680B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106817575A (en) * | 2015-11-30 | 2017-06-09 | 聚晶半导体股份有限公司 | The method of the method and automatic corrigendum of image capturing device and its generation depth information |
| US20190387165A1 (en) * | 2018-06-07 | 2019-12-19 | Eys3D Microelectronics, Co. | Image device for generating depth images and related electronic device |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI423143B (en) * | 2010-06-17 | 2014-01-11 | Pixart Imaging Inc | Image sensing module |
| CN102313515B (en) * | 2010-07-08 | 2013-10-23 | 国立清华大学 | Three-dimensional digital image correlation (3D-DIC) correcting method |
| DE102010062496B4 (en) * | 2010-12-07 | 2022-01-20 | Robert Bosch Gmbh | Method and device for processing image information from two sensors of a stereo sensor system suitable for image acquisition |
| TW201310004A (en) * | 2011-08-18 | 2013-03-01 | Nat Applied Res Laboratories | Correlation arrangement device of digital images |
-
2014
- 2014-01-28 CN CN201410044039.3A patent/CN104811680B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN104811680A (en) | 2015-07-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| TWI511081B (en) | Image capturing device and method for calibrating image deformation thereof | |
| US9697604B2 (en) | Image capturing device and method for detecting image deformation thereof | |
| CN107087107B (en) | Image processing apparatus and method based on dual camera | |
| KR102032882B1 (en) | Autofocus method, device and electronic apparatus | |
| US8233078B2 (en) | Auto focus speed enhancement using object recognition and resolution | |
| CN103197491B (en) | The method of fast automatic focusing and image collecting device | |
| US20160295097A1 (en) | Dual camera autofocus | |
| CN104363378B (en) | camera focusing method, device and terminal | |
| JP2017112602A (en) | Image calibrating, stitching and depth rebuilding method of panoramic fish-eye camera and system thereof | |
| CN104363377B (en) | Display methods, device and the terminal of focus frame | |
| CN104811608B (en) | Image capturing apparatus and image defect correction method thereof | |
| TWI471677B (en) | Auto focus method and auto focus apparatus | |
| CN109922251A (en) | The method, apparatus and system quickly captured | |
| CN103246044A (en) | Automatic focusing method, automatic focusing system, and camera and camcorder provided with automatic focusing system | |
| CN101221341A (en) | Initialization method for field depth composition | |
| US9747694B2 (en) | Method and apparatus for optimizing depth information | |
| CN106454090A (en) | Automatic focusing method and system based on depth camera | |
| CN104102068A (en) | Automatic focusing method and automatic focusing device | |
| CN108710192B (en) | Automatic focusing system and method based on statistical data | |
| CN108510540A (en) | Stereoscopic vision video camera and its height acquisition methods | |
| CN108020200A (en) | A kind of depth measurement method and system | |
| CN104811680B (en) | image obtaining device and image deformation correction method thereof | |
| CN106817575A (en) | The method of the method and automatic corrigendum of image capturing device and its generation depth information | |
| CN104811688B (en) | Image acquiring device and its image deformation detection method | |
| TW201443539A (en) | Auto focus method and auto focus apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |