CN103369247A - Camera unified focal length calibration method applied to multi-camera visual information processing - Google Patents

Abstract

The invention relates to multi-camera visual information processing, and provides a unified focal length calibration method for cameras in a multi-camera image processing system. In order to provide a more efficient solution of multi-image-source image processing for current various multimedia devices, according to the technical scheme, the camera unified focal length calibration method applied to multi-camera visual information processing comprises the following steps: 1) solving the image definition of a single camera, 2) solving a relation curve between the image definition of the single camera and a focal length, 3) solving an extreme value of a function of the image definition of the single camera, and 4) solving a unified focal length value of the multiple cameras in the system. The unified focal length calibration method is mainly applied to the visual information processing.

Description

The camera that is applied to the multi-cam Vision information processing is unified the focal length scaling method

Technical field

The present invention is a kind of method of fast and effectively multi-cam Vision information processing system being carried out the unified demarcation of camera, namely relates to the camera that is applied to the multi-cam Vision information processing and unifies the focal length scaling method.

Technical background

In contemporary image processing techniques, fast development along with digital image processing techniques, how utilizing computer that camera is carried out automatic calibration and sets is one of key technology in the multi-cam image processing system, it is some other Digital Image Processing, splice such as image intelligent, the prerequisite of panoramic video, intelligent video monitoring etc. and basis, how to obtain the multiframe depth of field and the angle of visual field all consistent image for further image processing and whole system, all be vital.

The automatic focusing of camera is by gathering multiple image, then each two field picture being carried out the assessment of image definition, as utilize gray variance method, histogram method, the methods such as Laplacian and method.By setting up the function between image definition and the camera focus value, to the function maximizing, just can obtain the pinpointed focus value of this camera.

And for the multi-cam image processing system, particularly with camera row image processing system in the same horizontal line, it often needs to gather simultaneously the image of all cameras, then this multiple image is further processed, for example, Image Mosaics, image co-registration etc.At this moment, if the focal length of each camera is not identical, the depth of field of their images of sampling and field, visual angle difference to some extent just so, thus carrying out just need to carrying out the quality that convergent-divergent etc. is processed the fusions of assurance Image Mosaics to such image before further image is processed.And on the other hand, the pinpointed focus value of each camera is slightly devious usually, in this case, just need to find out a unified focal length value, so that all cameras namely reach the requirement of image definition, guarantee again the consistent of focal length value, provide more reliable original image thereby process for further image.Focus by the unification to multi-cam, can greatly reduce the complexity that image is processed, improve the efficient of system.

Summary of the invention

For overcoming the deficiencies in the prior art, the present invention aims to provide a kind of method of camera in the multi-cam image processing system being unified the focal length demarcation, the solution that provides a kind of image of more efficient many image sources to process for diversified multimedia equipment now, for this reason, the technical solution used in the present invention is, the camera that is applied to the multi-cam Vision information processing is unified the focal length scaling method, comprises the steps:

1) finds the solution the image definition of single camera: utilize the evaluation function of image definition to find the solution the image definition of each two field picture of camera;

2) find the solution relation curve between single camera image definition and the focal length: by the image definition analysis under the different focal being obtained the relation curve of image definition and camera focal length, by the figure line match, find out the functional relation of image definition and focal length;

3) find the solution the extreme value of single camera image sharpness function: for each camera, can according to acquired relation curve, adopt data fitting method to provide its image definition y _iAnalytical function y with focus value x _i=f _i(x), then adopt the method for optimizing to provide extreme value

With its corresponding focal length

4) the unified focal length value of a plurality of cameras in the solving system: after obtaining all camera image sharpness functions, just can set up the multiple target valuation functions vector F (x) of system, utilize the ideal point method in the valuation functions method, multiple objective function is optimized, seek unique extreme point, be the pinpointed focus value, this is the unified focal length set point of multi-cam image processing system.

The unified focal length value of a plurality of cameras is divided into two kinds of situations in the solving system in camera adjustable focal length value scope D:

The first situation, the image definition function f of M camera _i(x) just in time have identical extreme point, i.e. x ₁*=x ₂*=...=x _N*, x here _NCorresponding focal length value when * corresponding N camera image sharpness function got extreme value, for x ∈ D, its image definition function has

$f_i\left(x_i^{*}\right)\≥f_i\left(x\right),i=\mathrm{0,1},\·\·\·,N$

Formula (1)

And simultaneously, this extreme point also just in time drops in the focal range of system's permission, and the corresponding focal length value of this extreme point is system's pinpointed focus value so;

The second situation, the image definition function f of M camera _i(x) have different extreme points, and at x ₁*, x ₂* ..., x _NWhen * not exclusively equating, the ideal point F* of objective function F (x) not necessarily drops in the image set F (D), introduces the p norm this moment in object space Rm

$u\left(F\right)={\left|\right|F-F^{*}\left|\right|}_p={\left[\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{(f_i-f_i^{*})}^p\right]}^{\frac{1}{p}}$

Formula (2)

As evaluation function, wherein 1≤p＜+∞.

Utilizing the evaluation function of image definition to find the solution the image definition of each two field picture of camera, is to adopt average gray gradient operator method:

$\begin{array}{c}\mathrm{GMG}=\\ \frac{\underset{i=1}{\overset{M-1}{\mathrm{\Σ}}}\underset{j=1}{\overset{N-1}{\mathrm{\Σ}}}\sqrt{\frac{{[g(i+1,j)-g(i,j)]}^2+{[g(i,j+1)-g(i,j)]}^2}{2}}}{(M-1)(N-1)}\end{array}$ Formula (3)

As shown in formula (3), be averaging by the mean square deviation to gray value between cumulative image pixel and the neighbor, can obtain the image definition value of current frame image, utilize between the average gray gradient operator value of every two field picture and the focal length value one to one relation, set up the image definition of single camera and the graph of relation of focal length, g (i.j) represents (i.j) individual grey scale pixel value, and i.j represents that respectively pixel gets level, vertical arrangement sequence number.

Behind the image definition curve that has obtained single shooting picture head, need to adopt the least square data fitting method to provide its image definition y _iAnalytical function y with focus value x _i=f _i(x), single camera is a quadratic function adopting average gray gradient operator method to obtain the image definition curve

f _i(x)=Ax ²+Bx+C

Formula (4)

Here, A, B, C are the coefficients that data fitting is tried to achieve.

Adopt the method for optimizing to provide extreme value

With its corresponding focal length

Specifically adopt the simplest derivative method.

Step 4) further is specially: the image definition function of M camera can form a Multi Goal Opinion Function vector F (x)

F (x)=(f ₁(x), f ₂(x) ..., f _N(x)) formula (5)

For this target function, adopt in the optimal method and find the solution its unique optimization extreme point for the valuation functions method of multiple-objection optimization, here, employing be ideal point method:

Setting a focal length value the most desirable is x,

Be function f _i(x) maximum is then utilized the beeline ideal point method, and namely norm p=2 finds the solution multiple objective function, finds out its maximum:

$\underset{x\∈D}{\max }\sqrt{\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{[f_i\left(x\right)-f_i^{*}]}^2}$

Formula (6)

Wherein, x is the focal length value of camera, and its size should be limited in the camera adjustable focal length value scope D, utilizes formula (6) to obtain unique extreme point F of target function ^*With and corresponding x ^*, this x ^*It is exactly the unified focal length value of multi-cam image processing system.

The present invention possesses following technique effect:

Focusing method with traditional camera compares, the present invention can carry out the image definition sampling of each camera more rapidly, set up the image definition function, find out more exactly the unified optimum focal length value of all cameras, thereby can unify to focus to a plurality of cameras more quickly and accurately, so that the image in later stage be processed more Simple fast.

And algorithm of the present invention also has fabulous extensibility and applicability, can be applied to the various image processing systems that have a plurality of cameras.

Description of drawings

The image sampling of each camera of Fig. 1 and the foundation of articulation curve.

Fig. 2 has the multiple target valuation functions schematic diagram of unique extreme point.

Fig. 3 is without the multiple target valuation functions schematic diagram of unique extreme point.

The unification of Fig. 4 multi-cam focuses the flow chart of method.

Embodiment

The present invention analyzes all cameras in the system in the definition under the different focal, utilizing optimal method to find out the unified focal length of all cameras sets, thereby the angle of visual field and the depth of field that the visual information that guarantees the multichannel camera collection is consistent, the difficulty of the Digital Image Processing in reduction later stage.

Among the present invention, proposed the camera in the multi-cam image processing system is found the solution the image definition function, utilized the method for multiobjective optimization to find out a pinpointed focus value, thus the method that all cameras are unified to focus.Utilization is a curve with the image definition of all cameras and the transformation of focal length, then seeks optimum solution in many curves, thereby camera is carried out unified setting, has reduced the difficulty that further image is processed.This technological invention mainly is divided into following components:

1. find the solution the image definition of single camera

In the present invention, be to utilize the evaluation function of image definition to find the solution the image definition of each two field picture of camera.

2. find the solution the relation curve between single camera image definition and the focal length

By the image definition analysis under the different focal being obtained the relation curve of image definition and camera focal length, by the figure line match, find out the functional relation of image definition and focal length.

3. find the solution the extreme value of single camera image sharpness function

For each camera, can according to acquired relation curve, adopt data fitting method to provide its image definition y _iAnalytical function y with focus value x _i=f _i(x), then can adopt the method for optimization to provide extreme value

With its corresponding focal length

4. the unified focal length value of a plurality of cameras in the solving system

After obtaining all camera image sharpness functions, just can set up the multiple target valuation functions vector F (x) of system.Utilize the ideal point method in the valuation functions method, multiple objective function is optimized, seek unique extreme point, i.e. pinpointed focus value, this is the unified focal length set point of multi-cam image processing system.

In camera adjustable focal length value scope D, be divided into two kinds of situations.

The first situation, the image definition function f of M camera _i(x) just in time have identical extreme point, i.e. x ₁*=x ₂*=...=x _N*, x here _NCorresponding focal length value when * corresponding N camera image sharpness function got extreme value, for x ∈ D, its image definition function has

$f_i\left(x_i^{*}\right)\≥f_i\left(x\right),i=\mathrm{0,1},\·\·\·,N$

Formula (1)

And simultaneously, this extreme point also just in time drops in the focal range of system's permission, and the corresponding focal length value of this extreme point is system's pinpointed focus value so, as shown in Figure 2.

The second situation, the image definition function f of M camera _i(x) have different extreme points, and at x ₁*, x ₂* ..., x _NWhen * not exclusively equating, the ideal point F* of objective function F (x) not necessarily drops in the image set F (D), as shown in Figure 3, introduces the p norm this moment in object space Rm

$u\left(F\right)={\left|\right|F-F^{*}\left|\right|}_p={\left[\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{(f_i-f_i^{*})}^p\right]}^{\frac{1}{p}}$

Formula (2)

As evaluation function, wherein 1≤p＜+∞.

The concrete steps of the unified focal length scaling method of the multi-cam image processing system that the present invention realizes as shown in Figure 4.

In the multi-cam image was processed, M camera gathered respectively image information separately, each camera corresponding N two field picture under the different focal value that needs to sample, and find the solution the image definition of each two field picture.

Adopting average gray gradient operator method here, mainly is because it calculates the quality that simply and has more intuitively reflected at the different focal hypograph.

$\begin{array}{c}\mathrm{GMG}=\\ \frac{\underset{i=1}{\overset{M-1}{\mathrm{\Σ}}}\underset{j=1}{\overset{N-1}{\mathrm{\Σ}}}\sqrt{\frac{{[g(i+1,j)-g(i,j)]}^2+{[g(i,j+1)-g(i,j)]}^2}{2}}}{(M-1)(N-1)}\end{array}$ Formula (3)

As shown in formula (3), be averaging by the mean square deviation to gray value between cumulative image pixel and the neighbor, can obtain the image definition value of current frame image.Utilize between the average gray gradient operator value of every two field picture and the focal length value one to one relation, set up the image definition of single camera and the graph of relation of focal length, g (i.j) represents (i.j) individual grey scale pixel value, and i.j represents respectively level, the vertical arrangement sequence number of pixel.As shown in Figure 1.

After obtaining single camera image articulation curve, the image definition curve of M camera can be put together, can intuitively and clearly see their articulation curve of different cameras and not exclusively coincidence, if set according to pinpointed focus value separately, the multiple image that obtains at synchronization of system must have the depth of field and visual field poor so, if do not carry out special processing, must make troubles for follow-up image work for the treatment of.

Behind the image definition curve that has obtained single shooting picture head, need to adopt the data fitting method such as least square and so on to provide its image definition y _iAnalytical function y with focus value x _i=f _i(x), as can be seen from Figure 1, single camera is a quadratic function adopting average gray gradient operator method to obtain the image definition curve

f _i(x)=Ax ²+Bx+C

Formula (4)

Here, A, B, C are the coefficients that data fitting is tried to achieve.

After obtaining the camera image sharpness function, just can adopt the method for optimization to provide extreme value

Here, be a quadratic function owing to what obtain, so adopt the simplest derivative method to get final product.

The image definition function of M camera can form a Multi Goal Opinion Function vector F (x)

F(x)=(f ₁(x),f ₂(x),...,f _N(x))

Formula (5)

For this target function, adopt in the optimal method and find the solution its unique optimization extreme point for the valuation functions method of multiple-objection optimization, here, employing be ideal point method.

Setting a focal length value the most desirable is x,

Be function f _i(x) maximum then utilizes beeline ideal point method (being norm p=2) that multiple objective function is found the solution, and finds out its maximum.

$\underset{x\∈D}{\max }\sqrt{\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{[f_i\left(x\right)-f_i^{*}]}^2}$

Formula (6)

Wherein, x is the focal length value of camera, and its size should be limited in the camera adjustable focal length value scope D.

Utilize formula (6) to obtain unique extreme point F of target function ^*With and corresponding x ^*, this x ^*It is exactly the unified focal length value of multi-cam image processing system.

By above-mentioned method, all cameras in the multi-cam image processing system can be set a unified focal length value, thereby make all view data of systematic sampling have the consistent depth of field and visual field, thereby for ensuing image processing lays a good foundation, reduce the difficulty of subsequent treatment, improve the image treatment effeciency, reduce system power dissipation.

Claims (6)

1. a camera that is applied to the multi-cam Vision information processing is unified the focal length scaling method, it is characterized in that, comprises the steps:

1) finds the solution the image definition of single camera: utilize the evaluation function of image definition to find the solution the image definition of each two field picture of camera;

2) find the solution relation curve between single camera image definition and the focal length: by the image definition analysis under the different focal being obtained the relation curve of image definition and camera focal length, by the figure line match, find out the functional relation of image definition and focal length;

3) find the solution the extreme value of single camera image sharpness function: for each camera, can according to acquired relation curve, adopt data fitting method to provide its image definition y _iAnalytical function y with focus value x _i=f _i(x), then adopt the method for optimizing to provide extreme value

With its corresponding focal length

4) the unified focal length value of a plurality of cameras in the solving system: after obtaining all camera image sharpness functions, just can set up the multiple target valuation functions vector F (x) of system, utilize the ideal point method in the valuation functions method, multiple objective function is optimized, seek unique extreme point, be the pinpointed focus value, this is the unified focal length set point of multi-cam image processing system.

2. the camera that is applied to the multi-cam Vision information processing as claimed in claim 1 is unified the focal length scaling method, it is characterized in that, the unified focal length value of a plurality of cameras is divided into two kinds of situations in the solving system in camera adjustable focal length value scope D:

The first situation, the image definition function f of M camera _i(x) just in time have identical extreme point, i.e. x ₁*=x ₂*=...=x _N*, x here _NCorresponding focal length value when * corresponding N camera image sharpness function got extreme value, for x ∈ D, its image definition function has

$f_i\left(x_i^{*}\right)\≥f_i\left(x\right),i=\mathrm{0,1},\·\·\·,N$

Formula (1)

And simultaneously, this extreme point also just in time drops in the focal range of system's permission, and the corresponding focal length value of this extreme point is system's pinpointed focus value so;

The second situation, the image definition function of M camera has different extreme points, and at x ₁*, x ₂* ..., x _NWhen * not exclusively equating, the ideal point F* of objective function F (x) not necessarily drops in the image set F (D), introduces the p norm this moment in object space Rm

$u\left(F\right)={\left|\right|F-F^{*}\left|\right|}_p={\left[\underset{i=1}{\overset{N}{\mathrm{\Σ}}}{(f_i-f_i^{*})}^p\right]}^{\frac{1}{p}}$

Formula (2)

As evaluation function, wherein 1≤p＜+∞.

3. the camera that is applied to the multi-cam Vision information processing as claimed in claim 1 is unified the focal length scaling method, it is characterized in that, utilizing the evaluation function of image definition to find the solution the image definition of each two field picture of camera, is to adopt average gray gradient operator method:

$\begin{array}{c}\mathrm{GMG}=\\ \frac{\underset{i=1}{\overset{M-1}{\mathrm{\Σ}}}\underset{j=1}{\overset{N-1}{\mathrm{\Σ}}}\sqrt{\frac{{[g(i+1,j)-g(i,j)]}^2+{[g(i,j+1)-g(i,j)]}^2}{2}}}{(M-1)(N-1)}\end{array}$ Formula (3)

As shown in formula (3), be averaging by the mean square deviation to gray value between cumulative image pixel and the neighbor, can obtain the image definition value of current frame image, utilize between the average gray gradient operator value of every two field picture and the focal length value one to one relation, set up the image definition of single camera and the graph of relation of focal length, g (i.j) represents (i.j) individual grey scale pixel value, and i.j represents respectively level, the vertical arrangement sequence number of pixel.

4. the camera that is applied to the multi-cam Vision information processing as claimed in claim 1 is unified the focal length scaling method, it is characterized in that, behind the image definition curve that has obtained single shooting picture head, need to adopt the data fitting method of least square to provide its image definition y _iAnalytical function y with focus value x _i=f _i(x), to adopt average gray gradient operator method to obtain the image definition curve be a quadratic function to single camera again:

f _i(x)=Ax ²+Bx+C

Formula (4)

Here, A, B, C are the coefficients that data fitting is tried to achieve.

5. the camera that is applied to the multi-cam Vision information processing as claimed in claim 1 is unified the focal length scaling method, it is characterized in that, adopts the method for optimizing to provide extreme value

With its corresponding focal length

Specifically adopt the simplest derivative method.

6. the camera that is applied to the multi-cam Vision information processing as claimed in claim 1 is unified the focal length scaling method, it is characterized in that step 4) further is specially: the image definition function of M camera can form a Multi Goal Opinion Function vector F (x)

F (x)=(f ₁(x), f ₂(x) ..., f _N(x)) formula (5)

For this target function, adopt in the optimal method and find the solution its unique optimization extreme point for the valuation functions method of multiple-objection optimization, here, employing be ideal point method:

Setting a focal length value the most desirable is x,

Be function f _i(x) maximum is then utilized the beeline ideal point method, and namely norm p=2 finds the solution multiple objective function, finds out its maximum:

$\underset{x\∈D}{\max }\sqrt{\underset{i=1}{\overset{m}{\mathrm{\Σ}}}{[f_i\left(x\right)-f_i^{*}]}^2}$

Formula (6)

Wherein, x is the focal length value of camera, and its size should be limited in the camera adjustable focal length value scope D, utilizes formula (6) to obtain unique extreme point F of target function ^*With and corresponding x ^*, this x ^*It is exactly the unified focal length value of multi-cam image processing system.

Images