CN106303275A - A kind of virtual reality three dimensional data collection method based on single light-field camera multi-focus - Google Patents

Abstract

A kind of virtual reality three dimensional data collection method based on single light-field camera multi-focus, comprises step: S101, utilizes single light-field camera to obtain lenticule calibration figure, and lenticule center, location also shoots object light field image；S102, obtains all multi-view image by multiple perspective principle；S103, sets image distance, obtains the image of a depth of focus by multi-focus principle；S104, resets image distance, carries out step S103, obtains the image at another depth of focus.

Description

A kind of virtual reality three dimensional data collection method based on single light-field camera multi-focus

Technical field

The invention belongs to three-dimensional reconstruction field, particularly to a kind of based on single light-field camera multi-focus virtual existing Real three dimensional data collection method.

Background technology

In prior art, the acquiring technology for the three-dimensional data of testee virtual reality is scattered in various document, But all it is respectively arranged with its shortcoming.Such as there is a list of references:

[1]Gonzalez R C,Woods R E.Digital image processing[J].Nueva Jersey, 2008.

[2]Levoy M,Hanrahan P.Light field rendering[C]//Proceedings of the 23rd annual conference on Computer graphics and interactive techniques.ACM, 1996:31-42.

Summary of the invention

It is an object of the invention to provide a kind of virtual reality three dimensional data collection side based on single light-field camera multi-focus Method.

The technical scheme is that, a kind of virtual reality three dimensional data collection side based on single light-field camera multi-focus Method, comprises the steps of

S101, utilizes single light-field camera to obtain lenticule calibration figure, and lenticule center, location also shoots object light field figure Picture；

S102, obtains all multi-view image by multiple perspective principle；

S103, sets image distance, obtains the image of a depth of focus by multi-focus principle；

S104, resets image distance, carries out step S103, obtains the image at another depth of focus.

For step S101, method is to place one piece of blank at a distance in light-field camera, aperture is adjusted to minimum, adjusts exposure Time makes image be unlikely to overexposure or under-exposure, and the photo now taken can be regarded as the luminescence immediately below by lenticule Point composition, calculates lenticular centre coordinate whereby,

The light field image of acquisition is done cutting, after the pixel of first, the given image upper left corner, according to initial setting (Δ x, Δ y) go laterally to search the lenticule center that the right is nearest, during the centre of location between lenticule lenticule pitch Away from real-time update, to reduce cumulative errors, it may be assumed that

Δ x=x_i-x_i-1 (1)

Δ y=y_i-y_i-1 (2)

When calculating lenticule center, using centroid method to calculate the coordinate at center, step is as follows:

First finding local bright spot by the method gone through all over pixel, centered by the bright spot of local, outrigger n × n is adjacent Territory, then the centre coordinate (X, Y) of bright spot is obtained with centroid method weighting, its formula can be write as:

$X=\frac{{\mathrm{\Σ}}_{i,j}{x}_{i,j}*v_{i,j}}{{\mathrm{\Σ}}_{i,j}{v}_{i,j}}---\left(3\right)$

$Y=\frac{{\mathrm{\Σ}}_{i,j}{y}_{i,j}*v_{i,j}}{{\mathrm{\Σ}}_{i,j}{v}_{i,j}}---\left(4\right)$

Wherein x_{I, j},y_{I, j}It is neighborhood internal coordinate (i, j) abscissa value at place and ordinate value, v respectively_i,jIt is (x_{I, j},y_i,j) The pixel value at place,

In the range of the required object carrying out multi-focus process is placed on camera lens so that background is to shooting object post processing Impact be down to minimum, adjust camera lens focal length so that the object front and back position being taken is positioned at lens focus plane both sides, adjust Whole time of exposure, shoots an image.

For step S102, according to multiple perspective principle, each lenticule self record positional information of light, and micro- Under lens, all pixels have recorded directive same position but the different light of angle, and process includes:

Take out and under each lenticule, represent certain pixel unidirectional, and these pixels are pieced together piece image, this It is the image at some visual angle, when the one other pixel again taken under lenticule, a different visual angles can be formed again Image.

For step S103, set image distance, obtain the image of a depth of focus by multi-focus principle,

If the distance of the positional distance main lens plane at lenticule plane place is F, it is desirable to the plane of focusing, i.e. refocusing The distance of plan range main lens is α F.Draw all light of the point that coordinate in refocusing plane is x ',

These light can with its with main lens and the intersecting point coordinate of microlens array (u, x) represents, therefore every light A point can be expressed as in u-x plane,

The quantity of light is limited, equal to main lens by discrete number.The relation of x ' and u, x can be write as:

$x^{\′}=u\·(1-\frac{1}{\mathrm{\α}})+\frac{x}{\mathrm{\α}}---\left(5\right)$

It can be seen that determine that rear u and x is linear at x ', therefore all light can fall at straight line in u-x plane On, when x ' changes, straight line can do corresponding translation along the x-axis direction.

The picture of the different visual angles obtained in step S102, its coordinate (s, t) just the u coordinate in (5) formula, and x coordinate Then can be determined by the absolute position of pixel, finding all light to sum up by the rule of (5) formula, whole process can be used down Formula represents:

Σ^{S, t}Perspec (s, t, x+ (α-1) s, y+ (α-1) t) (6).

For step S104,

Object distance is calculated by known image distance and main lens focal length and obtains, and is an amount with the uneven change of α value, corresponding Certain α, object distance can write according to geometric optics lens imaging formula:

The present invention can obtain the multiple depth of focus of object by the once shooting to scenery of the single light-field camera through algorithm process Place image, for certain specific depth of focus, be distributed in object thereon will obtain clearly as, remaining object will form mould The picture stuck with paste.The image of the multiple depths of focus obtained can be used by follow-up light field one camera algorithm, can also need not for some With the place of depth of focus image, as provided the depth of field data of input content for VR technology, improve the third dimension of VR output, feeling of immersion Deng.

Accompanying drawing explanation

Fig. 1 mono-light-field camera three-dimensional imaging schematic diagram

The calibration figure edge schematic diagram that Fig. 2 should cast out because of of poor quality

Fig. 3. calibration figure non-edge schematic diagram

Fig. 4. monochromatic light field camera calibrated figure

Fig. 5. centroid method neighborhood schematic diagram

Fig. 6. refocusing schematic diagram

Light is represented in Fig. 7 u-x plane

Fig. 8. refocusing effect schematic diagram near field in embodiment

Fig. 9. far field refocusing effect schematic diagram

The method flow schematic diagram of Figure 10 present invention

Detailed description of the invention

The present invention relates to a kind of refocusing method based on single light-field camera, being added by single shot object can be in advance The calibration image obtained, through algorithm process, can form the image at different depth of focus.Comprise the steps of

1) lenticule calibration figure obtains and lenticule centralized positioning

Owing to algorithm needs, carrying out all light field image post processings when, all can use lenticular coordinate information, And these coordinates can change over because of some external factor.Therefore before one group of light field photo of shooting, be required for carrying out once Calibration.Method is to place one piece of blank at a distance in light-field camera, aperture is adjusted to minimum, adjusts time of exposure and make image unlikely In overexposure or under-exposure.The luminous point that the photo now taken can be regarded as immediately below by lenticule forms.Can be whereby Calculate lenticular centre coordinate.

When calculating lenticule center, due to second-rate (see the Fig. 2) of image border, need original light field figure As doing certain cutting, cutting thickness is about 100 pixels.

After the pixel of first, the given image upper left corner, it is possible to according to initial lenticule pitch (Δ x, the Δ set Y) go laterally to search the lenticule center that the right is nearest.Lenticule pitch real-time update during the centre of location, to reduce Cumulative errors.That is:

Δx=x_i-x_i-1 (1)

Δ y=y_i-y_i-1 (2)

When calculating lenticule center, using centroid method to calculate the coordinate [1] at center, step is as follows: first with going through time The method of pixel finds local bright spot, outrigger 3*3 neighborhood centered by the bright spot of local, (if central point is distributed more widely, Bigger neighborhood can be used) Fig. 5 uses 3*3 neighborhood.The centre coordinate (X, Y) of bright spot is obtained again with centroid method weighting, Its formula can be write as:

$X=\frac{{\mathrm{\Σ}}_{i,j}{x}_{i,j}*v_{i,j}}{{\mathrm{\Σ}}_{i,j}{v}_{i,j}}---\left(3\right)$

$Y=\frac{{\mathrm{\Σ}}_{i,j}{y}_{i,j}*v_{i,j}}{{\mathrm{\Σ}}_{i,j}{v}_{i,j}}---\left(4\right)$

Wherein x_{I, j},y_{I, j}It is neighborhood internal coordinate (i, j) abscissa value at place and ordinate value, v respectively_{I, j}It is (x_{I, j},y_{I, j}) The pixel value at place.

2) object light field picture shooting

In the range of the required object carrying out multi-focus process is placed on camera lens so that background is to shooting object post processing Impact be down to minimum.Adjust the focal length of camera lens so that the object front and back position being taken is positioned at lens focus plane both sides, carries The high utilization rate to high-resolution areas.Adjust time of exposure, to obtain most suitable brightness.Finally, one image of shooting is i.e. Can.

3) multiple perspective picture is obtained

According to multiple perspective principle [2], each lenticule self record positional information of light, and under lenticule all Pixel have recorded directive same position but the different light of angle.The principle summarizing multiple perspective can be write: takes out every Representing certain pixel unidirectional under individual lenticule, and these pixels are pieced together piece image, this is some visual angle Image.When the one other pixel again taken under lenticule, the image of a different visual angles can be formed again.

4) set image distance, obtain the image of a depth of focus by multi-focus principle

As shown in Figure 6, if the distance of the positional distance main lens plane at lenticule plane place is F, it is desirable to putting down of focusing Face, the i.e. distance of refocusing plan range main lens are α F.Draw all light of the point that coordinate in refocusing plane is x ' Line.As shown in Figure 6, these light can with its with main lens and the intersecting point coordinate of microlens array (u x) represents.Therefore every Light can be expressed as a point in u-x plane, sees Fig. 7.The quantity of light is limited, equal to main lens by discrete part Number.The relation of x ' and u, x can be write as:

$x^{\′}=u\·(1-\frac{1}{\mathrm{\α}})+\frac{x}{\mathrm{\α}}---\left(5\right)$

It can be seen that determine that rear u and x is linear at x ', therefore all light can fall at straight line in u-x plane On.When x ' changes, straight line can do corresponding translation along the x-axis direction.

The picture of the different visual angles obtained in step (3), (s, t) just for we providing the u seat in (5) formula for its coordinate Mark, x coordinate then can be determined by the absolute position of pixel.The most just can find all light by the rule of (5) formula Summing up, whole process can represent with following formula:

Σ^{S, t}Perspec (s, t, x+ (α-1) s, y+ (α-1) t) (6)

5) reset image distance α, again carry out process 4)

Object distance can be calculated by known image distance and main lens focal length and obtain, and is an amount with the uneven change of α value. Certain α corresponding, object distance can write according to geometric optics lens imaging formula:

Effect such as Fig. 8 of final different focus plane, 9, Fig. 8 focusings are on nearest coin, and Fig. 9 focuses in farthest On notebook cover.It is true that focal plane can have numerous according to the difference of α, but light-field camera is for optical axis direction Resolution be limited, therefore it is nonsensical for taking α outside resolution capability more.Energy should be differentiated with it when choosing α interval Power matches.

Claims (5)

1. a virtual reality three dimensional data collection method based on single light-field camera multi-focus, comprises the steps of

S101, utilizes single light-field camera to obtain lenticule calibration figure, and lenticule center, location also shoots object light field image；

S102, obtains all multi-view image by multiple perspective principle；

S103, sets image distance, obtains the image of a depth of focus by multi-focus principle；

S104, resets image distance, carries out step S103, obtains the image at another depth of focus.

2. virtual reality three dimensional data collection method based on single light-field camera multi-focus as claimed in claim 1, it is special Levying and be, for step S101, method is to place one piece of blank at a distance in light-field camera, aperture is adjusted to minimum, adjusts exposure Time makes image be unlikely to overexposure or under-exposure, and the photo now taken can be regarded as the luminescence immediately below by lenticule Point composition, calculates lenticular centre coordinate whereby,

The light field image of acquisition is done cutting, after the pixel of first, the given image upper left corner, micro-according to initially set (Δ x, Δ y) go laterally to search the lenticule center that the right is nearest mirror spacing, and during the centre of location, lenticule pitch is real Shi Gengxin, to reduce cumulative errors, it may be assumed that

Δ x=x_i-x_i-1 (1)

Δ y=y_i-y_i-1 (2)

When calculating lenticule center, using centroid method to calculate the coordinate at center, step is as follows:

First local bright spot is found by the method gone through all over pixel, outrigger n × n neighborhood centered by the bright spot of local, then Obtain the centre coordinate (X, Y) of bright spot with centroid method weighting, its formula can be write as:

$X=\frac{{\mathrm{\Σ}}_{i,j}{x}_{i,j}*v_{i,j}}{{\mathrm{\Σ}}_{i,j}{v}_{i,j}}---\left(3\right)$

$Y=\frac{{\mathrm{\Σ}}_{i,j}{y}_{i,j}*v_{i,j}}{{\mathrm{\Σ}}_{i,j}{v}_{i,j}}---\left(4\right)$

Wherein x_{I, j},y_{I, j}It is neighborhood internal coordinate (i, j) abscissa value at place and ordinate value, v respectively_{I, j}It is (x_{I, j},y_{I, j}) place Pixel value,

In the range of the required object carrying out multi-focus process is placed on camera lens so that the background shadow to shooting object post processing Sound is down to minimum, adjusts the focal length of camera lens so that the object front and back position being taken is positioned at lens focus plane both sides, adjusts and exposes The light time, shoot an image.

3. virtual reality three dimensional data collection method based on single light-field camera multi-focus as claimed in claim 1, it is special Levy and be, for step S102, according to multiple perspective principle, each lenticule self record positional information of light, and micro- Under lens, all pixels have recorded directive same position but the different light of angle, and process includes:

Taking out and represent certain pixel unidirectional under each lenticule, and these pixels are pieced together piece image, this is The image at some visual angle, when the one other pixel again taken under lenticule, can form again the image of a different visual angles.

4. virtual reality three dimensional data collection method based on single light-field camera multi-focus as claimed in claim 1, it is special Levy and be, for step S103, set image distance, obtain the image of a depth of focus by multi-focus principle,

If the distance of the positional distance main lens plane at lenticule plane place is F, it is desirable to the plane of focusing, i.e. refocusing plane The distance of distance main lens is α F.Draw all light of the point that coordinate in refocusing plane is x ',

These light can with its with main lens and the intersecting point coordinate of microlens array (u, x) represents, therefore every light is permissible A point it is expressed as in u-x plane,

The quantity of light is limited, equal to main lens by discrete number.The relation of x ' and u, x can be write as:

$x^{\′}=u\·(1-\frac{1}{\mathrm{\α}})+\frac{x}{\mathrm{\α}}---\left(5\right)$

It can be seen that determine that rear u and x is linear at x ', therefore all light can fall point-blank in u-x plane, When x ' changes, straight line can do corresponding translation along the x-axis direction.

The picture of the different visual angles obtained in step S102, its coordinate (s, t) just the u coordinate in (5) formula, then may be used by x coordinate Being determined by the absolute position of pixel, finding all light to sum up by the rule of (5) formula, whole process can use following formula table Show:

∑^{S, t}Perspec (s, t, x+ (α-1) s, y+ (α-1) t) (6).

5. virtual reality three dimensional data collection method based on single light-field camera multi-focus as claimed in claim 1, it is special Levy and be, for step S104,

Object distance is calculated by known image distance and main lens focal length and obtains, and is an amount with the uneven change of α value, correspondence certain α, object distance can write according to geometric optics lens imaging formula: