CN102831637A - Three-dimensional reconstruction method based on mobile device - Google Patents

Abstract

The invention discloses a three-dimensional reconstruction method based on a mobile device. An three-dimensional reconstruction is applied to a cell phone mobile device with an Android system by adopting the luminosity three-dimensional technology; a screen with high brightness is converted into light sources of multiple different positions by adopting a programming mode; an object is shot by a camera of a cell phone in a dark environment so as to obtain images at each light source; subsequently, multiple images are analyzed and calculated through a luminosity three-dimensional algorithm so as to obtain depth value information of an object surface; and finally a model corresponding to the object is established according to the depth value information. The three-dimensional reconstruction method is applied to cell phone mobile devices and is a creation in three-dimensional reconstruction direction; establishing object models by adopting such mode has the advantages of low cost, rapid calculation speed, clear model and the like.

Description

Three-dimensional rebuilding method based on mobile device

Technical field

The present invention is applied on the mobile device, relates to a kind of three-dimensional rebuilding method.

Background technology

Three-dimensional reconstruction is an important directions of computer vision field as the main target of human vision research.It mainly is to adopt certain technological means to realize the shape modeling to three-dimensional body.Along with the promotion of the theory of vision computing of Marr, three-dimensional reconstruction has obtained breakthrough progress, has emerged many substantial algorithms and reconstruction mode, for example directly the method for measuring distance, rebuild the method etc. of the degree of depth indirectly.Wherein based on the three-dimensional reconstruction of two dimensional image; Because characteristics such as low cost, modeling is convenient, drafting is quick; Extensively praised highly use by people, wherein the most representative three kinds of methods are: based on the silhouettes line rebuild geometric model, adopt the stereoscopic vision method rebuild geometric model, based on the modeling method of zone coupling.

All the time, the technology of utilizing low cost equipment to obtain the 3D model of real-world object automatically all has very important meaning in practical application area.Nowadays the fast development of mobile phone mobile device is that hardware or software have all had qualitative leap, and this also makes three-dimensional reconstruction become a kind of possibility above being applied to.

Summary of the invention

The present invention has provided a kind of three-dimensional rebuilding method based on mobile device, is directed to the mobile phone mobile device with Android operating system.

Should may further comprise the steps based on the three-dimensional rebuilding method of mobile device:

The first step: the background of choosing light source, mobile phone screen is set to white according to the order of upper and lower, left and right, and other three positions corresponding when a position is set to white are black;

Second step: the N pictures { I of order toggle lights, shot object _j: j=1...N}, and each pictures all is at different lighting environment L _jUnder take, each pixel adopts following formula to confirm show white or black under the situation of the j pictures of current shooting on the screen so: $L_j(x,y)=\mathrm{Sign}(x\·\mathrm{Cos}\left(\frac{2\mathrm{\π\; j}}{N}\right)+y\·\mathrm{Sin}\left(\frac{2\mathrm{\π\; j}}{N}\right))$

Wherein sign () value is-1 interval scale black, be 1 interval scale white, and the center of screen is (0; 0), coordinate x, y arranges based on this; Confirm that under the situation of taking first pictures, the left-half background of screen is a white, the right half part background is a black;

The 3rd step: adopt Lambertian scattering model computing object surface normal value;

3.1 specifying the resolution of every width of cloth image is M * N pixel, and brightness value is expressed as

I=(I ₁, I ₂, I ₃), the light source direction that every width of cloth image is corresponding is:

S ₁＝(s ₁₁,s ₁₂,s ₁₃)?S ₂＝(s ₂₁,s ₂₂,s ₂₃)?S ₃＝(s ₃₁，s ₃₂,s ₃₃)

Adopt the form of matrix to represent, definition $S=\left[\begin{array}{ccc}{s}_{11}& s_{12}& s_{13}\\ s_{21}& s_{22}& s_{23}\\ s_{31}& s_{32}& s_{33}\end{array}\right]$

3.2 the brightness value to image carries out normalization, order $I(i,j)=\frac{I(i,j)}{\underset{1\≤i\≤M,1\≤j\≤M}{\mathrm{Max}}\left\{I(i,j)\right\}},$

Wherein (i=1,2 ... M, j=1,2 ... N);

3.3 order $\hat{N}(i,j)=\mathrm{\η\; N}(i,j)=S^{-1}I,$ Wherein (i=1,2 ... M, j=1,2 ... N);

The 4th step: computing object case depth value, calculate corresponding depth value according to the 3rd body surface normal direction value that obtain of step;

4.1 each point on the body surface adopts the mode of three angle sections to make up, and sets variable T _kRepresent k tri patch;

4.2 at tri patch T _kIn, three summits that constitute it are respectively P ₀=(x ₀y ₀z ₀) P ₁=(x ₁y ₁z ₁) P ₂=(x ₂y ₂z ₂) and the corresponding normal direction value in each summit be N ₁, N ₂, N ₃

4.3 the relation that exists between the position according to three summits of the normal direction value of each tri patch and it self is set up corresponding system of equations: $\left\{\begin{array}{c}{\mathrm{AN}}_x=(y_1-y_0)(z_2-z_0)-(z_1-z_0)(y_2-y_0)\\ {\mathrm{AN}}_y=(z_1-z_0)(x_2-x_0)-(x_1-x_0)(z_2-z_0)\\ {\mathrm{AN}}_z=(x_1-x_0)(y_2-y_0)-(y_1-y_0)(x_2-x_0)\end{array}\right.$

[N wherein _xN _yN _z]=(N ₁+ 2N ₂+ N ₃)/4;

4.4 above-mentioned formula is converted into matrix representation, use matrix A to represent the coefficient of depth value z, use matrix B to represent the normal direction value and the coefficient value on the whole system of equations left side, the matrix equation group of formation is Z=(A ^TA) ^-1(A ^TB);

4.5 the system of linear equations that adopts least square method to find the solution this overdetermination obtains final depth value.

The 5th step: adopt OpenGL ES shape library to draw out the model of object.

5.1 draw out number is the point of M * N, is the image of M * N corresponding to pixel;

Draw out a plane 5.2 adopt the GL10.GL_TRIANGLES instrument;

5.3 wherein a picture shot is attached to this above plane as texture;

5.4 based on the object surface depth value that the 4th step obtained, the point of correspondence is adjusted on reference axis Z, is formed final threedimensional model.

Beneficial effect of the present invention:

1. through the screen of mobile phone mobile device is divided, create out a plurality of light sources that satisfy condition;

2. adopt the photometric stereo technology, obtain the depth value information of body surface;

3. through drawing, obtain the corresponding 3D model of final object;

Description of drawings

Fig. 1 divides the mode that screen forms four Different Light;

The triangulation mode that adopts during Fig. 2 gauging surface depth value.

Embodiment

Below in conjunction with accompanying drawing the present invention is done further introduction.

The present invention has provided three-dimensional reconstruction has been applied to the method on the mobile phone mobile device, requires the condition that possesses:

1. dark environment;

2. the mobile phone mobile device that has Android operating system;

3. has preposition camera;

The flow process of entire method relates to theoretical algorithm and practical application, is divided into three parts: image taking, three-dimensional reconstruction algorithm, modeling rendering.

One, image taking

Choosing of light source: the present invention programmes on the Android of mobile phone mobile device operating system, through the brightness demonstration of control screen, reaches the purpose of a plurality of light sources.Concrete way is: the background of mobile phone screen is set to white according to the order of upper and lower, left and right, and corresponding other three positions are black, and when for example the left side was white, the background on top, bottom and the right then was a black.Like this, when maximum is transferred in the brightness of screen, background will provide levelling for the part of white, and the black part does not provide, and just can create out four light sources that the position is different, satisfies the needed shooting condition of photometric stereo algorithm.

The order of light source is switched:

We need the N pictures { I of shot object _j: j=1...N}, and each pictures all is at different lighting environment L _jTake down, each pixel adopts following formula to confirm show white or black under the situation of the j pictures of current shooting on the screen so: $L_j(x,y)=\mathrm{Sign}(x\·\mathrm{Cos}\left(\frac{2\mathrm{\π\; j}}{N}\right)+y\·\mathrm{Sin}\left(\frac{2\mathrm{\π\; j}}{N}\right))$

-1 represents black, and 1 represents white, and the center of screen is (0,0), coordinate x, and y arranges according to this.For example when taking first pictures, x and y are positioned at the pixel on the screen left side, owing to return positive number, so the sign method can return 1, are white; And the pixel on the right is a negative owing to what return, so the sign method returns-1, is black.Through like this, just can confirm that under the situation of taking first pictures, the left-half background of screen is a white, the right half part background is a black.

Photographing program is realized:

Realize the function of the image of shot object under Different Light through the Camera interface of Android operating system:

Create light source:

Create up, down, left, four view of right,, on the layout of current background, be provided with simultaneously, they are presented on the screen position of appointment, for example the setting of up position as the screen portions of four appointments:

For each view,, in needs, be set to white through the method for correspondence:

left.setBackgroundColor(Color.WHITE)

Successively four view are set to white background, afterwards screen intensity are adjusted to the highlyest, just can serve as four different light sources with screen.

Realize shoot function:

At first create the Camera object, and the picture that camera is captured is as prospect preview:

mCamera=Camera.open(1)

mCamera.startPreview()

Follow the touch event of calling mobile phone screen, catch operation in real time screen:

onTouchEvent(MotionEvent?event)

Here, our appointment is when mobile phone screen is sensed pressing of finger, promptly to carry out the shooting to object:

case?MotionEvent.ACTION_DOWN

mCamera.takePicture(null,null,new?PictureCallbackListener())

Wherein the parameter PictureCallbackListener () that calls of takePicture aspect is the callback method of camera photographic images, through this method, can get access to camera and take the image that gets off, and this is handled.At last, we realize the concrete operations of PictureCallbackListener () callback method:

Transform into the Bitmap form through byte arrays, promptly become and take the subject image of taking memory image.

Two, three-dimensional reconstruction algorithm

The calculating of body surface normal direction value:

Adopt the Lambertian scattering model, concrete steps are following:

One, specifying the resolution of every width of cloth image is M * N pixel, and brightness value is expressed as

I=(I ₁, I ₂, I ₃), the light source direction that every width of cloth image is corresponding is:

S ₁＝(s ₁₁,s ₁₂,s ₁₃)?S ₂＝(s ₂₁,s ₂₂,s ₂₃)?S ₃＝(s ₃₁,s ₃₂,s ₃₃)

Adopt the form of matrix to represent, definition $S=\left[\begin{array}{ccc}{s}_{11}& s_{12}& s_{13}\\ s_{21}& s_{22}& s_{23}\\ s_{31}& s_{32}& s_{33}\end{array}\right]$

Two, the brightness value to image carries out normalization, order $I(i,j)=\frac{I(i,j)}{\underset{1\≤i\≤M,1\≤j\≤M}{\mathrm{Max}}\left\{I(i,j)\right\}},$

Wherein (i=1,2 ... M, j=1,2 ... N).

Three, order $\hat{N}(i,j)=\mathrm{\η\; N}(i,j)=S^{-1}I,$ Wherein (i=1,2 ... M, j=1,2 ... N).

Four, so

that is, the surface normal direction at each point value.

The calculating of object surface depth value:

Under the situation that obtains the surface normal value, calculate corresponding depth value:

One, for each point on the body surface, adopts the mode of three angle sections to make up, set variable T _kRepresent k tri patch.

Two, at tri patch T _kIn, three summits that constitute it are respectively P ₀=(x ₀y ₀z ₀) P ₁=(x ₁y ₁z ₁) P ₂=(x ₂y ₂z ₂), and the corresponding normal direction value in each summit is N ₁, N ₂, N ₃

Three, the relation that exists between the position according to three summits of the normal direction value of each tri patch and it self, set up corresponding system of equations:

$\left\{\begin{array}{c}{\mathrm{aN}}_x=(y_1-y_0)(z_2-z_0)-(z_1-z_0)(y_2-y_0)\\ {\mathrm{aN}}_y=(z_1-z_0)(x_2-x_0)-(x_1-x_0)(z_2-z_0)\\ {\mathrm{aN}}_z=(x_1-x_0)(y_2-y_0)-(y_1-y_0)(x_2-x_0)\end{array}\right.$

[N wherein _xN _yN _z]=(N ₁+ 2N ₂+ N ₃)/4

Four, above-mentioned formula is converted into matrix representation, uses matrix A to represent the coefficient of depth value z, use matrix B to represent the normal direction value and the coefficient value on the whole system of equations left side, the matrix equation group of formation is Z=(A ^TA) ^-1(A ^TB).

Five, the system of linear equations that adopts least square method to find the solution this overdetermination obtains final depth value.

Three, modeling rendering

Adopt the OpenGL ES shape library of Android operating system lift-launch, draw out the model of object, concrete steps are following:

One, draws out the point that number is M * N (is the image of M * N corresponding to pixel);

Two, call the glDrawElements method, and the mode of specify drawing is GL10.GL_TRIANGLES, draws out a plane;

Three, wherein a picture shot is attached to this above plane as texture;

Four, the object surface depth value of coming out based on algorithm computation is adjusted the point of correspondence on reference axis Z, form final threedimensional model;

Claims (4)

1. based on the three-dimensional rebuilding method of mobile device, it is characterized in that, may further comprise the steps:

The first step: the background of choosing light source, mobile phone screen is set to white according to the order of upper and lower, left and right, and other three positions corresponding when a position is set to white are black;

Second step: the N pictures { I of order toggle lights, shot object _j: j=1...N}, and each pictures all is at different lighting environment L _jUnder take, each pixel adopts following formula to confirm show white or black under the situation of the j pictures of current shooting on the screen so: $L_j(x,y)=\mathrm{Sign}(x\·\mathrm{Cos}\left(\frac{2\mathrm{\π\; j}}{N}\right)+y\·\mathrm{Sin}\left(\frac{2\mathrm{\π\; j}}{N}\right))$

Wherein sign () value is-1 interval scale black, be 1 interval scale white, and the center of screen is (0; 0), coordinate x, y arranges based on this; Confirm that under the situation of taking first pictures, the left-half background of screen is a white, the right half part background is a black;

The 3rd step: adopt Lambertian scattering model computing object surface normal value;

The 4th step: computing object case depth value, calculate corresponding depth value according to the 3rd body surface normal direction value that obtain of step;

The 5th step: adopt OpenGL ES shape library to draw out the model of object.

2. the three-dimensional rebuilding method based on mobile device as claimed in claim 1 is characterized in that, wherein computing object surface normal value adopts following method:

3.1 specifying the resolution of every width of cloth image is M * N pixel, and brightness value is expressed as

I=(I ₁, I ₂, I ₃), the light source direction that every width of cloth image is corresponding is:

S ₁＝(s ₁₁，s ₁₂,s ₁₃)?S ₂＝(s ₂₁,s ₂₂,s ₂₃)?S ₃＝(s ₃₁，s ₃₂,s ₃₃)

Adopt the form of matrix to represent, definition $S=\left[\begin{array}{ccc}{s}_{11}& s_{12}& s_{13}\\ s_{21}& s_{22}& s_{23}\\ s_{31}& s_{32}& s_{33}\end{array}\right]$

3.2 the brightness value to image carries out normalization, order $I(i,j)=\frac{I(i,j)}{\underset{1\≤i\≤M,1\≤j\≤M}{\mathrm{Max}}\left\{I(i,j)\right\}},$

I=1 wherein, 2 ... M, j=1,2 ... N;

3.3 order $\hat{N}(i,j)=\mathrm{\η\; N}(i,j)=S^{-1}I,$ I=1 wherein, 2 ... M, j=1,2 ... N.

3. according to claim 1 or claim 2 the three-dimensional rebuilding method based on mobile device is characterized in that wherein the compute depth value adopts following method:

4.1 each point on the body surface adopts the mode of three angle sections to make up, and sets variable T _kRepresent k tri patch;

4.2 at tri patch T _kIn, three summits that constitute it are respectively P ₀=(x ₀y ₀z ₀) P ₁=(x ₁y ₁z ₁) P ₂=(x ₂y ₂z ₂) and the corresponding normal direction value in each summit be N ₁, N ₂, N ₃

4.3 the relation that exists between the position according to three summits of the normal direction value of each tri patch and it self is set up corresponding system of equations: $\left\{\begin{array}{c}{\mathrm{AN}}_x=(y_1-y_0)(z_2-z_0)-(z_1-z_0)(y_2-y_0)\\ {\mathrm{AN}}_y=(z_1-z_0)(x_2-x_0)-(x_1-x_0)(z_2-z_0)\\ {\mathrm{AN}}_z=(x_1-x_0)(y_2-y_0)-(y_1-y_0)(x_2-x_0)\end{array}\right.$

[N wherein _xN _yN _z]=(N ₁+ 2N ₂+ N ₃)/4;

4.4 above-mentioned formula is converted into matrix representation, use matrix A to represent the coefficient of depth value z, use matrix B to represent the normal direction value and the coefficient value on the whole system of equations left side, the matrix equation group of formation is Z=(A ^TA) ^-1(A ^TB);

4.5 the system of linear equations that adopts least square method to find the solution this overdetermination obtains final depth value.

4. according to claim 1 or claim 2 the three-dimensional rebuilding method based on mobile device is characterized in that, wherein draws object model and adopts following method:

5.1 draw out number is the point of M * N, is the image of M * N corresponding to pixel;

Draw out a plane 5.2 adopt the GL10.GL_TRIANGLES instrument;

5.3 wherein a picture shot is attached to this above plane as texture;

5.4 based on the object surface depth value that the 4th step obtained, the point of correspondence is adjusted on reference axis Z, is formed final threedimensional model.